@Searke @Szabolcs It's fascinating to me that is a popular support article. It strikes me as so esoteric. I'm sure I would have tried pasting a plain Integrate version inside of the NMaximize before I even considered asking for help about symbolic uses of a custom definition that uses an explicitly numeric function.

Just look up NIntegrate::inumr or anything similar. It's basically crazy that the error message doesn't point to an article like this.

I think at some point there was a discussion

about having numerical functions... intelligently wait or something

if they had undefined symbols in them.

@Searke Well I certainly understand the usefulness of a message when NIntegrate receives something symbolic, and I see under "possible issues" they point out a recursive function where numeric evaluation is faster than symbolic and that you can take advantage of it with a ?NumericQ argument test. Well I guess now that I'm thinking more about it, the situation in the article doesn't seem so obscure.

Having a function defined in terms of an integration that is too complicated to be done symbolically that you then need to maximize. I guess I just wish I had a custom function that was so important to me.

One of my friends asked me for help understanding this paper today:

stat.uchicago.edu/~lekheng/meetings/mathofranking/ref/…

So I coded the results as I read it and sent him back this PDF:

onedrive.live.com/…

He seemed impressed, but then he said he really just needs to be able to do one First@Eigenvalues[m] calculation about a month from now on a 9x9 matrix. I wonder if I'll ever have some complicated traditional math expression I need to run through NMaximize and NIntegrate a bunch of times.

Although I guess neither the paper nor my coding helped me understand why the principal eigenvector is a better measure of the rankings/priorities from a comparison matrix compared to just taking the totals of each row.

It's awfully quiet...

@MichaelHale Of course, you can use the second argument of Eigenvalues[] to restrict the output to a single eigenvalue or a subset of eigenvalues.

@J.M. It's nearly always quiet nowadays

@J.M. how's it going? Started the new job yet? Since you haven't been back since before Christmas, I was worried you'd had to pawn your computer again...

Hi @Oleksandr, yeah, the new job has been splendid thus far. I was really busy so that I hadn't any time to contribute until now.

The computer's still doing fine, for a nine-year-old.

Hey @J.M., nice to hear that the job is nice!

Question to everyone. I haven't googled, I just looked through the docs and haven't found something.

When I try to show the stochastic independence of categorized variables, you have to create a crosstable (contingence-table, don't know the English word) and then you look up the ChiSquareDistribution of the summend relative differences.

Is such a test included in Mathematica? The PearsonChiSquareTest looks only like a distribution test.

@J.M. Glad to hear you like the new job :).

Nice to see you(r avatar) again!

@J.M. Ah, I figured there was such a thing but hadn't double checked.

@halirutan, admittedly, I feel out of my depth a lot of times, but, what's an interesting job that doesn't push you into unchartered territory, eh? ;)

Nice to see you too, @Jacob.

@halirutan, sounds like ANOVA…

@J.M. Probably it's best to give a small example. Here on the wiki page you have a very small example with cities that are categorised.

Using the table there, we can calculated the teststatistic of independence by

crossTable[c_] :=
 With[{f = Append[#, Total[#]] &},
  ReplacePart[Transpose[f /@ Transpose[f /@ c]],
   Dimensions[c] + 1 -> Total[Flatten[c]]]
  ]

expectedTable[mat_] := Module[{m, n, tot},
  {m, n} = Dimensions[mat];
  tot = mat[[m, n]];
  Table[mat[[i, n]]*mat[[m, j]]/tot,
   {i, m}, {j, n}
   ]
  ]

and then

m = {
  {90, 60, 104, 95},
  {30, 50, 51, 20},
  {30, 40, 45, 35}
  }

Total[(Abs[# - expectedTable[#]])^2/expectedTable[#] &[crossTable[m]],
   2] // N

This gives 24.5 which is the chi^2 of the test-statistic. We can calculate the p-value by using the dimensionality of the table (3-1)*(4-1)=6

1 - CDF[ChiSquareDistribution[6], 24.5] // N

Is there a better way than writing this down myself?

@Searke The problem with that is that as a function gets smarter, it also gets more complicated and less predictable. Doing that would help beginners and casual users, but it would be bad for the rest of us. There is value in simplicity too.

@halirutan, mathematica.stackexchange.com/q/5271

As a person proceeds with learning Mathematica (or any other language), sooner or later they will encounter counterintuitive behaviour. There's no way around it. Making functions too smart will just delay that moment of (negative) surprise, but will probably also make the surprise even worse when it comes.

(if I'm understanding you correctly; in which case, apparently nothing built-in and documented)

Welcome back @J.M.!

Hi Szabolcs, thanks! Work's been keeping me busy lately.

To offer an idle opinion: sometimes I hate it when the software's seemingly trying to be too smart for its own good; it lulls people into becoming careless.

@halirutan Is PearsonChiSquareTest not useful there?

LIGO LIGO LIGO

Yes Virginia, there really are gravitational waves. ;)

@J.M. Basically what I did but thanks for digging it out :-)

@MichaelHale Is it? I don't see it because you need to calculate the cross-table for the absolute occurences etc..

@Szabolcs Oh I agree. I'm torn about that particular idea. I think in the end it doesn't end up adding a terrible amount of complexity. I feel like that's characteristic of the language. How it gives up on some deep simplicity to give the language a kind of appearance of simplicity for casual users. Can you imagine if Mathematica's arbitrary and symbolic numbers were "chalkboard simple" instead of "easy to use". shudders

Deep simplicity is important. Most of the users of Mathematica though will never know what that means though.

@halirutan Yeah, I give up. I tried the following as a last ditch attempt, but seems you have to do it manually. Weird that with so many named statistical functions M can't do one of the ones I remember from high school.

PearsonChiSquareTest[
   Flatten@MapIndexed[Table[#2[[1]], {#}] &, Flatten@#],
   EmpiricalDistribution[
    Flatten@Outer[Times, Total[#]/Total[#, 2],
       Total[#, {2}]/Total[#, 2]] -> Range@Length@Flatten@#]] &@{{90,
   60, 104, 95}, {30, 50, 51, 20}, {30, 40, 45, 35}}

Does anyone know what's the deal with infinities passed to Mathematica through LibraryLink packed arrays? They are handled correctly! That surprised me because of this. @halirutan @OleksandrR.

Is this a MathLink-LibraryLink difference? Or is it because there are multiple possible representations for infinity as a C double on x86? Maybe what I get through MATLink from MATLAB is not the same as what I get from igraph through LibraryLink.

@Szabolcs The difference is probably historically driven as MathLink is much older than LibraryLink. I guess they had to deal with that from the very beginning and were forced to support the early decisions.

Other than that, I cannot say much about this topic as I haven't investigated in this with great detail. I knew already about the discussion you linked as I have read it the time you posted this. Even then, I had the feeling that the topic is quite hard if you try to solve it for all systems (and probably versions).

@MichaelHale I was surprised too. I understood and recalculated some of the statistical tests on paper when I had to use them. Since now, I found everything reflected in some built-in function.

It's not that I need that! I just wondered whether I miss an obvious function.

@Szabolcs I'm a bit surprised as well since LibraryLink has to serve for Compile too. If an infinity is produced there, it reevaluates at the top level. But if it returns correctly, this shouldn't be necessary. Maybe it only does it because machine infinity may not really be infinite, but just come from a sufficiently large number.

@OleksandrR. Exactly my question. I tested some stuff withCompile and Log[0], Log[0.0] and 1/0 and all I got was that it's evaluated on the Kernel again.

I was too lazy to try it in a real lib-function.

@Searke that sounds unusual, because we don't have a policy of deleting duplicates. Questions that are closed and have no upvotes and few comments will be deleted after 30 days, if I recall rightly. Could it have been one like that?

@halirutan @MichaelHale :) PearsonChiSquareTest doesn't do what people think it should do. PearsonChiSquareTest is supposed to be designed to do what you should think it should do.

@OleksandrR. Most likely, I was just being incompetent and couldn't find it.

I do agree with Leonid. This site isn't ideal for giving a tutorial. That's better done in a classroom setting. I've usually found that asking questions (of others) as a beginner isn't very helpful because it just substitutes someone else's experience for your own without the necessary context. Asking questions of oneself is how the real progress happens.

@OleksandrR. This site doesn't work for me because helping people usually requires a conversation. Usually the conversation is helping them understand what they're asking and then helping them understand how they can learn how to answer that question.

Stackexchange is great, but it tries to shoe-horn in those kinds of dialogs and things like community wikis into the wrong kind of format. Only real experts are capable of asking questions that are capable of being answered.

@Searke I totally agree with you that the site isn't good for this. My point is that I think that's alright, because there are other, better approaches for helping people who need that kind of support than having them ask naive questions on online forums. A question and answer site can never hope to serve people who don't properly know what their question is.

Few people self-study any more. Few read around a subject so that they can know what concepts are involved and what vocabulary is needed to describe them. Everyone wants quick results with minimal independent effort. But unfortunately that's just not the way the world works. People at that stage of their learning should be in a classroom or library, in my opinion.

@Searke I understand that for tech support it's a special problem because you have an obligation to help people who don't have access to libraries or classrooms (in industry, for example)--they just have the product. In this sense I think the video tutorials are a good resource. Do you find many people who ask support questions have done the tutorials, or not?