@Pickett Maybe Association should be made Flat then. (@TaliesinBeynon) I might be missing something though.

Consider asc = <|"a" -> <|"a" -> 1, "b" -> 2|>, "b" -> <|"a" -> 3, "b" -> 4|>|>. Then Total[asc["a"]] is 3, Total[asc] is <|"a" -> 4, "b" -> 6|> and Total@Total[asc] is 10.

Wouldn't it be reasonable for Total[asc, 2] to also yield 10 then?

@Szabolcs I think you are right about both Flat and Total[asc,2]

This is sort of controversial too, although I guess every map service has had to make a choice about it:

Although the politically correct solution appears to be to mark it as its own "disputed" territory.

Can someone tell me when exactly the completion of special arguments should pop-up? Example: TextCell[string, style] should automatically suggest the different style types. Let's say I start typing TextCell["blub", does the pop-up appear? Because on Linux it doesn't and in general this works unpredictable.

@halirutan I have to type the first two letters of the style. TextCell["blub","Ti and then "Title" appears in the auto-completion list.

Any ideas?

Well, now it works but I'm still interested in ideas because this intermittency is not new.

@MichaelHale OK, this works here too. I have to investigate more. Thanks for trying.

Am I the only dumbass who struggles with procedural programming in MMA? I'm working the procedural programming chapter in an MMA book and I just can't seem to begin how to solve a problem procedurally. Even when I do, I lose track of my iterators all the time and my evaluation hangs so often, probably due to some infinite recursion.

This is so frustrating. I breezed through the functional programming chapter but procedural programming just seems very intimidating to me. It's kinda makes me dislike MMA a little.

@seismatica I can assure you that you are not alone.

I've spoken to several people who have found Mathematica off-putting. But I'm not sure about the procedural part.

@bobthechemist You were asking about the missing functions in the Raspberry Pi release: mathematica.stackexchange.com/questions/57342/…

Hmmh: Out[1]= 10.0 for Linux ARM (32-bit) (August 4, 2014)

But where's that 10.0.1 or such?

that Linux ARM release was just to catch it up to the released version of Mathematica 10

Don't worry, we still have the same amount of bugs, the just show up more slowly.

Hi @MrWizard, are you around?

Is there anyone here who uses MATLink on OS X earlier than 10.9 and could run a few tests for me?

If yes, please drop me a mail.

My address is in my profile.

@halirutan are you around?

@brama Depends on what happens when I say Yes.

@halirutan haha....you shouldn't be worried at all..All I have are simple stuff...

@halirutan you referred me to this some time ago mathematica.stackexchange.com/questions/7017/… and I am trying to understand how I can use that knowledge for my application

@brama Hmm, so no girls... damn.

@halirutan So your wife is surely not on SE!!

@brama No, currently she is in the cinema. But she checks my chat-logs periodically for hot chicks I might have a crush on :-)

Never found one though.

@brama What you first should consider is to ask yourself whether you really need parallelization. Is it really worth the effort.

@halirutan So you are good at timely deleting stuff

@brama I have moderators doing that for me.

@halirutan The term "sub-kernels" always confuses me....in your thread..you meant multiple kernels Mathematica launches when you run compiled code with parallelization

@brama There is a difference between the main kernel you are using all the time and the sub-kernel which is used as worker slave by the main kernel.

So when you start up Mathematica and type 1+1 this is evaluated by the main-kernel. When you do a ParallelTable then then main-kernel evaluates the call and gives all the work to the sub-kernels.

@halirutan But in the context of Linux clusters, I have noticed that Mathematica only works on one processor.(.as you showed in your thread) so one needs to explicitly activate them SystemOptions and ParallelThreads

@brama That's a different story.

@halirutan How about when you run a compiled code with parallelization and listable?

@halirutan does the same thing happen?

@brama yes, exactly this is a different kind of parallelization.

@brama When you define a function as Listable and use all the parallelization options etc and compile it, then this is fundamentally different from using ParallelTable.

@halirutan Then, when/why would you need to use LaunchKernels[], if at all both parallel* and compiled code automatically create sub-kernels?

@halirutan How are they different in their operation in terms of assigning work to sub-kernels?

@brama compiled parallel code does never launch sub-kernels. It is parallelized through a different method.

I have seen that using the notebook FE evaluation, both the cores on my computer are utilized (~65% CPU utilization)

@brama Let me try to put it clearly:

Both ways of utilizing parallelism are completely unrelated. Let's start with the listable compiled functions.

ok

In earlier Mathematica versions you always could compile down simple functions. This was always restricted to numerical code which means you can compiled functions that take numbers or tensors of numbers and output numbers or tensors of numbers. You couldn't do for instance Expand[(a+b)^5].

(Actually, you can do this in compiled code but then the compiled code will call-back to the mathematica kernel to get an answer for this task)

Let us concentrate on the compiled functions that are purely numerical and can be compiled down completely.

ok

@brama So let's say we have a compiled simple function which doesn't need the sophisticated mathematica kernel to calculate its result but it can run on its own. (This is not entirely true because if you don't compile to C then your compiled function is run be a virtual machine similar to java running byte code. This is called now the Wolfram Virtual Machine. Let's ignore that)

Now, having such a function fc which takes numerical arguments and creates numerical output, you could theoretically call it more than one time in parallel because it doesn't depend on the kernel which you have only one.

Now, you need to know that there are several "threading libraries" like pthread which let's you call subroutines independent from each another.

ok

@brama OpenMP or even CUDA use the same paradigm which is called Single Program Multiple Data (I always knew it as SIMD which stands for instruction, but this seems to fit better).

@brama Now, let's assume you want to calculated your function fc for a list of numbers (or arguments). Then you could use a for-loop to go through every number of your list and call fc or you divide your list into several sublists and calculate them in parallel.

@halirutan yes

@brama This is exactly what happens when you define your function listable and parallel. Your mathematica kernel just runs many fc functions in parallel. Have you noticed that we don't need more than one Mathematica kernel because your function fc is somewhat independent of a kernel and can run on its own?

@halirutan But I have just noticed that when I run the following code on my laptop, it starts a new sub-kernel (I can see it in Task Manager - details).

cf1 = Compile[{{x, _Real, 0}, {y, _Real, 0}}, Min[x, y], Parallelization -> True, RuntimeAttributes -> {Listable}];
x = RandomReal[1*^6 {-1, 1}, 5*^6];
y = RandomReal[1*^6 {-1, 1}, 5*^6];
cf1[x, y];

@brama But you know that you always have 2 processes. One which is called (here on Linux) Mathematica and one which is called MathKernel.

@brama You see two MathKernels, yes?

@halirutan yes

When I quit kernel from Evaluation, one of the kernels disappears

so when I run the code, a new kernels starts and I have two kernels

@brama This is quite odd and never happens to me on neither Linux nor Mac.

@brama Let me make a small example to test something.

ok

btw I use Windows 8.1 and Mathematica V10

@brama Can you test, which kernel is used when you do

Do[RandomReal[], {10^9}]

@halirutan new kernel is created

@brama Then there is something else wrong because this is definitely no parallel call

yes..you are right..but Not sure y?

forget it...Tell me how the paralellization is supposed to work differently with compiled code and ParallelTable

@brama The front end uses another kernel AFAIK. Maybe you see this process there. Do you have additionally a Mathematica process?

yes

what is that 25 in the 6th column?

@brama You tell me. I have no idea. The anniversary of Mathematica? :-)

hahaha...

@brama I'm back in a minute...

The number becomes 0 after the evaluation stops...So I am guessing it's # of threads?

@brama You have 4 processors?

yes 2 cores and 4 processors

@brama Then I guess it's the % of your cpu usage. 25% is one of 4 cores running.

@brama OK, back to topic:

hmmmm but the performance shows upto 32% and this 25 does not change

@brama Then I have no idea. Isn't there any heading in the table you showed?

Yes..It is CPU

anyway...Let's go back to the topic

@brama OK:

one thing before we go too far..does it influence how parallelization happens with CompilationTarget "C"? My guess is it will not...but will you please confirm!!

@brama AFAIK you can run the compiled code in parallel the same way if you use C or the WVM. There are surly differences but I hope nothing major.

@brama If something comes up, we would need to look at it. Back on topic:

ok

Since you really need a MathKernel for your task, there is no other option than starting other MathKernels and distribute your work. Exactly for this reason Roman Maeder (afaik) started to write the Parallel Computing Tools.

@brama As you might know the front end is just for input/output and some fancy stuff but the calculations you evaluated are sent down to the MathKernel and the result is read back. Now the idea was to give your main-kernel a bunch of slaves which it can use to distributes its work. To make this accessible to the user, a set of new functions aka the ParallelTable, ParallelDo etc were written. Clear so far?

yes

But none of them require you to execute LaunchKernel[] prior..right?? They all automatically start slave kernels

@brama Now the approach is the following: You simply copy the snip of Mathematica code which is required to calculate your stuff to each sub-kernel and initialize all the values and variables it needs and then a ParallelTable will divide your iteration into sub-iterations and each sub-kernel will run a small part of the whole Table. At the end, the result is sent back to the main-kernel and you put it together to your final list. Most of the stuff (copying code, initializing)

is done automatically nowadays.

@brama Even this is done automatically.

@brama Try this:

ParallelTable[i, {i, 100}]
LaunchKernels[]

and note the warning:

> LaunchKernels::nodef : "Some subkernels are already running. Not launching default kernels again."

As I said, most of the stuff happens under the hood.

ok

@brama Now, you see that the two ways of parallelizing are independent of each other, yes?

Yes....So in case of compiled function the the main kernel runs parallel computations and in the case of ParallelTable, many slaves are created and work is shared!!

@brama Correct.

Does it work this way even for complicated compiled function?

@brama You can watch at it completely independent: For compiled functions you need to ensure that you don't have a MainEvaluate call in your compiled code. Then Mathematica can run your fc's in parallel. For the many kernels approach you need to weight the pro's and the con's of parallelizing your code.

ok

@brama Copying large expressions back and forth to the sub-kernels is time-consuming. Some algorithms cannot easily be parallelized etc.

Those are things to consider.

now tell me this, If I have a compiled function, all it needs is one main kernel and in that case, it does not matter if I have a 10 core machine or a 1 core machine, if the processor speeds are same... right!!!

multicore machine only gives me ability to create more slaves...no special benefit for compiled functions that only need one core!!

@brama Slowly! Yes, you need only one Mathematica kernel. For the speed, it clearly makes a difference if you have 10 CPU cores or 1 CPU core.

@brama Do we speak of parallel listable compiled functions or just simple compiled functions?

@halirutan parallel listable

@halirutan of course assuming no other process is executed simultaneously, is this still true? "For the speed, it clearly makes a difference...."

@brama Let's say you want to calculate x+x for a numerical x. If you have 1 CPU, then you can calculate only one computation at a time. If you have 10 CPU's then you can calculate 10 x+x at the same time. Ergo, it needs only 1/10th of the time.

oh..So a single kernel can use more than one core!! pardon my ignorance...

@brama Even if you have only one CPU, you could just start 10 Mathematica kernels and run computations on them but the time benefit is zero.

I know you can start more kernels than the CPUs, but did not know I can have one kernel using more than 1 CPU

@brama Yes. Your main Mathematica kernel can run fc[{1,2,3,4,5...}] and it does it by running the calculations for fc[1], fc[2], etc in many thread in parallel.

and threads use different cores..yes?

@brama No, not necessarily. You have to read how hyper-threading works.

How does it works in default MMA?

or is it a processor thing?

@brama I mean, look: you have 4 CPU's. Do you really think currently on your machine there are only 4 processes? Virus scan, browser, porn, Mathematica, porn, email and maybe another porn window.

This all happens simultaneously.

not the porn for sure..I am over it....

ok

@brama You did not study computer sciences, did you?

I am thinking all these processes are accommodated on (I have 2 cores and 4 processors) 4 CPUS

nope..I am a civil engineer..far far away from Computers...long time ago

@brama I guess you have 2 processors and each processor has 2 cores :-)

@halirutan see the right bottom on chat.stackexchange.com/transcript/message/17156648#17156648

@brama Nope, still don't believe it. Look here. Are you sure you have 2 physical CPU's in your machine?

Otherwise, what you see can be explained by having 1 Intel i5 CPU which has 2 Cores and uses Hyperthreading so you can have 2 threads on each core = 4

ark.intel.com/products/65708

@halirutan I think I see what you are saying at ark.intel.com/products/65708/…

hahaha

@brama hehe :-)

@brama Is this a laptop?

@halirutan yes

so the conclusion is that I have 2 CPUs?

@brama Then you probably have only 1 Intel CPU and what you see is the window for the 4 independent threads.

ok....

Now continuing to Linux clusters...I request # of processors using a PBS script...

@brama Look, I have a quad-core which is 1 Intel i7 CPU (the physical chip I plugged onto the mainboard) which has 4 independent Cores. Each core has 2 thread = 8 "cpu"s you see here

@halirutan yes

@brama OK, so no matter how many CPU's, Cores or Hyperthreads do you have, you can just start any number of Mathematica kernels with one restriction:

so generally all laptops have 1 CPU..right because you generally have only one chip!!

The license

@brama There are high performance laptops but since everything needs battery it is usually only one CPU

@brama Usually, you only have the right to start 2 main Mathematica kernels

@halirutan ok

This is opposed to the sub-kernels where you now can have 16 (?, I'm not really sure) if you have a single user license.

yes..16 for me

ok....going back to the topic...running a simple numerical computation based compiled function in a Linux cluster setting..

ofcourse it is parallelized and listable

I requested 8 processors on PBS. Do I need to specify 8 parallel threads using system options so that the computations can be faster?

because I have noticed that only processor is being used and the other are idle

@brama What does SystemOptions["ParallelOptions"] tells you?

@halirutan did not try...let me do it

LEt me not keep you waiting..if it says parallel threads ->0, make it 8 by SetSystemOptions[ "ParallelOptions" -> "ParallelThreadNumber" ->8]

@brama Yes.

but compiled function does not need multiple slave kernels. So I still need to define parallelThreads? You defined threads for using sub-kernels/slaves

in this case, does every processor work as a core? so more the merrier for performance of a simple compiled function?

@brama It is really just a setting. When Mathematica calls your fc for a list, Mathematica decides how many threads to start and for this it looks at this option. You can easily test this:

@brama Try on your laptop

SetSystemOptions["ParallelOptions" -> "ParallelThreadNumber" -> 1]

and then

cf1 = Compile[{{x, _Real, 0}, {y, _Real, 0}},
       Do[Cos[y] + Sin[x], {10^6}], Parallelization -> True,
       RuntimeAttributes -> {Listable}];
x = RandomReal[1000000*{-1, 1}, 1000];
y = RandomReal[1000000*{-1, 1}, 1000];
cf1[x, y]

and look on your task-manager-thingy

one thread is doing the bulls work