@brama What do I do with that link?

@mfvonh that took me some time to understand too!

@mfvonh I have also used an alternative to that one, that might or might not work in some or most cases

@Rojo I'm all ears. I've been running into this a lot recently and decided today that I want to understand the intricacies of it.

@mfvonh You've been running into what, the need to evaluate one step?

Yes.

@Rojo In his implementation, I see that P = (P = works but I don't understand why

@mfvonh Well, P is a function that, when called, changes itself

It changes itself into a function that, when called, exits the scan returning that first argument

So, the scan will only return that second time P is applied.

But it's hard to undertstand, I don't have it fresh in my mind

My implementatin takes a different approach, probably simpler, that used to work almost always like Wizards (haven't tried in v10)

At the time, I must have seen a difference, because I created a super step function with the option "Attributes" that, if True, used Wizardś , adn if False, used mine. Why I did that, don't remember

@Rojo Yeah it's strange. I understand what it does, but still don't have good intuition about it. For example if I change TraceDepth -> 2 and then do step[4 * (2 + 3)] I get Times. Same if I set it to any number greater than 2. But Times is never by itself in the Trace.

@Rojo I though you were looking for Times in Linux..my bad if that's not what you were looking for!!

@brama Wasn't that a link to some question about an poerator in a bash script?

Haha

@Rojo It's of course in the TraceScan so I see how the answer shows up. But I'm just trying to wrap my head around all of it is all.

@mfvonh Lets see

@Rojo It seems actually that if TraceDepth is absent or anything besides 1 you will get the outer head.

@Rojo Let me just shut!!..was doing a quick and dirty search to see if I can help!!!..btw I though it talks about multiplication operator as "*"

@brama Ahh, Times New Roman :D

@Rojo No need to spend any time on it unless you feel like it :) I haven't spent much time trying myself yet

@Rojo oh....

@mfvonh Oh, I have sidetracked because my implementation seems to not work. Furthermore, I feel like it (because I have other things to do)

@Rojo That's the best time to get distracted on SE :)

@mfvonh WAIT

It's not that it doesn't work, it's that it interprets a "step" differently

Ok, good. Now, let's understand wizard's versoin

@mfvonh Do you have a sample output that you don't understand?

ugh one sec I'm getting copy/paste bullshit from MMA 10

output: 312500/. 3->1

Sure that makes sense

But then I do TraceScan[Print, 4*(2 + 3)^f[1 + 2] /. (3 -> 1), TraceDepth -> 1]

@Rojo hi, may I interest you in a spot of graph theory, then?

Well now that I am spelling it all out I feel like it is a simple question of precedence

(joking of course)

@acl Hehe, I know I can't help in graph theory, but I am always willing to waste my time

@mfvonh Hehe, ok

@Rojo well actually I guess my question is, if I do that TraceScan line with TraceDepth->1 I never see ReplaceAll as a head. Why does it show up with TraceDepth > 1 ?

Hold[4*(2 + 3)^f[1 + 2] /. (3 -> 1)] // FullForm == Hold[ReplaceAll[Times[4,Power[Plus[2,3],f[Plus[1,2]]]],Rule[3,1]]]

@mfvonh ReplaceAll is the head

it's like the 0th argument

it gets evaluated one level deeper

Try Trace[yourExpresion, TraceOriginal->True]

all, could someone run the following pieces of code and tell me what they give? I'd appreciate it

ClearAll[timeIt];
SetAttributes[timeIt, HoldAll]
timeIt[expr_] := Module[{t = Timing[expr;][[1]], tries = 1},
  	While[t < 1.,
   	tries *= 2;
   	t = AbsoluteTiming[Do[expr, {tries}];][[1]];
   	];
  	Return[t/tries]]

randomSparseMatrix[n_, nonzerosites_] := SparseArray[
  Thread[RandomInteger[{1, n}, {nonzerosites, 2}] ->
    RandomReal[{0, 1}, nonzerosites]],
  	{n, n}
  ]

Monitor[
 Table[
  (s = Ceiling[#];
     g = (Normal[
         randomSparseMatrix[s,
          Ceiling[s/10]]] /. {0 -> \[Infinity]});
     {s, timeIt[
       cc = ConnectedComponents[WeightedAdjacencyGraph@g];]}) &@i, {i,
    E^Range[Log[100.], Log[2000.], Log[2000/100.]/10]}],
 i
 ]

@Rojo oooh right

should take a minute or so

@Rojo 10-q :P

@acl I don't know. Theoretically it should be linear in the number of vertices PLUS the number of edges, or something like that. I've seen weird complexity from Mma before.

@Szabolcs OK thanks.

@Szabolcs could you run that code and let me know the timings? I am wondering if I've changed some option some time in the past

(should be 20-30s)

Wiz's step steps from all level 1 transformations. Argument evaluation, application of attributes, iteration

I'm slooow

thanks guys

(my laptop is slower than either of yours...)

@acl It's a new 12-core workstation ... my laptop would be (much?) slower

@Szabolcs I don't think this stuff uses more than a single core

anyway, both of your timings jump at around the same point as mine

@acl sorry wasn't paying attention

This is on a not-great laptop

@mfvonh cheers!

here's the timings, x-axis is size of matrix, vertical is time taken, colours indicate by whom (@rojo, @Szabolcs, @mfvonh, myself)

OK at least I know it's not something I did

does the same on a number of linux machines I just checked.

@acl It is something you did...to allof us

lol

@Rojo oh yeah, I screwed up your options telepathically

I guess I had to be good at something

@mfvonh Why are you needing the step function lately?

@Rojo Debugging mostly. I have an application with a symbolic search architecture for non-technical users and it has lots of *Values defined

@acl. I ran your code on V10. About 30 seconds later, without having done anything new with MMA, I got beep and found that the kernel had crashed. Further, the suggestions bar turned red and displayed an error message.

@m_goldberg thanks, same step as the rest of us

@m_goldberg oh. here it runs fine on v10

Hope I did not cause you to lose any unsaved work!

@acl. No, nothing lost. Wasn't doing anything else with MMA at the time.

There are lot of places in MMA where that sort of thing happens. Consider maxD]= OrderDistribution[{NormalDistribution[], n}, n]; NestList[{#[[1]] + 1, First@AbsoluteTiming[Mean[maxD]] /. n -> #[[1]] + 1]} &, {1, 0}, 5] returns {1, 0}, {2, 1.217399}, {3, 1.220662}, {4, 1.221624}, {5, 1.220797}, {6, 247.914156}}. Note huge step at 6.

woo I just passed my 1k rep mark!

@seismatica. Congratulations.

thank you!

Is my machine cheating

@seismatica congrats :)

@m_goldberg yes, it seems most higher level functions do this in mma.

@hwlau. How much ram to you have? I suspect you are giving MMA more memory to work with.

48G

@hwlau. That's a lot. I only have 8 GB. Could make a lot of difference.

i don't see where in the code requires a lot of memory

@m_goldberg do you think that matters?

let me check

@acl. Just guessing. I've given it no real thought.

OK with 4GB on a macbook air (with an i7) I get

with a Xeon E5540 and 6GB I get

{{1, 0}, {2, 3.607335}, {3, 1.057821}, {4, 1.064664}, {5, 1.065650}, {6, 1.071432}}

with a large number of AMD Opterons and 768GB RAM I get

{{1, 0}, {2, 6.636359}, {3, 1.801243}, {4, 2.459989}, {5, 1.661686}, {6, 1.508520}}

so something else is going on

768G!

well, it's not mine

@halirutan Are you around? Have you ever used strings with LibraryLink? Their memory management looks really messed up ...

@acl. The result I showed you was from V9. Haven't tried on 10. Let me try it there. They might have fixed something.

the last two were on v9

@Szabolcs No, I have never really used it in depth.

on a xeon X7550 with even more RAM,

{{1, 0}, {2, 8.122596}, {3, 2.418880}, {4, 2.226984}, {5, 2.288424}, {6, 2.409060}}

> When a string is returned as a result, the Wolfram Language accesses the memory to convert it to its own internal string format, but does not attempt to free the memory. Thus, if your program allocates a string for a result, it also needs to free that memory, but in a separate function from the one setting the string result, since the Wolfram Language needs to access the string memory after the library function has returned.

Mystery solved

it should be the problem of the L2 cache of the CPU

The disjoint shown in your graph

and it does not look like it's MKL or something like that

So to return a bloody string, I need to define two library link functions, one that returns it, and one that has to be called right after to release the memory!?!

@Szabolcs Hehe, sound nice. And there isn't any Library functionality you can use to allocate the string?

mine occurs around 2000

@hwlau what is this on?

@acl, Perhaps it is OS X specific. WRI tech support said they reproduced it and said they would look into it. It just finished on V10. No change in the timings.

@m_goldberg are you referring to my results or to yours?

Rojo got the same on mine, and he's on Linux.

@acl. my results

maybe it is related to L2 cache

@halirutan Doesn't seem so ... at least they didn't rename all the MTensors to WTensors!!

@m_goldberg no, the results I showed were all from Linux machines (except from the air). Where would I get a mac with 1TB memory? :)

@acl Results for up to 4000

@hwlau I mean what CPU

i7 3930K

@Szabolcs You could turn your string into ASCII chars and put the character codes into an MTensor to send it back :-)

@hwlau 12MB cache?

ya

let me see

I doubt it's that simple but who knows

ya, but it is likely related to memory bandwidth

not the others

@hwlau well either way I tried on one with 18MB cache. the thing simply produces nothing after size 1500.

who knows why

maybe I'll solve this some other time

18M maybe the L3 cache in some CPU

@hwlau hm good point let me recheck