Wolfram Mathematica

12:39 AM

I am trying to execute a command for many things such as many images or many knots:
1. "Table[Graphics3D[i], {i, KnotData[]}]"
2. LaplacianGaussianFilter[#, 2] & // ImageAdjust \@myImages

but fail, for example with 2, what is the problem?

rm -rf

@hhh what is \@ supposed to do?

also, you should read up on operator precedences in Mathematica

hhh

Execute the command on a list like myImages={oneImg, twoImg, threeImg,...}

err @@, I think I mistyped...

In Python, I would write it with list comprehension

[Graphics3D[i] for i in KnotData[]] <--- how can I do this in Mathematica?

Basically command Graphics3D[#]&@@KnotData[] fails because It does not do this, it tries input the whole list to the Graphics3D?

Matariki

1:00 AM

@hhh Have you had a read of what KnotData[someKnot] does? It provides you with an image of that knot not something that is a 3D primitive and can be feed into Graphics3D

hhh

[Knotdata[i] for i in KnotData[]], how to do this? Where [...] is pythonic list-comprehension aka anonymous function.

(I wish there was something like pythonic list-comprehension in Mathematica, easy to do things with them.)

halirutan

KnotData /@ KnotData[]

hhh

How do you think this? It is not like traversing one-by-one?!

halirutan

What do you mean by how do you think this?

hhh

1:18 AM

Too implicit, where do you specify the variable(s) and how to preprocess easily like with list-comp? Suppose more complicated thing such as

[LaplancianGaussianFilter[i,2]//ImageAdjust/ for i in myImages]

Or let say you want to take specific terms from myImages. In python I would do

[x^2+y+z for x,y,z in zip(X,Y,Z)]

In Matlab, just X.^2+y+z. I don't see yet *how* the traversing works explicitly, suppose the input contained material not possible to input into some function hence some preprocessing with anonomyous function like pythonic list-comp

rm -rf

3:30 AM

@Rojo Are you here?

Rojo

@rm-rf Hi

I ate too much, hic

What's up

rm -rf

@Rojo If I have a symbol Global`a and pass it to a function A`foo (defined as A`foo[x_] := something with x), does it create a copy of Global`a as A`x? (i.e., are there two identical valued symbols in memory, however temporary)

Rojo

@rm-rf Not as far as I understand, why?

You tested and it happens? You have some test code?

rm -rf

@Rojo I'm trying to simplify and narrow down the root cause of the problem (right now I can only describe the data structure, don't have an MWE yet)... I'm not sure if it's related to copying or unpacking, but it's annoying me, no doubt.

grrr

Rojo

@rm-rf Hehe, happens. Let me know if I can help

rm -rf

3:44 AM

Ok, I'll give this vagueness a shot as I also work on narrowing it down. Do you see any problem with such a structure? I have data files (saved as .mx files) that look like:

myData[{"str1" -> val1, "str2" -> val2, "Data" -> giant packed array}]

Rojo

@rm-rf And what's the problem you run into?

rm -rf

I now have a = {myData[contents of file 1], ..., myData[contents of file N]}, which is passed to a function func[a, ...], that basically looks like :

func[dat_, ... ] := Module[{var = "str1" /. First@dat, ...},
    couple more lines similar to "str1"/. First@dat;
    big function
]

I inserted some print statements, and it seems to be taking waaaaay to long to even get to big function

everything before that is just a bunch of simple replacements

ByteCount[a] is approx 3GB

Rojo

@rm-rf That {"str1" /. First@dat, ...} isn't exactly like that, right?

rm -rf

@Rojo woops... fixed myData. It only has 1 list (all the rules are inside that list)

Doing each of these outside that function runs in like 0.000000x seconds

Rojo

@rm-rf But in your Module initialization list, you have your var=..., right?

rm -rf

3:51 AM

All the ... are negligible initializations

Rojo

Module[{"str1" /. First@dat, ...}, or Module[{someVar="str1" /. First@dat, ...},?

rm -rf

@Rojo Oh, yes

grrr

sorry

Rojo

Just checking, hehe

I figured it was a typo

I don't see anything wrong. Could you find line that slows everything down?

So you think it may be because it somewhere makes a copy of your 3GB argument passing, or because it is unpacking it?

rm -rf

@Rojo That's what I'm doing now... dismantling it till I find the crazy part.

Earlier today, I ran into major issues with unpacking... Turns out, I had a Chop somewhere, and since it was FFT data, the first bin, which is real, became Real instead of Complex. That blew up my data file to 7-8GB (should've been 1.xGB)

Rojo

You know how to check if something is being unpacked. It would only make sense to me if your "str1" isn't matching so ReplaceAll has to check inside the arrays

Ohh, I should bear that in mind, thanks.

rm -rf

4:01 AM

@Rojo They all match... :/

Rojo

@rm-rf Somewhere along the line your memory usage goes up more than it should?

rm -rf

@Rojo Yes... I start with a ~3GB data file, do a chain of transformations (using Composition) and end up with a ~1.3 GB file

My memory usage however spikes to 11GB

Rojo

@rm-rf Then perhaps you could trace that somehow

perhaps something like this? TraceDialog[whatever, _ /; MemoryInUse[] > threshold]

and then inspect the stack?

rm -rf

Hmm... looks like that's what I should do

(uncharted territories...)

Rojo

Good luck, I don't think I ever did that. Remember ExitDialog@Unevaluated@Abort[]

rm -rf

4:10 AM

@Rojo What is that for?

Rojo

@rm-rf Exiting a dialog and aborting

avoiding falling back into another dialog

rm -rf

Ah, ok

@Rojo I think I see the problem... I haven't traced it yet, but I see a potential way for lots of unpacking to happen. Also, 11 GB sounds about right for what my unpacked data (+ overhead) should be

Rojo

@rm-rf :)

Sjoerd C. de Vries

5:20 AM

@szabolcs There are many quick tests that one could do at the start of the Q function to directly see that it's not a isomorphic subraph. Of course you have the test on numbers of vertices and edges. The big graph should have the same number or more. A less trivial test would be to compare the sorted list of number of connections for

each vertex. Whenever the small graph's list has a number on the same position that's larger than the number in the same position of the other list then you know it can't be a subraph. This list could be used for more optimizations in the search process.

rm -rf

5:52 AM

@Rojo I solved most of the problems (without having to do trace/stack voodoo)

the slowdown was partially caused by some Dynamic. I had used ParallelMap and was constantly monitoring the subkernel usage (which is dynamic) and that was causing some weird slowdowns. There was also some unpacking going on because of MapThread (didn't realize it unpacked). I rewrote some nested map threads (which in retrospect, was horrendous) with a single transpose (with the second argument, which I now love and am comfortable with) and now nothing unpacks

halirutan

7:50 AM

@hhh This has nothing to do with too implicit. The functional way to apply a function to a list of values is to use Map(/@) which I used.

This exists even in python. If you want something similar to list comprehension, then just use Table. So when you would have [i^2 for i in mynumbers] you write in Mathematica

Table[i^2, {i, mynumbers}]

@hhh The next question is, why is it too implicit to leave out the arguments, when I know my function has only one parameter?

There is no order in which the arguments need to be passed, so you can just write f/@{a,b,c} to get {f[a], f[b], f[c]}

Of course, if it is too implicit for you, then just write it out:

Function[theArgumentOfF, f[theArgumentOfF]] /@ {a, b, c}

or shorter

f[#1]& /@ {a,b,c}

or shorter

f[#]& /@ {a,b,c}

or shorter

f /@ {a,b,c}

but wait.. that's where we started.

kirma

8:56 AM

Listable functions would do "automatic list comprehension" too

Mr.Wizard

9:30 AM

As the data has now been shared this might be reopened:

-4

Q: Problem with interpolation

I have a problem with the function interpolation. when I use it my points are all very shifted. Do you have an idea of the solution? data = Import["EQE.txt", "Table", "HeaderLines" -> 3]; eqelambda = Table[{data[[n, 1]], data[[n, 5]]}, {n, Length[data[[All, 1]]]}]; h[x_] := hplanck*c/(x*10^-9) ...

interpolation

J. M.

@Mr.Wizard The data should probably be mirrored somewhere for safety, just in case. Can you do it? (I'm on a slow connection, so I can't.)

Mr.Wizard

10:10 AM

@J.M. At the moment DropBox isn't working for me; I cannot download the file. It's probably some bad browser setting or plugin.

Anyway, I need sleep. See you another day!

rm -rf

1:47 PM

@J.M. Do you happen to remember a question on converting a rectangular grid of (r,θ) to a 2D polar plot?

J. M.

@rm-rf I think it was of Jens's answers...

rm -rf

I seem to remember the pictures of the plots very well, but sadly, those aren't good enough for searching :D

J. M.

@rm-rf This?

rm -rf

2:04 PM

@J.M. Hmm... I already saw that, but thought that he simply masked it with a region function

J. M.

Okay, still poring through the stuff I recall...

Well, there's the one with polar stream plots, but I don't think you want that...

I got nothin'.

rm -rf

@J.M. Ah well.. ok. Now's the best time to put my MATLAB link to good use =) I'll just use existing tools there (ListDensityPlot was going to choke and die anyway...)

J. M.

@rm-rf Oh, lots of data?

rm -rf

@J.M. not very large in real terms, but I've seen ListDensityPlot choke like hell for even 300x300 matrices

hhh

3:55 PM

My mathematica crashes on startup because it tries to open my old windows having intensive calculations. Is it possible to start Mathematica fresh?

I am on OS X.

halirutan

@hhh Start it from the command line with -clean option

Note, that this resets all your settings.

hhh

-

Moved here.

acl

5:08 PM

@hhh have you tried the "Close windows when quitting" option? in the preferences, here:

hhh

Mathematica clean does not close the windows :(

halirutan

@hhh Are all your Mathematica windows closed??

Or is it still running?

hhh

They still tries to open up even with the clean flag

acl

@hhh if you try unchecking the option I mentioned above?

hhh

@acl I cannot do it because my mathematica crashed before I have time to click it

acl

5:12 PM

if that does not work I can tell you what files (and where) to kill

halirutan

@hhh But you were able to start Mathematica with the command-line, right?

acl

no that's in the system preferences

hhh

@halirutan yes but it crashes similarly as in GUI

acl

not the Mathematica preferences

@hhh from the apple menu on the top left, then System Preferences...

hhh

@acl I tried it but it did not work

(it still tries to open the windows)

acl

5:17 PM

hm, can't find my notes on where the old windows are stored

@hhh I need to do something else; I'll ping you when I find them

halirutan

@hhh What happens if you just move the notebook it tries to open?

I mean, it tries to load it from your hard-drive or not?

hhh

@halirutan the problem is that it only tries to open Help windows -- I modified examples in the Help and tweeked/twisted so I have no idea where they are stored

halirutan

@hhh those changes are always lost the moment you close the help-browser.

It's very unlikely, that this is the problem

acl

@hhh try rm -rf ~/Library/Saved\ Application\ State/com.wolfram.Mathematica.savedState/

rm -rf ~/Library/Saved\ Application\ State/com.wolfram.Mathematica.savedState/

however if you have only played with Help windows, those changes aren't saved, so it's unlikely that this is your problem

hhh

@acl It worked! :D

Thank you, now it does not crash :)

acl

5:25 PM

great

hhh

(I had over 10 windows there)

acl

@rm-rf rm -rf ~/Library/Saved\ Application\ State/com.wolfram.Mathematica.savedState/

@rm-rf (nothing, I just thought it would be funny to do that)

hhh

I wish this thread could be reopened: mathematica.stackexchange.com/questions/22566/… and this kind of things moved to real answers.

(it is clear that it could help future visitors...

hhh

6:14 PM

I am trying to play tic-tac-toe against Mathematica but I am still trying to detect x from the image

How would you detect circles and x alphabets from writing?

CountourDetect is cool but how to convert ContourDetect object into image for ImageLines function?

hhh

7:06 PM

[1] Moved here a subproblem about detection crosses and circels.

kirma

@hhh edge detection and other preprocessing trickery + Hough/Radon transforms would be my bet. I guess Hough transform for circles/ellipses you have to write yourself.

most certainly there are ad-hoc methods for this specific task.

ImageLines, and some of the morphological transforms (such as BottomHat transform) might help too

Personally I'd like to see solutions based on training sets and some sort of similarity measure search on detected components, but I don't know how many would be eager enough to write such an elaborate solution.

hhh

@kirma you are welcome to answer the question [1]: anyone can try -- I personally tried totally different methodology, firstly smoothing: i.stack.imgur.com/1mgs5.png

kirma

@hhh well, probably some preprocessing is needed...

hhh

(I thought I need to smooth the image a bit before processing it to make it easier for mathematica to solve) Yes.

kirma

I rarely have time to write elaborate solutions I'd like to write, but despite that, heuristic one-of-the-kind solutions seem a bit inelegant to me. On many image recognition questions, I'd love if someone would come up with training or image similarity search methods. :)

2

Sjoerd C. de Vries

9:10 PM

Anyone got an idea why Pick[{a, b, c, d, e}, {1, 0, 1, 0, 0}, Except[0]] doesn't work (i.e. doesn't return {a,c})? The third argument of Pick is a pattern and Except[0] is a pattern AFAIK.

rm -rf

@SjoerdC.deVries wow... that's weird. It looks like Pick just ignores everything and returns the input when you use Except for the third argument. For example:

Pick[{a, b, c, d}, {1, 0, 1, 0, 0}, Except[0]] should ideally give a Pick::incomp error

Sjoerd C. de Vries

@rm-rf Looks like a bug then, right?

I'll contact WRI support

rm -rf

@SjoerdC.deVries yeah, I'd say so... definitely not what's expected

Sjoerd C. de Vries

OK, thanks for confirming

drN

11:56 PM

I find that the interpolating function generated by using Interpolate[...] on a list is perfect when compared against the actual list data used to create it. Am I missing something? Is the interpolating function really perfect?

halirutan

@drN What do you mean by it is perfect?

drN

@halirutan There is no error that I find between the Interpolating function and the actual list when plotted together.

Transcript for