I'm working with a set of numerical data: numerous (a few thousands) files in the style of list of vectors (quadruplets x, y, vx, vy). Files are named as v1.txt, v2.txt, ..., vk.txt, where k is a few thousands. I am using For loop in Mathematica, in each cycle several steps are performed:
1) loa...
Hard to say. The code looks okay to me. You could also try to wrap the body of the loop in a Module and scope all temporary variables within, so that they get cleared by automatic garbage collection.
Maybe one of the built-in functions has a memory leak. You could narrow down the problematic part of the code by commenting out various parts of the loop's body and see if the memory leak persists. Should you find one, I would advise you to contact the support.
@HenrikSchumacher: could you please expand on that Module idea? By looking at the documentation, I don't quite get what I should do inside the loop...Module[{v, Absv, LDP, LSP}, the body of the loop]? This does exactly the same thing as before: my memory eventually leaks.
Aha. Okay, I did not really expect it to help. You did exactly what I meant. Next step would be to identify where exactly the mess happens. I would not be supprised, if it happens with the Import/Export calls...
If nothing else helps, chop the For loop into several smaller chunks and use Exit[] in between to restart the kernel. Only a workaround of course...
@HenrikSchumacher Not sure if I've gotten your idea, but I wrapped the export (to image) calls in comments and the result is the same: memory leak. Of course, no images are produced, just a bunch of memory is wasted on...something. So the culprit is not the export call. I don't know how to try the import calls: if I comment out the import calls there wouldn't be any data left to make plots from, so the loop would produce just a bunch of errors. Can't I use that Exit[] clause at the end of the For cycle (or would it be too slow)?
Aha. It's the Import then. You could play with it by using different export filters (instead of "Table") such as "Text", "Data", "CSV" what not. Maybe you find one that uses a different, hopefully leak-free backend... Thinking about it: Can you also try to strip the Monitor?These Dynamic things also feel a bit suspicious. You can use If[Mod[i,100]==0,Print[i]]; instead for monitoring.
Okay, I tried Data, up to no avail. I also tried Text and CSV, however I have no clue how to convert those to proper numeric data inside Mathematica (Text produces just a huge string of numbers divided by space which I don't know how to chop into table again, CSV behaves weirdly when I try to apply partition to it (it won't chop it into pairs of pairs as it did with "Table" option). Also, removing monitor did nothing regarding the memory leak.
I changed "Table" to "Text", however I had to remove flatten and partition (asi those are not interpreted correctly and produce a bunch of errors). I also had to comment out everything after that Import clause, as the rest works with the imported data (not possible if the data is string)...with this version (only imported as text), there are no memory leaks. However, keep in mind I commented the rest of the cycle, only import was left. How can I shape the imported string into tabular expression as intended originally?
Almost correct, one issue though: I tried this on one file and the output looks like {{{0.99017, -0.968496}, {-6 + 2.40753 e, -6 + 3.97776 e}}, {{0.986971, -0.958663}, {-6 + 4.56787 e, -5 + 1.27839 e}} ...} so the scientific notation (2.40753e-6) is not correctly interpreted (-6 + 2.40753e instead of 2.40753 * 10^-6)
Wait. I tested that leak again. If I comment out LSP (ListStreamPlot) and the export of Show[LDP, LSP], used memory goes up much much slower (several MBs every ten seconds, instead of half a GB every ten seconds)
Is it possible that the leak will be there anyway, just the ListStreamPlot is much more demanding than the density plot?
P.S.: if I comment out only the export of Show[LDP, LSP], the leak persists. It must be that streamplot.
It's been running for a few minutes now, just exporting density plots and not the combined stream+density plots. The memory stays around 1GB, not going up wildly.
Once I add ListStreamPlot and the export of Show, the memory leak occurs again.
But it's hard to reproduce...I will have to come up with an example. But it seems this has nothing to do with my special set of data - I can generate it inside Mathematica and then liststreamplot it demonstrating my problem. Or use just streamplot instead and see if it leaks as well. Either way, thanks for your effort. Is there any way I can reward you with some rep?
It's enough reward for me if you contact the support.
You can send them also the link to your post.
Last time when I contacted them, they send me the link to my own post ^^
Anyways, a minimal example would be great. Please post it here and ping me when you are done; I will try it also on my machine tomorrow. I am going to bed now. Bye!
