Wolfram Mathematica

Nasser

03:07

hi @halirutan You said on Meta to ping you here if one gets error using SEUploader. I started getting this error 2 days ago. I just deleted SEuploader and did fresh install again using the command Import["http://goo.gl/rQtfHZ"] but still get the same error. Each time I select a cell and want to update to site, I get this error message

SETools`SEUploader`Private`stackImage::err server returner error: You didn\u0027t enter a valid URL

I am using V 12.1 on windows 10. If you like me try anything or more info, please let me know. Thanks

Chris K

03:44

@Nasser oh yeah, I got that on 12.1 on Mac yesterday too

Nasser

04:04

@ChrisK thanks for confirmation! Yes, I started seeing it like 1 or 2 days ago. I think some change at stackexchange itself must then be causing it. I can't live without SEUploader :)

Lukas Lang

06:19

@CATrevillian You should be able to just wrap the whole thing in Legended

user21

@ChrisPurcell Thanks Chris. Indeed I'd be interested. What is the main benefit? However, I must say I can not promise that I can/will implement this and the main reason is that the way we assemble right now works and it's not bad performance wise. So I need to estimate if something (fundamental) like this is worth it compared to implementing requested features. What are your thoughts on this?

Henrik Schumacher

08:00

@chris The error messages are indeed not very informative. I guess the problem is that ToElementMesh does not like meshes with dimension lower than the ambient space. In a not shell: ``NDSolve`FEM```cannot do surfaces.

chris

08:37

@HenrikSchumacher It would be useful to 'mesh' cross sections of 3D mesh would it not? How would you do this?

Henrik Schumacher

ContourPlot or SliceContourPlot + DiscretizeGraphics maybe.

chris

DiscretizeGraphics is different from ToElementMesh then?

Thank you very much for your help btw.

Szabolcs

@b3m2a1 I didn't know about ExpressionStore. This seems to be precisely the answer to my earlier caching question ... I have a big graph that takes a lot of memory. I have a function that can return True/False for a graph and takes some time. I want to cache the result, but I do not want to hold a reference to the big graph in my cache because of memory concerns. It sounds like I should use an ExpressionStore and attach the cached value directly to the graph expression, right?

@JasonB. See above please ^

chris

09:02

@Nasser I get the same pb. I assumed it was only on my end

Szabolcs

09:17

@b3m2a1 @JasonB. @LeonidShifrin Can you take a look at this proposed solution and let me know if it is safe and sane? I am deep into uncharted territory here and I am a bit worried about using such undocumented functions that I just learned about. However, it does seem to work all the way back to version 10.0, and it appears to be exactly what I needed.

@TimStone Why is the indentation of the 5th code block line messed up in the rendered markdown here? Compare the source vs the rendered version. There are no tabs, only spaces. A conflict with $ and MathJax?

Leonid Shifrin

10:05

@Szabolcs I have commented under your answer there.

Jason B.

14:40

@Szabolcs Leonid is correct that by using an expression store it will only be cached for the single instance of the Graph

so if you do cachedFun[distinguishableTaggedEdgesQ][sameGraphDifferentInstance] it will recompute the result

Chris Purcell

@User21. This is new work which I have not seen yet, so can't defend its benefits. However the developer who pitched this to me is a FEM expert I have worked with for over 3 decades, so I trust his instincts. I agree that doing a cost/benefits analysis before embarking on a new foundational component would be necessary. Can we move this chat to email at [email protected] ?

Jason B.

@Szabolcs I use them to cache a ManagedLibraryExpression against a Molecule expression, so that when all references to the molecule are gone then the library expression will be cleaned up. If I stored the library expressions in a normal hash table instead then there would be no automatic cleanup.

kirkus

@LeonidShifrin in the above example from @Szabolcs would e.g. Hash[tg1]/Hash[tg2] really be unsuitable for caching a simple true/false result?

is the "Expression" Hash algorighm that prone to collisions for temporary use within a kernel session?

Jason B.

14:58

@kirkus I was thinking the same thing, pastebin.com/raw/nSTPmgpA

I would think the chance of collision negligible, but I say that with no real knowledge of hash-collision likelyhood :-)

kirkus

@JasonB. yeah - clearly the cryptographic algos like "SHA256" etc. would be highly rock-solid, but for a temporary cache of a true/false result, it seems like you could just Hash[] the entire graph and store the expensive function result once.

and the crypto functions are quite slow, "Expression" is very fast!

Leonid Shifrin

15:33

@kirkus I don't know. The question is, can you afford a collision? This would potentially be a very hard to catch bug, if you get from the cache something computed for a different graph that happens to have the same hash. Because most of the time things will work just fine, but in some rare cases there will be problems. Those problems won't be easily reproducible either, since their presence will depend on what has been already stored in the cache. So I'd prefer cache to be 100% correct.

kirkus

@LeonidShifrin do we know anything about the internals of "Expression"? Is it designed to be similar to a CRC32? Or MD5? Or not really designed to be collision resistant? For example, I would be completely confident in "SHA256" being resistant, but I don't see many details in the docs for "Expression".

in other words, is it really feasible that two expressions in memory for a given kernel session could collide?

Leonid Shifrin

15:57

@kirkus I don't know the details you are asking about. I'd base my decision on what is the price one pays for the collision and subsequently incorrect value picked from a cache. If the computations have some statistical nature, perhaps the price is not too high. If every result must be correct, then one has to resolve collisions, or possibly find a custom function that will be guaranteed to produce unique ids for given structures (graphs here).

kirkus

thanks :53971750. It would be interesting to know the properties of the algorithm. If the probability of collision is 1 in 50,000, it's clearly a no-go, but if it's on the order of 1 in 2^64, I would consider that "guaranteed" for most of my use cases.

Emilio Pisanty

17:54

@halirutan are you around?

I'm getting the same SE Uploader error as in mathematica.meta.stackexchange.com/questions/2536/…

Nasser

@EmilioPisanty yes, many of us see the same message chat.stackexchange.com/transcript/message/53966191#53966191 it could be something changed in stackexchnage itself

Emilio Pisanty

@Nasser then I would invite you to comment, and add that chat link, on the meta thread I just linked.

Feeds

18:20

Stephen_Wolfram's Twitch video RSS: Fun with Math: Surprises with Arithmetic and Numbers - LIVE

posted on April 02, 2020

Science & Technology

Szabolcs

18:37

@JasonB. I understood that, but isn't that unavoidable if I want to avoid holding on to the graph in the cache? I think that's fine actually for the main use cases.

I had a Hash-based implementation similar to what @JasonB. posted, but using DataStructure["LeastRecentlyUsedCache"]. But I wasn't comfortable about the possibility of hash collisions, especially since I (as some of you pointed out) we don't know how likely that is

@JasonB. Interesting fact about Hash: it caches the result too. So if you benchmarked it with RepeatedTiming, that does not give the full picture.

Jason B.

Yeah - I did use repeated timing, and said wow that 's instantaneous :-)

kirkus

not sure if small empirical tests would be reassuring for your use case, but having run this over 100 times has never given me a Hash match: graphHashes =
Table[Hash[Graph[Range[100], RandomInteger[{1, 200}, {300, 2}]]],
1000000] // Counts // Max

halirutan

19:06

@EmilioPisanty Yes, and I've also seen the @Nasser has the same problem. So we need to find a way to debug this.

halirutan

19:17

Well, on the positive side... I can reproduce the error now

The other positive side is that the issue seems to be entirely on the "sending data" site and not all the hacks how we rasterize a decent image from all things you throw on the uploader :)

Nasser

19:34

@halirutan it must have been on the stackexchange side, because it suddenly started to happen 1-2 days for me without me doing any changes on my end. I think they changed something in their server or such. May be someone at stackexchange IT department knows what changes they made.

halirutan

@Nasser Yes, it is

My Java unit-test doesn't succeed anymore and it does nothing more than sending a small image to SE.

jose

20:07

@Szabolcs Hi. We released yesterday an automatic paclet update of the Dataset code. This should make your example behave better. Can you please confirm? Thanks.

halirutan

20:49

@LukasLang @b3m2a1 @CarlLange I remember that some (if not all) of you have experience with JS and web-dev. Please, pretty please, take a look at this post.

chuy

21:12

@jose How can I make sure I have the latest paclet update? I don't want to necessarily crash my session testing this out.

b3m2a1

@chuy PacletFindRemote["*Dataset*"] then PacletInstall the most recent one, probably

István Zachar

22:15

Hi @Szabolcs, @b3m2a1, @halirutan Do you have an idea about what is the fastest way to append thousands of likely new key-value pairs to an association?

halirutan

@IstvánZachar Do you need to change an existing association or can you create a new one?

István Zachar

@halirutan I prefer not to copy it but to change it in place as it is supposed to be even bigger (millions of keys).

I assume that copying a huge association around in memory is slow and memory-consuming.

halirutan

@IstvánZachar What about

ass = <|"key" -> 1|>;
pairs = Table[{FromCharacterCode[i], i}, {i, 65, 90}]

and then

AssociateTo[ass, Rule @@@ pairs]

This mutates ass

(alright, this sounds weird. Should have chosen a better variable name)

Leonid Shifrin

@IstvánZachar I would not be so sure about assocs. They effectively share memory, so when you add new rules, chances are that the previously existing one is not directly copied but reused

@halirutan Noticed that too :)

halirutan

@LeonidShifrin :)

jose

22:24

@chuy In 12.1, check the result of Information[PacletObject["TypeSystem"]]["Version"]. If you have the paclet update, this should give "12.1.0.1".

István Zachar

@halirutan Yes, I forgot to specify that the association is part of a larger association, so AssociateTo would not work. Or does it work on Datasets as well? Oh heck, this is getting out of hand...

@LeonidShifrin Even if said association is part of a Dataset?

halirutan

@IstvánZachar It's always these customers that don't know what they want :)

Lukas Lang

@halirutan just in case you didn't get a notification: I have added a potential solution to your answer in the linked post

halirutan

@LukasLang I got it and already gave you praise publicly!

Thanks so much.

Leonid Shifrin

@IstvánZachar Yes. Dataset is just a wrapper around lists and assocs, which brings some fancy formatting and a query language which some people find more convenient / concise. It does not fundamentally change the underlying data structures.

István Zachar

22:30

@halirutan Business as usual, isn't it? :) Anyway, do you perhaps know of a post with benchmarked (timing and scaling) Association/Dataset operations?

Lukas Lang

@halirutan Ah, in that case I am the one that didn't see the notification 🙈 And no problem :)

halirutan

@LukasLang mathematica.meta.stackexchange.com/questions/2536/…

No problem.

Lukas Lang

Thanks, I saw it now

István Zachar

@LeonidShifrin Ok, good to know it from a reliable source! But how should I know when my copied Association reuses previous instances under the hood and when it doesn't? Is this documented?

halirutan

@IstvánZachar Simple datasets contain a "list of associations". AssociateTo works afaik only when you have variable. So you really need to create a new dataset from your existing one.

Leonid Shifrin

22:36

@IstvánZachar It should always do that. I don't think it is documented, but when you add a single key-value pair to an assoc, it can be considered O(1) operation regardless of the size of an assoc. If you add a new assoc with say M entries to an existing one with N entries, then I'd guess it will be O(M) operation, regardless of N (which can be much larger than M).

I hope I am not wrong, but it has to be double-checked.

István Zachar

@halirutan Unless.... unless I do piecemeal AppendTo[myDataset["myKey"], key_i -> value_i] for each i of a thousand. It does in-place mutation. Can it be made faster?

halirutan

@IstvánZachar For instance

a1 = <|"a" -> 1, "b" -> "c", "d" -> {1}|>;
a2 = <|"e" -> 1, "f" -> "g", "h" -> {1}|>;
<|a1, a2|>

So the question is, if you have a list of associations (a dataset), do you want to append something on each of the associations?

István Zachar

@LeonidShifrin Ok, thanks, gonna check it tomorrow!

Leonid Shifrin

@IstvánZachar Was glad to help. Good luck.

István Zachar

@halirutan No, only to e.g. a2["h"], assuming that it has an association as value and not {1}.

So perhaps my structure is not a Dataset, but rather a nested, ragged, irregular association.

halirutan

22:44

@IstvánZachar Like this?

a1 = <|"a" -> 1, "b" -> "c", "d" -> {1}|>;
a2 = <|"e" -> 1, "f" -> "g", "h" -> <|"test" -> 1|>|>;
a2["h"] = <|a2["h"], a1|>;
a2

István Zachar

@halirutan Hmmm, yes, this should work. Thanks, I am going to test it out in the more complex actual case.

halirutan

@IstvánZachar Good luck

István Zachar

@halirutan Good night for now!

halirutan

Sleep well

Transcript for