Sašo Živanović

@cfr No, I haven't included that yet. I don't think the support code is general enough at the moment, or tested enough, and the issue is not at the top of the todo list. Finding a way to test the new scripts on Windows resides there ...

@Qrrbrbirlbel You might have seen the pull request with my take on memoizable scalepicture. I mention there that it might be nice to have a library creating \pgfpictureid from MD5 sum of the picture code. I cooked up something rudimentary: github.com/sasozivanovic/pgf-md5sumpictureid

Bottomline: I'm removing the \pgfsys@getposition hack right away. By my tests and your reports, advancing \pgfpictureid appears to do the job, and it's surely also conceptually cleaner.

The first picture gets memoized on the first run, when the green picture has ID 2; the third picture does not get memoized because of the undefined reference. On the next run, the first picture is not memoized and the other IDs shift; so the third picture, which now gets memoized, thinks green is in the current picture and performs a wrong shift.

Because the reference was unintentional (the node should have existed in the current picture), the current tree had no remember picture and got memoized. FYI, I attach a some code illustrating the problem — but I'm unable to reconstruct the part of the original where the final result was pretty enough to fool me into thinking that \pgfsys@getposition is doing a real job.

@cfr Ok, I'll settle for "almost". It seems that the hack was actually solving an inexistent problem. As far as I can reconstruct what was happening two years ago, I managed to erroneously refer to a node not existing in the current picture, but existing in a previous picture (I was developing some complicated styles for dynamic trees in Beamer).

\documentclass{article}
\usepackage{memoize}
\mmzset{
  deactivate csname/.style={deactivate/.expand once={\csname#1\endcsname}}, % so that it runs on 1.0.0
  deactivate csname=pgfsys@getposition,
}
\usepackage{tikz}
\begin{document}
\section{Foo}
\label{sec:foo}
\begin{tikzpicture}
  \node(blue)[blue,circle,draw]{memoized first};
\end{tikzpicture}%
\begin{tikzpicture}[remember picture]
  \node(green)[green,circle,draw]{remember picture};
\end{tikzpicture}%
\begin{tikzpicture}% [remember picture,overlay] % if this was uncommented, there would be no problem

@cfr And I've got the \pgfsys@getpositionmess almost figured out. I'll report soon.

@cfr This is great news! So advancing \pgfpictureid actually does some work.

@cfr Yeah.

@cfr You're not alone. I'm testing this at the very moment, and I can't reproduce the problem that hacking \pgfsys@getposition allegedly solved. :-)

@cfr Many many thanks for catching this!!!

@cfr No.

@cfr deactivate it.

@cfr The latter, i.e. the .aux contains \pgfsyspdfmark with a wrong \pgfpictureid. But if there is no mismatch, \pgfsys@getposition doesn't get executed and memoization isn't aborted. (As I said, these mismatches should be gone in the new version, but I still have to test whether that's true myself.)

Btw, I think you have convinced me to change the abortion UI to require \mmzAbort{<explanation>}.

@cfr This is to be expected. Memoize only reports abortion at the end of memoization.

Bottomline, can you try deactivate csname=pgfsys@getposition? I'm guessing when this macro is deactivated, things should work in the new version, while weird stuff should happen to the subsequent pictures in the old version (if they use remember picture).

If I remember correctly, I have introduced this hack of aborting upon \pgfsys@getposition to solve the problem of "remembering the wrong picture": in v1.0.0, once a picture was memoized, \pgfpictureid of subsequent pictures was changed, leading to problems. However, in response to Qrrbrbirlbel's trouble, the new version of Memoize (the one on GitHub) increases \pgfpictureid when utilizing a TikZ-extern, so — I realize now — the \pgfsys@getposition might have become obsolete.

@cfr This is the culprit, for sure. The ultimate source of abortion is some remember picture, but specifically, memoization is in your case aborted when encountering \pgfsys@getposition, which retrieves the position of the picture on the page saved in the .aux (which contains a \pgfsyspdfmark).

@cfr Yes, please use \mmzset{prefix=memos/}. No need for a mkdir either.

It should be fairly complete, I just have to write some minor updates of the docs, do some testing on Windows, and (most importantly) run the updated extraction scripts by Karl.

I pushed the work-in-progress version of 1.1.0 to GitHub, including the installable files in the release.

@cfr admirable 🤪. Trace helps me with funny "delays" a lot.

@cfr well, that's why i wrote memoizable which noone wants to use 😁

In other news, I managed to clean up most of the changes today, so I believe I'll push the wip version to github tomorrow.

@cfr have you tried trace to help you figure out the culprit?

@cfr no, if context changes during memoization, we don't abort. Some contexts can obly be figured out by compiling the code (so memoizing), \refs are an example of this.

@cfr you could say foo/.meaning to context/.try btw, have I defined this handler? (I'm mobile at the moment)

@cfr On second thought, Memoize already informs the user about activate=chronos, if that happens. Also, I guess you should not enable memoization in your package, as you don't want to force the author to use Memoize; but here I might be only overly precise about terminology, as auto does not enable memoization.

@cfr Well, you can always say \PackageInfo or whatever to give extra info to the author ...

@cfr Interesting idea. The problem is that most of the time, Memoize doesn't know the reason for abortion, either. The only time Memoize aborts of "its own free will" is when an error occurs, which should be clearly visible from the log. But most of the time, from its perspective, someone (like some package) requests the abortion, and Memoize obeys. So the burden of explanation would be on whoever calls \mmzAbort, and I'm not sure I want to have an obligatory explanation as a part of UI.

@cfr I haven't yet pushed any updates to GitHub, because the intermediate stages of the code were a mess. It should happen during this week.

Ah, not useful but usable. I don't know how I managed to misread this. Everything is usable ;-) You could input the c(c)-memo, appropriately redefining various macros before, to process it. That's what Memoize does (but not for code and context, in particular).

@cfr It is stored in those files forever. I found it usable for testing. Perhaps you could also grep through the c-memos to find the one corresponding to a particular piece of code. Other than that, who knows.

@cfr All this might depend on your system settings. I tested attachments with Atril, and if I remember correctly, Atril is a conservative fork of Okular? I normally use Zathura, but it doesn't support attachments / embedded files ... :(

@cfr Dumber answer: I had no clue there's even a difference. The example files under the paper clips are stored inside the PDF, so I guess they should count as embedded?

@cfr You open an example file, if your PDF viewer supports attachments.

@cfr The former, sure. The latter, probably not out of the box ...

@cfr Thank you for diving into all this!

@cfr I believe you're right. And even this should be doable with proper support from tcolorbox itself.

@cfr unmemoizable only applies inside memoization. nomemoize prevents memoization from ever taking place.

@cfr Yes, they would.

@cfr No, because tikzpicture, when contained in other environments, like forest, doesn't break their memoization.

@cfr You see, my bad writing wins. :-)

@cfr Indeed it would.

@cfr My bad, I should've written auto={tcolorbox}{nomemoize}, of course.

@cfr A contest! :D

So if you want to simply prevent memoization of all those tikz pictures used by tcolorbox internally, you say auto={tikzpicture}{nomemoize}.

unmemoizable, on the other hand, is meant to be applied to commands which can occur in some environment (like a tikzpicture) which is normally memoizable, but that command causes trouble if the author uses it in there. It is analogous to abort. TikZ's remember picture has abort set (and unmemoizable would work as well), so that those and only those pictures with remember picture get aborted.