Trace and friends have to evaluate the expression being traced, if I am not mistaken.

@Leonid Ugh... did you read the linked question/answer and evaluate the logic of this? If you did and this (and step) is a failure I'm crushed. EDIT^2: still seems to be working right; I'll start considering unusual patterns.

No I did not yet - my comment was just based on the general considerations. I will look into it.

Yes. For example, here: valueQAlt[{b[1]}] - it leaks evaluation.

@Leonid I am sorry, how is {b[1]} an object that could have a value?

This is not the point. The point is that we should be able to feed any expression into valueQ, and be sure that it won't be evaluated. Imagine using valueQ inside e.g. Cases, something like Cases[Unevaluated[expr],sub_/;valueQ[sub]:>HoldComplete[sub],Infinity]. In other words, at least in my interpretation, this question is in the first place about making valueQ with a well-defined and general evaluation semantics.

@Leonid then I have fundamentally misunderstood the issue at hand. I shall rewrite this to address it (hopefully successfully).

Well, take some g[b[1]] with undefined g in place of a List, if you wish - it then will make sense also in more narrow context.

@Leonid please test the updated function; thank you.

@Leonid I think I am coming to understand the deeper use of ValueQ (which I must say I cannot recall using in any but the most simplistic ways). Let me see if I understand this finally: valueQ[{f[1]}] should return False, but not leak Print[1]. Is that correct?

@Leonid no that doesn't seem right either, judging by ValueQ. I'm rather befuddled by all this, unless my current function in fact is correct. I'll come back to this later when I might have a fresh perspective.

This is what I am thinking myself about at the moment. If valueQ[{f[1]}] should give False, then your approach does not seem to work. But, alternatively, we may define the semantics of valueQ such that it gives True whenever an expression at all evaluates if we let it - in this formulation, your approach works and my does not. So, the question is, which semantics is the "right" one. I would think that valueQ[{f[1]}] should return False, but this is not that clear-cut to me. And, your approach is both more elegant and more general than mine. So, I don't know, really.

@Leonid ValueQ[{f[1]}] returns True -- since the OP states: "Is it possible to implement a more robust version of ValueQ, which does not evaluate its argument in an unexpected an possibly dangerous way?" I believe that the answer function should also return True; mine now does.

Here is a problem though: ClearAll[f, g];f[x_] := False;g[x_] /; f[x] := x^2;valueQ[g[1]]

@Leonid I don't think that ValueQ very well defined. Do you have to to join a chat right now to try to hammer out some details?

Yes, I agree - ValueQ is not quite well-defined

Sorry, I just had to leave then and did not see your comments

Hello.

Hello. Sorry for leaving without a note

Actually, your solution is the only sensible general definition of ValueQ in Mathematica. The problem is that Mathematica evaluation semantics is very different from more usual languages, while it tries to imitate them (function calls). It is this imitation that is a problem

(I am waiting until you tell me you are done writing.)

I just want to add that I agree with you that in general there is no way to totally prevent evaluation leaks

done now

@Rojo Hello

Hello

Okay. I see three options here. One is the "sandbox" mentioned above -- allow evaluation to take place, but have it done in a shielded environment of some kind (separate kernel) where it will not have a permanent effect.

I posted a question for you two on a comment

Hello Rojo. Let me look.

An InheritedBlock

You probably can't prevent evaluation leaks when you mess with system symbols, in which case you can try to at least stop them at one step with Spartacus methods...

and prevent permament effects somehow

But as to user symbols, Leonid's way doesn't leak

Seems reasonable to mix the two options (and hopefully it doesn't take too long to run to be of practical use in big lists or loops)

@Rojo I found you comment. I would rather deal with that separately. Also, I would like to try to work out a few things with Leonid before adding another member to the party. It is not that I do not respect you, but I am having a hard time keeping this in my head as it is. Would you mind being an observer for a few minutes?

Oka

As I was saying, option 1: Sandbox.

@Rojo 1. My method also can leak: f[x_]:=(Print["*"];False); g[x_]/;f[x]:=x^2, but IMO those who place side effects in conditions deserve some punishment. 2. Agree with @Spartacus - on your comment - let's deal with that later (it's not hard though)

@Spartacus Ok, sandbox also crossed my mind. This has problems however

Option 2: if any evaluation takes place, consider it to be an expression with a value and return True. This is what I believe the present form of my function does. Let us call this valueQ2, and the version with TraceDepth -> 1 valueQ1.

@Spartacus I think your option 2 is the only sensible and general way to define ValueQ in Mathematica, but the question is, how useful it is.

Option 3: allow the leaks necessary to see if the entire expression has a rule attached, perhaps doing fire-control somehow. This is what I believe valueQ1 does. It still avoids a number of the leaks from ValueQ and seems to generally return the same result.

(pardon my slow typing)

Logically what do you make of a case like f[1] = 5; x = 1; valueQ[f[x]] ?

And then what if x = 2 ?

I think that there may be a better way for option 3 (regarding leaks), if we combine our approaches. Which option to choose also should depend on the priorities: if the priority is to avoid leaks at all costs, it's one thing, if not - another.

okay, so there are the three options; you already stated support for #2, but that disagrees with the built-in function.

If x=1, then True, otherwise False. I know that this disagrees also with my function.

As to a built-in, I would not care about it much

Interesting. I believe that will require evaluation, unless you want to re-implement the engine.

one moment, let me check something

Conceptually, my suggestion would be to follow the evaluation process but exclude side effects from it. It is easier said than done.

That's an understatement.

I once wrote a custom evaluator which does some of it, but that seems to be a wrong road to take in this case, since you can never guarantee that it will give results identical to the full evaluation.

A sandbox options also crossed my mind. There are technical problems with it though, since Mathematica seems to lack clean mechanisms for that. Plus, imagine the overhead of copying expression trees (dependent symbols, their definitions, etc)

Actually, you probably can replace side-effects with local rules which take effect for the evaluation of everything after the "assignment" took place.

But what about Defaults, Options, etc. That'd be harder

@Verbeia Hello

Yes, I agree. Frankly I don't see the point. My own use of ValueQ has been very limited, to things like checking if a key is in a hash (of the form hash[key]).

I would like to get a better sense of the deeper usages of ValueQ where all this becomes an issue.

Hi, was just popping in, I will go back to the main room now, this is way over my head :)

I doubt that, but thanks for stopping by. :-)

Rojo and Verbeia, do you have thoughts about that specific question?

@Spartacus Well, I think, this is an important conceptual question when we think about the core language. But I agree, from the usability viewpoint.

I also think that your function is good in that it answers a well-formulated and general question: will a given expression have any non-trivial evaluation steps? OTOH, it has other limitations, e.g. this valueQAlt[Unevaluated[Print["*"]]] gives True, while, from the user's viewpoint, it probably shouldn't

which function are you calling valueQAlt?

@Spartacus Sorry, I had to rename yours to this, to compare with mine (I did not use Block)

I know, I mean what I call valueQ1 or valueQ2? That is, with or without TraceDepth -> 1? I suppose you mean without in that case.

I just want to make sure we're "on the same page."

I meant, without TraceDepth (I actually think that TraceDepth->1 is not a good idea)

Please tell me why (and if relevant compared to what).

Because it is not clear which purpose does it serve. It leads to evaluation leaks, and it is not clear which question does it answer unambiguously. The one without TraceDepth is better in two respects: it does not leak evaluation, and it answers (mostly, but perhaps not always, as my example with Unevaluated shows) a well-formulated question of whether an expression is going to at all evaluate non-trivially

What do you mean by non-trivially or trivially?

I mean, it is always correct, since Unevaluated is stripped in the evaluation process and reconstructed back if nothing else changed, so in a sense evaluation involving Unevaluated is always non-trivial, but that may not be apparent from the user's viewpoint (since the user only sees the end result with Unevaluated recovered back)

You're talking about the same version of valueQ that's on Spartacus answer, right?

@Rojo non-trivially probably should mean that there was at least one global rule or attribute applied, or Unevaluated stripped and recovered (the non-triviality of the latter is technically correct but debatable from the user's viewpoint)

@Rojo Yes, about that version

@Rojo I am calling that valueQ2 for reference.

Leonid, what about a case such as Q["a" + "b"] ?

But that version gives me False for {2} and True for Hold[2]. How can the latter have a less trivial evaluation?

@Rojo and there is another example.

Each of these has its issues I feel.

@Rojo I get False for Hold[2] with it

@Leonid I get True! Let me make and copy the Trace

Since I am using Trace as the core, this is what is taking place in my function:

TracePrint[Hold[2]]

@Rojo the problem is that "triviality" can be generally and unambiguously defined only with respect to Mathematica evaluation, and for the end-user it may often seem "trivial". This is exactly the point I am trying to make.

@Mr.Wizard, @Rojo sorry, I got True as well. I was using my function initially :)

because of this and the way I wrote my function, this counts as a form of evaluation and returns True.

(this topic is confusing, as you can see)

@Spartacus Yes, and that is exactly my point! The "triviality" for us and for mma can differ very much

Right. So the question becomes one of usage I believe.

I still feel there is use in valueQ1, even if it leaks (and it appears to leak less than ValueQ). Do you disagree? Do you think it is rough but can be improved?

Yes, and nothing tells us it will mean the same in future versions

I do disagree, because your function with TraceDepth still triggers the first evaluation, which may have nothing to do with what is being questioned (as in the example f[x_]:=False; g[x_]/;f[x]:=x^2), and it leaks evaluation

"which may have nothing to do with what is being questioned" -- will you elaborate on that, please?

I rather think that your version without TraceDepth might be possibly improved. Perhaps, we may be able to combine my and your approaches, although at the cost of a loss of generality of your approach.

Again, interesting. What do you have in mind specifically? Or, are we not yet to specifics?

valueQ1 is yours Spartacus, and Q1 is Leonids, right?

@Spartacus Yes: if I want to know whether or not g[x] would evaluate to something other than itself, I could not care less about the mechanism, such as "well, it does not, but you know, this indidious fucntion g..."

@Spartacus I have to think about it, can not suggest a specific solution yet. I just have a feeling that this may have some potential

@Rojo I am sorry for the confusion; I am callings Leonid's valueQ, my first version with TraceDepth valueQ1 and my second without valueQ2.

Ok, ok

@Leonid alright, thank you.

sorry, insidious

and function

the idea of a permanent chat transcript (which we cannot edit) is evil, no? :^)

Since we are now considering alternatives, have to return to Rojo's InheritedBlock. Forgetting entirely about memory and efficiency, what do you think would happen if we tried to InheritedBlock everything in Names[]?

@Spartacus It is evil :) Why don't we return to this topic in some while? If I am able to combine our ways, I will post it, and if you do, please do post.

I think that so far I only understood the behaviour of the built in version

and trying to emulate that as Slabolcz sked

I think that Spartacus way can be improved to behave like the built in but at least avoiding some evaluation... And Leonid's approach can avoid all evaluation on some symbols...

@Rojo I am not sure I even understand that.

so perhaps mixing both we get something that behaves like the built-in and leaks less

@Leonid did you leave already?

I wouldn't worry about leaks in conditions at all unless they changed the behaviour of the function... It's probably always a bad idea to have conditions that can create trouble

@Rojo The built-in one took a very cavalier approach to evaluation, but it answers the question, I'll give it that.

No, just give me a minute

@Rojo That's what I said - those who put side effects in conditions, deserve to be punished :) But, I am guilty, I do (or, more precisely, did) that myself a few times.

Actually, the problem we are trying to resolve is the other face of what is known in CS as continuations

So, should the language support the callcc function (call with current continuation), and this would likely be a solution. Continuation is more or less a snapshot of the call stack, cast as a function. This function contains the "rest of the computation", and can be called from arbitrary other function.

Ok, all, I have to leave now. Will drop by later.

I'm finding 19110 symbols in Names["`"], I put them all in an InheritedBlock

Haha

It's struggling and making noises

Installing stuff from wolfram server

Launching kernels

"Rule called with 6 arguments, 2 arguments are expected"

Haha, I aborted it, got scared

@Spartacus, not a good idea

@Leonid, bye

lol -- sounds like fun.

Rojo is there anything else you want to say? I am also going to work on something else of that is OK.

Sure, have fun

@Mr Wizard Hello

Hello Leonid. I have another idea I am working on but I think I need someone who understands things like Stack[] better than I do.

I am still pursuing the use of TraceScan as a solution.

And you think that would be me ?! :)

Yes Leonid, I do.

Well, let's try.

I won't have lots of time now, though

How many minutes do you have? (please)

Or was it seconds? :^)

Well, I have may be 15-20 mins. Sorry for delay, I was relying on the sounds to tell me when you reply, but this failed for some reason.

heh, that should be enough I hope. Thanks. I'll try to be breif. I am trying to use the fourth argument of TraceScan but I have run into a problem I don't understand.

Well, I understand it even less -- what I was about to paste doesn't do what I thought it would. Darn. Give me another minute.

I never really understood that last arg either

Okay this is behaving even more erratically than I thought, or I am making a really boneheaded mistake somewhere. Let me try a different approach: what circumstances can you think of that Catch[ TraceScan[_, _, _, Throw[x] ] ] would result in:

Throw::nocatch: Uncaught Throw[True] returned to top level. >>

give me a second...

This is happening some times and not others. Sound like anything at all familiar? (sure)

Yes, sounds familiar

Can you give a non-trivial example for both cases (when is does and does not happen)?

I think it may have something to do with Print itself. Just one more moment -- I am sorry for taking your time when I was not ready (I thought I was.)

Okay, try this:

i = 0;
Catch[
 TraceScan[
  Print[i, " ", ##] &,
  Sin[x^2] + b,
  _,
  If[i++ == 3, Throw[True]] &
  ]
 ]

I see what you are aiming at. I used this technique as well.

i = 0;
Catch[
 TraceScan[
  Print[i, " ", ##] &,
  Print["this is strange"],
  _,
  If[i++ == 3, Throw[True]] &
  ]
 ]

Seems to work for me

In the second case I have the Throw/Catch error. It appears to be linked to Print. You do not get this?

Oops, sorry, saw the first one only

Give me a moment...

(Yes, I am trying to watch for "nontrivial evaluation" and abort if found. This is giving me issues.) Please take your time.

hi

@Szabolcs Hello

Hello Szabolcs. Welcome to the party.

Was just trying to read al the comments on MrW's deleted answer

They are all here at the top.

I think that answer adds a lot and deleting it was not a good idea, but the owner has a final word

Um... I just want to fix it first. I'll keep that form of the function in there somewhere since if you find it valuable it must be. However it is not really a good answer since it triggers on lots of odd things.

let me just say that when I asked the question I knew that valueQ[{f[1]}] would not work when based on MrW's step function. I was interested mainly in "fixing" ValueQ for situations like valueQ[head[arg1,arg2,...,argn]] only

@Szabolcs The problem is that the semantics of ValueQ is not well-defined, regarding both what we expect from it, and how it should handle evaluation (leaks)

that is true

that's what I get for posting a question at 3 AM ... :)

@Leonid what should I see there? If the point is that Throw works, it's an issue of timing. Try:

i = 0;
Catch[TraceScan[Print[i, " ", ##] &,
  StackComplete[Print["this is strange"]];, _,
  If[i++ == 5, Throw[True]] &]]

If the Throw happens in the middle of Print it goes loony.

@Szabolcs we are free, of course, do define this semantics as we want it, but the point is that we should do it before trying to implement a final version. As to the question itself, I think it was a good one

It certainly was a good question.

@Mr.Wizard Sorry, you are right

btw (unrelated) someone copied an old question of mine word by word and re-posted it here. Can you vote to close and delete? (I really don't understand why someone would do this, what's his motivation?)

@Szabolcs Yes, sure. I saw that and was puzzed, but since I did not remember where was the original question, I moved on with my stuff.

Could the Print call be messing with streams or output channels, or the stack or something that was not intended to be interrupted?

let me try that...

I should have defined exactly what this new valueQ must do. Let me go back and check again what else is the problem with MrW's soltuion, and read yours fully too

be back in a while

@Szabolcs added my vote. @Mr.Wizard It's your call now (unless you voted already)

@Leonid that appears to be working but I have no understanding of it. Do you?

(sorry, voted on what?)

@Szabolcs It is a pretty hard-to-define problem, because it looks well-defined on the surface, but actually goes very deep into the language and evaluation. I actually expressed my opinion on this in the comments above: Mathematica has a very different evaluation semantics from other languages, yet it attempts to emulate their function call semantics. It is this emulation which is problematic.

@Mr.Wizard I meant closing that dupe question that @Szabolcs mentioned

@Mr.Wizard My guess is that some evaluations (perhaps including Print) modify evaluation stack or may even use a different stack. I don't fully understand this yet, but Catch seems to be fooled because we go out of the stack it watches. This is all very interesting.

Leonid I am afraid that shields the evaluation from TraceScan making the exercise pointless.

@Szabolcs missed that entirely

Mr.Wizard Well, I can not add much more right now. This seems an interesting direction to follow though. I have been thinking for a while whether it is possible to implement continuations in Mathematica using Stack and exceptions (Throw-Catch), but did not come to a definite conclusion.

Leonid I'll leave you to explore it on your own time. I'm sure you'll have some amazing insight by the end of it. :-)

@Szabolcs that question is now closed. Honestly it looks like an error of some kind to me. Maybe and edit box became a post box somehow? User-side scripting gone wrong?

@Mr.Wizard the problem is that Mathematica Stack is really different from the call stack in traditional languages, since it is an expression stack. It is similar in some respects, and different in others. I discussed some of it in my answer to this question:stackoverflow.com/questions/4481301/…, but I don't have a complete understanding of the possibilities we have with it

@Mr.Wizard, @Szabolcs Sorry, guys, really have to go now... Later.

That was a great answer deserving of more than 8 votes.

Bye Leonid.

Bye Mr.Wizard

Szabolcs I should be going as well. It is almost 3:00AM here.

@MrWizard You can vote to delete now

@MrWizard First time my "trusted user" privilege is good for something

I don't believe that I can. Not enough rep yet. (You just had to rub it in, didn't you?)

@MrWizard You know you are going to overtake me within 10 days the most :-)

We'll see. You've been a force on this new site.

What time is it there?

Anyway, I shall be going now.

Good night @MrWizard! It's 12:30

I mean 13

@Leonid are you here now?