last day (16 days later) » 

18:43
2
Q: How can transpilers benefit from compiler optimizations which are compatible with semantics, if other "optimizations" in the target language aren't?

supercatConsider the following function: unsigned test(unsigned x, unsigned factor, unsigned mask) { unsigned i=1; while((i & mask) != x) i*=factor; return i; } Proving that in the absence of optimizations, the function would either return a value with no side effects, or not return at all, wo...

@kaya3: C allows compilers to artificially generate arbitrary side effects in scenarios where a program would execute a side-effect-free loop, and clang will do precisely that. GCC last I checked didn't seem to do that in C, but does so in C++.
@kaya3: Slight mistake in the code I sketched. Code should have tested (i & mask) against x.
Adding dummy side effects would achieve correct semantics, which should be the first and foremost goal of any kind of compiler, transpiler, interpreter, or other such language tool. It would undermine the usefulness of the potential optimization, but not as much as would having a compiler perform "optimizations" that break what would otherwise be easily verifiable memory-safety invariants.
@kaya3: I've added another paragraph; are things clearer now?
@kaya3: Clang interprets the permission to "assume" that loops terminate as freedom to behave in arbitrary fashion if such assumption is violated, and gcc ignores that provision except in C++ mode. Whether or not the authors of C11 intended such nonsense, no later standards have changed the text to unequivocally forbid it.
@kaya3: If one weren't concerned about optimizations, the transpiler could ensure correctness by adding dummy side effects. The question is whether one can avoid having loose semantics in the target language completely undermine any benefits that compiler "optimizations" in that language might otherwise be able to offer.
@kaya3: A compiler which inserts dummy side effects could not achieve all of the optimizations allowed by the source language unless it performed such optimizations by itself, but it could still process the program correctly. The fact that an implementation doesn't achieve all possible optimizations isn't necessarily a "problem". On the other hand, one of the supposed advantages of transpilers is that they can exploit optimizations in the back end. I fail to see how that is possible unless the semantics of the source language are as severely constrained as those of the target.
@kaya3: If the transpiler adds a dummy side effect to the loop, then the loop would have defined behavior whether or not it terminates. The resulting machine code would likely be less efficient than if semantics could have been achieved without the dummy side effect, but correctness would be achieved.
@EldritchConundrum: What would really be most helpful would be if the Standard could draw a distinction between specialized optimizing dialects which are only suitable for a narrow range of tasks where "anything can happen" UB would be an acceptable response if some bit patterns are received as inputs, and general-purpose dialects which could be suitable for a wider range of tasks where "anything can happen UB" isn't an option, and transpiler writers could target the latter without having to add all sorts of junk code which blocks useful optimizing and maniacal "optimizing" transforms alike.
Would you like to edit the title to focus on the particular question in the post, about possibly infinite loops? Currently it seems over-broad.
So it seems to me that we ideally want to achieve three things: (1) if the loop always terminates and the C compiler can prove it, it's deleted; (2) if the loop does terminate but the compiler can't prove it, it executes without unnecessary overhead; (3) if the loop does not terminate, then an infinite loop is actually executed. Adding a dummy side effect gives you (3), but breaks (1) and may add some overhead to (2).
One idea I had was to add (let's say) a 64-bit counter, incremented on every loop iteration, and have a dummy side effect (e.g. read a volatile variable) which only executes when the counter rolls over. Then if the compiler can prove the loop terminates in less than 2^64 iterations, it can delete it, so we make some progress toward (1), and we still have (3). But if the compiler can't prove termination, then it will have to actually generate code to increment and test the counter, which adds some overhead in case (2).
Another idea is to execute atomic_store_explicit(&dummy, 0, memory_order_relaxed); on each iteration. This counts as a side effect, so an infinite loop is not UB. But in theory, the compiler is allowed to hoist it out and just do the store once, since there's no
@EldritchConundrum: My actual interest is broader than infinite loops: can a transpiler for a language which would allow optimizing transforms to affect program behavior in cases that aren't characterized as "anything can happen" UB, receive any benefit from a back-end optimizer based on "anything can happen" UB. I think the loop example best illustrates the concept of optimizing transforms yielding program behavior which is inconsistent with treating a program as a sequence of steps, but would not impede a program's ability to e.g. maintain statically-verifiable memory-safety invariants.
@NateEldredge: Hmm... I'd like that atomic store idea, except for the facts that (1) it would only be usable on implementations that support atomics, and (2) I don't think Standard would unambiguously treat the store as a side effect that needed to be treated as occurring within the loop. Neither clang nor gcc seems to be equipped to recognize situations where a "physical" operation may be hoisted but some semantic effects need to be treated as though they happened in proper execution order. What makes the clang/gcc behavior objectionable is that they...
...consolidate an if test which is performed after the loop with the test of the loop body condition, without recognizing that this creates a control dependency between evaluation of the loop's exit condition and the body of the if. I suspect the reason clang and gcc don't keep track of such dependencies is that doing so would create phase order dependencies which compiler writers have spent the last 20ish years in a misguided quest to avoid. Compiler theory has gotten attached to the idea that if T1(X) and T2(X) would both be valid ways of transforming program X, then...
@supercat: The C and C++ standards are quite clear that "atomic operations" count as forward progress, so that a possibly infinite loop which performs an atomic operation can't be assumed to terminate and isn't UB. This would even appear to include, e.g., an atomic load whose value is discarded, which could otherwise be optimized away (even though current compilers don't). An interesting question is whether that also includes fences. clang will optimize out an infinite loop containing a fence, but gcc won't.
...T1(T2(X)) and T2(T1(X)) should both be valid as well. If T1(X) is "eliminate the loop" and T2(X) is "eliminate a conditional test which would always be true if the loop exit condition is satisfied" would mean that T1(X) and T2(X) to be valid, then T1(T2(X)) and T2(T1(X)) would eliminate both the loop and the 'if'. As much as I'd like the idea of using the atomic store as described, I'd be skeptical as to whether clang or gcc could process it efficiently without reintroducing the problems it was added to solve.
18:43
Oh, my earlier comment got truncated. I was going to say that although in principle the compiler could hoist out an atomic store, in practice they don't, as you suspected.
@NateEldredge: I suspect that as soon as clang or gcc gains the ability to hoist the store instruction, it will cease to treat the store as representing "forward progress" within the loop. Compiler philosophy has latched onto the idea that what rather than recognize that it should sometimes be hard for compilers to determine with certainty which of several mutally exclusive transforms is most useful, but that heuristics should be good enough for most practical purposes, it's better to characterize as "defective" any program that would require treating transforms as mutually exclusive.
@supercat: If it does then that will be a clear violation of the standard, and should be unambiguously considered a bug. But what seems to me more likely is that it would only hoist it out if the loop is provably terminating.
@NateEldredge Clang and gcc have for years had behavioral anomalies which are inconsistent with what would seem like a normal straightforward reading of the Standard. Every once in a while one gets fixed, but it's unclear which things the maintainers view as faults in the compiler or as faults in the Standard. Look at godbolt.org/z/9cGdzG8M3 and ask whether anything in the Standard would allow code to...
...(for either function) store 2 to y[0] but return 1 in the circumstance where x[] is a single-element array that happens to immediately precede y[] in memory? Both clang 17 and 18 will behave that way for test1(). For test2() the older clang would observe that the only way the comparison could be true would be if p was equal to x+1, but the newer one actually performs the multiplies as written.
 
2 hours later…
20:37
Another thought I had was printf("");, which compilers know is a no-op. It looks like gcc and clang both handle this properly when compiling C: godbolt.org/z/1eocaGPrM. Unfortunately, in C++, they both delete a loop while (b) printf(""); which I think is clearly a bug. I'll plan to report it.
Oh, it's not a good test in C either, because they also don't delete while(b) { }.
20:58
I did notice that gcc and clang both support -fno-finite-loops which I think gives the semantics you are looking for.

  last day (16 days later) »