Peter Cordes

But anyway, my point was that 80386 is an in-order non-pipelined CPUs, very different from out-of-order exec CPUs (lighterra.com/papers/modernmicroprocessors). (Although in terms of tuning, less different than i586 Pentium, which was in-order pipelined but didn't break instructions into uops so it favoured the RISC-like subset of x86.) Dependency chains and latency don't matter on 386, each instruction fully executes before the next one starts. Multiply is relatively much more expensive on 386 than now. IDK how to summarize in a meaningful way for the OP who doesn't know asm. :/

@dirkt: The OP would need an actual 386 to test on, as they're interested in performance on retro CPUs, not on the modern CPUs that compilers are primarily tuned for. gcc -O3 -march=i386 will set the tuning options that exist to settings that are good for 386, but some things are just baked into the compiler. (I don't have a specific example of something GCC does which is sub-optimal on 386, though. It does plenty of stuff that's bad on the modern CPUs it's supposed to be optimizing for, like using indexed addressing modes even when unrolling with AVX for Intel CPUs...)

@Obie2.0 - I don't think that's what Kvothe is arguing. It's an interesting comparison. For a human, writing down copies of a book is when copyright infringement happens, whether that's from memory or by looking at the original while doing it. The copies of book embedded in a trained AI model aren't directly readable, only accessible in response to prompts. It could be argued that copyright infringement only happens when the operator of the AI-backed web site (or whatever) produces copyrighted text as an output, or when they send that to users in response to prompts.

@Fe2O3: Go ahead and edit it to uint8_t* if that would be better. Or fix your version to work with a generic T*. (Sorry I didn't notice that gotcha when editing T, T to T*, size_t... I was going to fast; meep meep :P)

Your memcmp idea is great, actually. It's much faster than the asm GCC made for my version, and Clang did a bad job, too. Correctly applying the magically-fast tools in the C library toolbox is a useful think you can do in your VW bug :P

Added an answer with some of these comments, and an attempt at portable C that auto-vectorizes. Not great with either GCC or Clang. :/

But getting the compiler to inline the SIMD code has the downside that you need to let it use at least AVX2, so it won't run on older CPUs. Unless you do multiversioning with ifunc dispatching or something, like libc does for memcmp versions that take advantage of different CPU features if present.

If you care about large arrays, you could apply that no-early-out strategy in chunks of 128 bytes or something. (2 cache lines, several vectors to compare + vpand together like glibc memchr does). So you could get auto-vectorized code that only load the data once, with aligned loads (relative to the start of the array at least), as an inner loop that gets fully unrolled hopefully. But getting the compiler to emit vpmovmskb / cmp eax, -1 / je loop_top` might be tricky, and extra overhead in the reduction matters if this is only an inner loop for small chunks. :/

Also related: Is it safe to read past the end of a buffer within the same page on x86 and x64? - a scalar loop that would stop at the first mismatch doesn't touch later elements, so it would be legal to call it with a shorter array than the size, unless you use T *arr[static size], but compilers don't use that hint to vectorize. Again, always reading every element with no early-out avoids that problem. And How fast can you make linear search? is an example of applying the no-early-out technique.

One way to work around it is to write C that always touches every element, with no early-out. e.g. mismatches |= (arr[i] != arr[0]); or allmatch &= (arr[i] == arr[0]). That's bad for large arrays with a mismatch near the start, but might be acceptable in some cases. See the how to auto vectorization array comparison function for an example of auto-vectorization of counting matches.

Unfortunately most compilers can't auto-vectorize loops whose trip-count isn't computable ahead of the first iteration. i.e. with a data-dependent loop-exit condition, such as a strlen or memchr loop. Or this case, like memchr but looking for the first mismatch. ICC Classic (not LLVM-based ICX) can do that in some cases, and I think I read that the most recent GCC or Clang version can do it.

In usage like this (C++ templates, C generic functions, or examples/pseudocode), T is the same type in both appearances, which is very bad for T size. Just use size_t size because you don't want the size-type to be parameterized. If you wanted to use a different wildcard type, use T* arr, S size.

Anyway, your overlapping memcmp trick could probably go somewhat less than half as fast as optimal SIMD machine code on arrays that are already hot in L1d cache, on typical modern x86. For arrays farther away from the CPU, probably still limited by cache/memory bandwidth getting the data to L1d.

The other standard-library function that could do the job is strspn, at least if your first byte isn't 0, the C string terminator. But it normally only checks one byte at a time after making a lookup table since it takes a whole set of accept-characters. It can use SSE4.2 pcmpistri for small-enough accept-sets, but that's still not as fast as a simple byte compare. Oh, and it works on C strings so would need a terminator or mismatching byte following the array. There's no memspn even in glibc.

isSameByte( T arr, T size ) - huh? Surely you meant T *arr. But why would you want the size to have the same type as the array elements, rather than size_t? Yes, partially-overlapping memcmp is a good way to leverage the standard library hand-optimized SIMD compare code. Modern CPUs have good unaligned load support, at least on x86-64. For 32-byte vectors, one of every four will be split across a 64-byte cache-line boundary, if your main array is aligned by 32. Not great, but not a problem unless data is hot in L1d already.

If I hadn't already thought of that, it could potentially spur someone to look at that in more detail as the possible cause. I don't know a good way to investigate other than looking at uiCA traces based on a simulation of the real HW. Hardware performance counters can't (usefully at least) record a sample for every instruction, and even then it would be for uops_executed.port_0 when it dispatches, not at issue.

It also says ICL's loop buffer (LSD) can only deliver 4 uops / cycle. It's 5, the width of the issue/rename stage. Oh, it also says "The NOP creates a more favorable pattern for Ice Lake's dependency management hardware", which is even closer to what I think the right answer is. So maybe 4/10 or even 5/10. But it never mentions resource stalls or uop allocation port ports and losing cycles on dep chains.

@JonathanReez Yeah, maybe 3/10 for that. I think the correct answer is almost certainly the dependency chains and losing cycles on them due to resource stalls from imperfect uop allocation to ports. It does talk about the dep chains. But then says "The strategically placed NOP creates a breathing space between instructions that read the loop counter and instructions that modify it". There's no breathing space; we still want the loop to run at 1 iteration per cycle.

(Since this is a microbenchmark for understanding CPUs, not a program we want to tune. That's something that kept tripping up ChatGPT I think.)

@JonathanReez It would save time in understanding why something is running slower than you expect based on the normal simple parameters like uop throughput bottleencks and best-case latency bottlenecks. It doesn't shed any light on whether an AI would be helpful at knowing how to change a function (or whole program) to make it more efficient; it could perhaps do that without formulating an English answer to questions like this.

e.g. it's proud of itself for "focuses purely on front-end delivery stalls in the LSD when the loop µop count isn’t a multiple of 5.". I suspect the difference is in uop allocation to execution ports when the loop size matches the pipeline width vs. when each issue/rename/alloc group contains about 1.25 iterations, so we get different allocations for different uops, and sometimes miss a cycle of executing one of the loop-carried dep chains, from which we can't catch up.

@JonathanReez Still useless. "The major mistake in v1 was treating this as a general front-end inefficiency when the actual issue was a specific LSD delivery quirk on Ice Lake." - nope, we know that ICL's LSD can deliver a 4 or 5 uop loop at better or equal than 1 iter / cycle. The question is why something else is running slower than that. ChatGPT has just focused in on a complete misunderstanding of the question, or at least on an aspect of the answer which doesn't explain anything.

Anyway, 1 or 2/10 for some semi-related correct facts. 0/10 for anything resembling a plausible explanation of the performance effect. For someone who didn't know better, they could be pointed in totally the wrong direction if they didn't realize ChatGPT was spewing nonsense. So still a perfect example of something ChatGPT totally faceplants on.

Also totally misses the point for Skylake; there, the relevant test is a 3-uop loop padded up to 4 or not. But it's still talking about a 4 uop loop padded up to 5, running at 1.25 cycles per iteration (with optimal uop-cache throughput, or I think 1.33 c/iter on Broadwell or Haswell due to an LSD bottleneck with loops that aren't multiples of the pipeline width. ChatGPT might be right about that, but that's the wrong question.) Again still trying to give general optimization advice like unrolling loops, not explain this microbenchmark effect.

@JonathanReez: In summary, an incoherent repetitive jumble of facts and mis-stated non-facts. And it never says anything that would explain the performance effect even if it were true! It keeps wanting to talk about how to get more useful work done (by loop unrolling), rather than come closer to 1 iteration per core clock cycle for the actual dec/jnz loop branch plus whatever loop body. (update: It does at eventually mention the limitation of 1 loop branch executed per cycle.)

... Or maybe mov-elim isn't relevant; later tests reproduced an effect with just inc and nop uops. Anyway, ChatGPT wanted to optimize for the wrong things, like having the ROB not get full or not running out of physical registers. We'd normally want the front-end to run ahead of the back end; there's no reason to expect the back-end to run slower just because its buffers are full of work. ChatGPT seems to make this implicit assumption and never explains itself. It keeps talking about utilization of the front-end (5 slots), but that doesn't matter if the front-end isn't the bottleneck

@JonathanReez: 0/10. Maybe 1/10 at best. Full of tangential facts, and some non-facts misinformation (like that aligning the top of a loop by 16 is relevant for macro-fusion of dec/jnz at the bottom - actually what matters is that the dec/jnz itself isn't split across a cache-line boundary, and loop size is arbitrary.) Also, it claims that Ice Lake does mov-elimination. For GPRs, this was disabled by a microcode update in 2020-nov (realworldtech.com/forum/?threadid=200635), and might be the relevant difference between a 3 uop loop on Skylake vs. a 4 uop loop on Ice Lake.

@JonathanReez: Stringify template type for inline assembler is a syntax / toolchain problem which AFAIK is impossible to solve the way the OP was hoping. The deleted AI-generated answer has showstopper problems. Just for the record, I'm not claiming the performance questions can't be answered by AI; probably someone who understood the question and the general shape the answer should have could give the right prompts to get an answer. But whatever low-effort users do produces horrible results, probably just pasting the whole SO question as prompt?

@JonathanReez: I checked my flags raised to find some AI answered badly; C# and SIMD: High and low speedups. What is happening? attracted a dozen bad AI answers until a semi-related answer to one aspect took it off the list of high-score-unanswered questions that AI exploiters target. I posted some ideas in comments, but never wrote up a full answer. Also Why does a NOP (as a 5th uop) speed up a 4 uop loop on Ice Lake?. I don't know the answer, and wasn't able to repro on a different microarchitecture.

@JonathanReez: The volume of bad answers has dropped a lot in the past half year; I don't remember the last such answer I've seen about explaining a performance effect or about how to manually vectorize something. But I know I haven't seen a useful one from the naive low-effort low-understanding way people use it to churn out obviously-AI SO answers. Always just generalities and/or misunderstanding the question, or making stuff up. Especially on "tricky" questions that remained unanswered because the question and/or answer is complicated.

@GregBurghardt: 100% this. ChatGPT invents false statements presented in well-written ways that make them look plausible if you aren't an expert reading carefully. In Stack Overflow's [cpu-architecture] / [simd] tags, every ChatGPT answer I've seen has been useless, or worse: misleading and wrong, asserting false statements as the premise for its argument (detailed example). (Of course those are low-effort trolls that asked it to answer the question itself, not improve existing "expert" text, but those errors could creep in anywhere).

Anyway, 1 or 2/10 for some semi-related correct facts. 0/10 for anything resembling a plausible explanation of the performance effect. For someone who didn't know better, they could be pointed in totally the wrong direction if they didn't realize ChatGPT was spewing nonsense. So still a perfect example of something ChatGPT totally faceplants on.

Also totally misses the point for Skylake; there, the relevant test is a 3-uop loop padded up to 4 or not. But it's still talking about a 4 uop loop padded up to 5, running at 1.25 cycles per iteration (with optimal uop-cache throughput, or I think 1.33 c/iter on Broadwell or Haswell due to an LSD bottleneck with loops that aren't multiples of the pipeline width. ChatGPT might be right about that, but that's the wrong question.) Again still trying to give general optimization advice like unrolling loops, not explain this microbenchmark effect.

@JonathanReez: In summary, an incoherent repetitive jumble of facts and mis-stated non-facts. And it never says anything that would explain the performance effect even if it were true! It keeps wanting to talk about how to get more useful work done (by loop unrolling), rather than come closer to 1 iteration per core clock cycle for the actual dec/jnz loop branch plus whatever loop body. (update: It does at eventually mention the limitation of 1 loop branch executed per cycle.)

... Or maybe mov-elim isn't relevant; later tests reproduced an effect with just inc and nop uops. Anyway, ChatGPT wanted to optimize for the wrong things, like having the ROB not get full or not running out of physical registers. We'd normally want the front-end to run ahead of the back end; there's no reason to expect the back-end to run slower just because its buffers are full of work. ChatGPT seems to make this implicit assumption and never explains itself. It keeps talking about utilization of the front-end (5 slots), but that doesn't matter if the front-end isn't the bottleneck

@JonathanReez: 0/10. Maybe 1/10 at best. Full of tangential facts, and some non-facts misinformation (like that aligning the top of a loop by 16 is relevant for macro-fusion of dec/jnz at the bottom - actually what matters is that the dec/jnz itself isn't split across a cache-line boundary, and loop size is arbitrary.) Also, it claims that Ice Lake does mov-elimination. For GPRs, this was disabled by a microcode update in 2020-nov (realworldtech.com/forum/?threadid=200635), and might be the relevant difference between a 3 uop loop on Skylake vs. a 4 uop loop on Ice Lake.

@JonathanReez: Stringify template type for inline assembler is a syntax / toolchain problem which AFAIK is impossible to solve the way the OP was hoping. The deleted AI-generated answer has showstopper problems. Just for the record, I'm not claiming the performance questions can't be answered by AI; probably someone who understood the question and the general shape the answer should have could give the right prompts to get an answer. But whatever low-effort users do produces horrible results, probably just pasting the whole SO question as prompt?

@JonathanReez: I checked my flags raised to find some AI answered badly; C# and SIMD: High and low speedups. What is happening? attracted a dozen bad AI answers until a semi-related answer to one aspect took it off the list of high-score-unanswered questions that AI exploiters target. I posted some ideas in comments, but never wrote up a full answer. Also Why does a NOP (as a 5th uop) speed up a 4 uop loop on Ice Lake?. I don't know the answer, and wasn't able to repro on a different microarchitecture.

@JonathanReez: The volume of bad answers has dropped a lot in the past half year; I don't remember the last such answer I've seen about explaining a performance effect or about how to manually vectorize something. But I know I haven't seen a useful one from the naive low-effort low-understanding way people use it to churn out obviously-AI SO answers. Always just generalities and/or misunderstanding the question, or making stuff up. Especially on "tricky" questions that remained unanswered because the question and/or answer is complicated.

@GregBurghardt: 100% this. ChatGPT invents false statements presented in well-written ways that make them look plausible if you aren't an expert reading carefully. In Stack Overflow's [cpu-architecture] / [simd] tags, every ChatGPT answer I've seen has been useless, or worse: misleading and wrong, asserting false statements as the premise for its argument (detailed example). (Of course those are low-effort trolls that asked it to answer the question itself, not improve existing "expert" text, but those errors could creep in anywhere).

Perhaps a good example would be how Fireball or Lightning Bolt interacts with AMF: the AMF carves out a chunk of area where damage doesn't happen, but the spell effect exists on both sides of the AMF after being generated only on one side. With Lightning Bolt, it's clearly not going around, just through but without manifesting any effect while inside.

In the case of this Q&A, there's no obvious reason for a news organization to make this up, and the underlying question is a concrete enough fact (scurvy diagnoses) that shouldn't be too hard for a journalist to check. So citing news orgs and their overall reputation is reasonable. But it's also reasonable to choose wording that doesn't imply everything published by reputable news orgs is always 100% true. And/or about the reasons why this case is particularly believable. You don't need to edit, but it would be a better answer if you did.

@FranckDernoncourt: But yes, as your answer points out, apparently some people think extra expense should have been spent to make it less of a solid wall somehow, exactly because of accidents like this where a plane probably wouldn't have gone all the way to the water even with no barrier, and a barrier resulted in serious impact causing loss of life and more damage to the plane. Oh, and it was concrete? Not just a pile of earth that might give some under impact. That's pretty far from ideal.

@FranckDernoncourt: Piling up some earth was presumably cheaper and easier long-term. (e.g. for maintenance access you just walk up it). What would they do instead? A specially designed structure of wood or metal that was just strong enough to not fall down on its own even in strong winds, but which a plane could knock over in an emergency? Or a giant sand dune to act as a brake on runway overshoots? Or a longer more gently-sloped piece of earthworks so a plane could go up it instead of hitting a wall? All of these are cost tradeoffs, and presumably they gambled on cheap+easy, and lost?

@Vexorr: use @ username (without the space) to notify people when you reply to them. You get notified by any comments under your question, but other people won't see your replies unless they happened to check back.

@rexkogitans: Yeah, I know it doesn't. I think Windows has multiple APIs for file access, and backup programs can at least open and read files that are already open and running. I seem to recall that system-update programs have to queue replacement of running executables to be done by the OS during reboot. Because yeah, you can't unlink or even rename a running executable on Windows, AFAIK.

@rexkogitans: note that truncating a running executable does cause a problem; see my previous comment re: SIGBUS on /tmp/dd if=/dev/zero of=/tmp/dd count=102400B doing open(O_TRUNC). If you unlinked a file and created a new one, you'd be fine, but that couldn't "shred" the blocks that were holding the old file.

Of course none of this would apply if writing to the block-device holding the filesystem. Then, as long as the executable and libraries are hot in disk cache, the kernel won't notice anything because raw disk writes aren't coherent with what the filesystem driver is doing; it assumes nobody else will be messing with a filesystem while it's mounted.

Anyway, > /tmp/dd is obviously fatally flawed. of=/tmp/dd is much more interesting. Also, maybe you meant to read from /dev/zero (infinite stream of 0 bytes) rather than /dev/null (empty file). With strace /tmp/dd if=/dev/zero bs=102400B of=/tmp/dd, I get a successful openat(AT_FDCWD, "/tmp/dd", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3, but then the next thing is a SIGBUS when the executable tries to demand-page more of itself, but the file backing the mappings for .text/.rodata and .data has now been truncated. It's the same as mmap and truncating the underlying file.

/tmp/dd if=/dev/null bs=102400 > /tmp/dd gets the shell to open("/tmp/dd", O_TRUNC|...) for the redirect after fork but before execve of /tmp/dd. When I try the same thing with strace /tmp/dd ... > /tmp/dd, I get -1 ENOEXEC (Exec format error). But bash reports no error because it treats an empty file is a valid shell script. strace bash -c /tmp/dd reports the same failure of execve, but then Bash makes open and read system calls.