CMQ is it true that it is faster to sum an array by summing the numbers at even induces, then the ones at odd invoices, then adding the two results? If so, why?
"a simple out-of-order summation method, that adds terms with even and odd indexes separately, then adds together these two partial sums. Out-of-order summation provides more potential for instruction-level parallelism"
@Simd it is faster to sum an array by summing the numbers at even indexes into one sum, and at odd indexes into another, then adding – but you have to do the two sums in parallel, rather than one after the other, so that you can use the same memory reads to contribute to both sums
with SIMD instructions (are you named after those?) it can even be faster to do 4/8/16 sums in parallel, depending on the data type you're operating on and the processor you're using
@ATaco on modern processors, multithreading and SIMD are both normally a mostly free 4× speedup on problems they're appropriate for (and you can use both at once for a free 16× speedup)
and can be more depending on processor model and (for SIMD) the width of the data types you're using
compilers can sometimes manage to SIMD things on their own but they're still not very good at it
the worst part is, you can SIMD directly in languages like C and Rust, but the way you do it is so obscure that doing it directly in asm is actually easier to understand
(this is the primary reason I wrote the fizzbuzz in asm rather than C)
I have been thinking about a compiler that writes SIMD code by default, using just one of the multiple data (but meaning that you could run any function simultaneously on, e.g., 4 different data) – the performance properties would be very different from a normal compiler, and probably worse unless you tried to take advantage, but it would be interesting
hmm, convolution + sum array is the same as sum array then multiply by constant, isn't it?
on another subject: someone emailed me and asked whether I knew of anyone attempting to implement GPT-alikes on Turing machines; I didn't but feel like I at least owe it to them to ask in the only places where such people are likely to be, and I guess this might be one of them?
but my guess is "unfortunately, nobody is doing that at the moment"
I was actually more negative than that – I said that you'd basically have to transpile from another language, but then watching the program would look more like "a Turing machine implementing language X" rather than "a Turing machine evaluating an LLM"
I think it might not be too bad to put a neural network evaluator onto a Turing machine – it's basically just alternating between matrix-by-vector multiplications and a nonlinear function, which is a simple enough pattern that it might be doable semi-directly, and that's probably the bulk of an LLM
but I'm not sure whether you can do neural networks properly with integers or whether you need floats
(float multiplication shouldn't be too hard to do on a Turing machine, although float addition would be annoying to implement)
still, this is the sort of thing that falls into the category of "this would be impressive if somone managed it, but it almost certainly wouldn't be impressive enough to justify the energy spent on it"
