last day (15 days later) » 

17:31
6
Q: What is the origin of executable compression?

Leo B.According to the wikipedia article on the topic, the earliest executable compressor listed is Realia Spacemaker for IBM PC (since 1982, written by Robert B. K. Dewar, SM.COM, signature "MEMORY$"). One of the pages referred by the wiki entry claims that the distinction of being the first executabl...

The Wikipedia article doesn't claim that SpaceMaker is the earliest; it's just the earliest one listed (at least among those that are dated at all). The blog post also doesn't claim it's the first executable compressor, just the first on IBM PC, and even that's only a guess, if a plausible one.
@berng I've corrected the wording.
This OS/2 Museum update, “Yep, Norton Did It” provides further evidence that SpaceMaker is the oldest PC executable compressor (but that doesn’t exclude other platforms of course).
Many system's executable-file formats have, long before 1982, provided means by which programs can reserve areas of zero-initialized storage without having to write runs of bytes in the file. If the question is about utilities which can take an executable produced by an unaffiliated build system and compress it, it would be good to make that clear.
If we extend the scope of the question to include cases where the executable is compressed, but the compressor is the process that creates the executable, and the decompressor is the system process that launches the executable, then the first of these goes way back into the 1960s or 1950s, as soon as these were executable files.
17:31
@WalterMitty Please see the updated question.
@supercat Please see the updated question.
Also note that, historically, the incentive to package up files differently started with telecommunications. First you needed to make sure that binary files were safe to transfer (expanding them instead of compressing them), and after LZW was published and patented, compression took off. This is very different from keeping uninitialized sections out of executable formats, which is what SpaceMaker does, and which other executable formats don't need because it's baked in. So be careful not to compare apples and oranges when looking for "earliest".
These days compressed executables are rarer, primarily because disks have become cheaper and faster, but also if the bits on disk are the same as the bits in memory the operating system can use those instead of swap space when swapping out.
@supercat even DOS .EXE files (strictly speaking, MZ files) provide some support for BSS — not full-blown, but a .EXE can request more memory at startup than the size of its file, and many (most?) compilers’ startup code supports clearing a BSS, going back quite a long time. So a compiler would place BSS-stored variables after the end of the executable, and specify in the executable header that the loader should allocate enough memory for them, without storing zeroes in the file. Ironically SpaceMaker converts .EXE files to .COM format, precluding this...
@StephenKitt: If Spacemaker converts small EXE files to COM files, it sounds like its goal would be to replace fixed-size relocation structures with variable-sized ones, and incorporate a loader which can parse those, so there would be no irony whatsoever in its conversion of EXE to COM files. On the flip side, some COM files produced by Turbo Pascal, especially those that include in-line assembly, do have runs of zeroes internally because they place variables in the code segment as a means of having more than 64K of data available to a program.
@StephenKitt: Not sure if SpaceMaker tries to do anything to compress runs of zeroes in those cases, but if the actual code for a program would be 20K plus 12K of run-time library, the program will be able to use and quickly access up to 32K of "typed constants" in addition to 64K of ordinary variables.
@supercat fair enough, I was ignoring the gains from relocation handling (and in many cases, there wouldn’t even have to be any startup relocation with a .COM file, as produced e.g. by EXE2BIN from a compliant MZ file). I just found it amusing that one of SpaceMaker’s two purported advantages could be handled cleanly in MZ files but not so much in .COM files. (The SpaceMaker ad explicitly says “the relocation entries are eliminated”.)
17:31
@StephenKitt: It would be simple to prefix for a COM file to start with code that would copy itself onto the stack, load SI with the last address in the COM file as written, DI with the proper ending address, and AX with zero, fetch a word from [SI] into CX, and based on the MSB, either perform a REP MOVSB or mask of the high bit and perform REP STOSB, iterating until DI is less than 0x100, thus reaping both benefits.
@supercat not quite: the major benefit that I see is that the loader in DOS knows ahead of time how much memory is required to load the executable, including uninitialised variables, in the MZ case. The .COM workaround would have to deal with that itself, and it wouldn’t be able to fail the load call from whatever started it — it would have to report errors itself too.
@StephenKitt: By what mechanism were DOS 1.0 COM files supposed to know whether addresses out to DS:0xFFFF should be considered usable, so as to be able to abort if insufficient space was available?
@supercat for .COM files, SP on startup points to the end of allocated memory minus one byte, so in your scenario that would be a simple comparison since the proper ending address is known.
@StephenKitt: That seems pretty simple. A COM file unpacker would probably need to include code to show an out-of-memory message if the stack was too low, but otherwise the code could be very compact, and a utility could easily check if the savings from eliminating zero bytes would be greater than the size of the header and simply leave the file alone if so.

last day (15 days later) »