« first day (623 days earlier)      last day (641 days later) » 

ASCII-only
ASCII-only
05:34
@Dennis Would it be possible to write an automatic compressor for Jelly strings
Dennis
Dennis
Sure. There's already a semi-automatic one, so you just need some logic to check which chunks can be replaced by dictionary words.
ASCII-only
ASCII-only
Hmm, 1. would it be possible to make it golfier :P 2. can I steal copy the compression from Jelly
Dennis
Dennis
There's definitely room for improvement. For starters, the words are sorted by length, then alphabetically. They should be sorted by frequency instead.
ASCII-only
ASCII-only
Hmm, okay so if I replace with a list sorted by frequency then the comression will be much better? :P
Pretty sure you can find one of those in a lot of places
@Dennis Anything else?
Dennis
Dennis
It's probably wise to split the words in common and uncommon, as they're currently split in short and long.
Also, there's a problem with spaces not being able to be encoded in certain positions. I don't recall where right now.
ASCII-only
ASCII-only
05:49
Hmm, okay. Could you give a basic overview of how the compression works? :P Right now AFAICT it uses one or more bits for a flag before each word, which doesn't seem right
Dennis
Dennis
Not actually bits. Before each new token, you have to choose between a character, a short word, and a long word.
ASCII-only
ASCII-only
Hmm so is it divmodded by 3?
Dennis
Dennis
← 12 messages moved from talk.tryitonline.net
ASCII-only
ASCII-only
Oops :/ sorry
Dennis
Dennis
No worries.
ASCII-only
ASCII-only
05:54
It seems like it could be possible to shorten the choosing possibly if some assumptions are made about token frequency?
Dennis
Dennis
@ASCII-only Yes. After a character, you put an integer from 0 to 94 (printable ASCII and newline). Before a word, there are two bits (capitalized or not), (leading space or not). Spaces have different defaults depending on where the word appears.
ASCII-only
ASCII-only
Hmm, how long are the short/long word dictionaries respectively?
Dennis
Dennis
20453 and 227845 words
ASCII-only
ASCII-only
thinking in progress
Dennis
Dennis
@ASCII-only Oh, I'm sure there's lots of room for improvement. I didn't really waste much thought on this. Afaik Jelly was the first golfing language with a custom string compressor.
ASCII-only
ASCII-only
06:01
Oh lol Charcoal has way too many different compression methods
But no dictionary compression yet
Dennis
Dennis
Such as? I'm curious.
ASCII-only
ASCII-only
Brotli, LZMA, charset + indices, permutations of symbols/lowercase/uppercase/newline or space/digits, base conversion
Not that any of them are any good lol
Oh also RLE
Dennis
Dennis
Well, most of these aren't exactly custom. I got the idea from several languages using shoco, and I thought I could do better for Jelly.
ASCII-only
ASCII-only
True :P but I'm not sure whether there are many custom compressors except for dictionary-based ones
ASCII-only
ASCII-only
06:16
Okay, not sure if my math is right but seems like a small dictionary size of size ~400 would be best, but not sure if it's worth it to have a separate compressor for large texts? (And I have no idea what are the optimal sizes for short phrases/sentences, and I'm not sure where I could find some examples)

« first day (623 days earlier)      last day (641 days later) »