@CarlLange If you're still interested, I found out most details about the regex-problem in the meantime.
Basically, using a trie is a good thing and is done by other regex optimizers too. However, some go a step further. As you know, a trie is nothing more than a prefix tree which can also be thought of as a deterministic automaton (DFA) which reads input and goes from state to state (nodes in this case). What other programs do in addition is that the minimize the DFA once it is constructed and build a regex from that.
This merges internal pathways and gives further optimization.
I have asked a question here and the answer led me to all information I needed
The tool: Strings To Regex - Ternary Tree
namedCharacters (formatted https://pastebin.com/HPJRGxuf)
Regex1: \\\[(?:A(?:Acute|Bar|Cup|DoubleDot|E|Grave|Hat|Ring|Tilde|kuz|l(?:eph|i(?:as(?:Delimite|Indicato)r|gnmentMarker)|pha|tKey)|n(?:dy?|g(?:le|strom))|quariusSign|riesSign|scendingEllip...
So one difference between the full-blown optimization and my simple trie can be seen in this inner node which matches something with Arduino. My regex results in
udio(?:InputDevices|OutputDevices)
while the full-blown solution reduces this further by merging suffixes
udio(?:In|Out)putDevices)
The timing of the person who answered gave almost exactly the timing differences that I also found with Kotlin, but I'm not sure if his program used JS or some other language.
I can't believe I haven't heard about that before. This opens up a whole new range of use cases for WL.
(I should've said "non-production" but that distinction is not that important to me. I learned about it here, and using WL on AWS EC2 is exactly the kind of thing I might want to use it for.)
Stephen is working on a blog on this topic (I think it will be published tomorrow)
We're still working on getting final builds for the "Wolfram Engine" product
The tldr; for advanced existing users: This is a kernel-only product, so the notebook interface is not included. The frontend is included so all rendering services will work (rasterizing, of images, typesetting etc. etc. )
The licensing is for 'non-production use' (Stephen will write up the details and there are details in the licensing.txt file in the top directory of the layout
Perhaps one way to describe the licensing is an open ended (not time limited) trial version of the full Wolfram Language, sans notebook interface.
The positioning of the product emphasizes deployment into software engineering systems and interactivity with other systems, databases, and file formats (so, for example using it with the Wolfram Client Library for Python (see recent blog post on that)).
The product layout is very much what is being used to power Wolfram|Alpha and the Wolfram Cloud at the moment
@CarlLange I'm not sure it's worth the effort. Let me ask a dumb question: Could you implement a simple speed test in JS for my question number 2 of the SO post?
This is, you take the pure regex and simply match through all the positive and negative examples and time the performance.
Added support for a bunch of the built-in stylesheets in my IDE (it was low-hanging fruit):
If anyone has palettes they've written for working with specific types of notebooks (where the type is based on the stylesheet it builds off) I can try to include those as toolbars for these things, too.