For anyone interested in databases and distributed systems, I found Design Data Intensive Applications has good coverage. Not sure if anyone has heard of it or read it.
haha! A coworker used to have a sticker with "There is no cloud — it's just someone else's computer." (from youtube.com/watch?v=oRZoeDRACrY -- "End-to-end encryption: Behind the scenes" by Martin Kleppmann, Diana Vasile)
@Kusalananda since the OP is specifically asking about parallel, it might make more sense to use parallel as the main solution and offer xargs as the alternative here.
Especially since parallel doesn't need any loop or printf and can just do parallel -P 16 sha256sum ::: *
Ah no, the OP wants each one to be in its own file.
Still possible, but slightly more complex. Never mind.
@Kusalananda piping probably not, I'm not sure. But yes, the ">" is odd. But I find the extra sh` for xargs very odd too. Unless you're very familiar with this sort of thing, you'll have no idea what it's doing or why it's needed. But, de gustibus...
-L max-lines Use at most max-lines nonblank input lines per command line. Trailing blanks cause an input line to be logically continued on the next input line. Implies -x.
-n max-args, --max-args=max-args Use at most max-args arguments per command line. Fewer than max-args arguments will be used if the size (see the -s option) is exceeded, unless the -x option is given, in which case xargs will exit.
All I know is that each time I've posted some convoluted bash thing to answer a question, Stéphane comes along with his "In zsh, just say the magic word and it works" answer.
What really brought me over was some question about getting the N most recently modified files from a directory hierarchy. That's a short glob modifier in zsh.
@Jesse_b I'm sure someone has figured out how to attach enough rockets and/or jet engines to accomplish that.
(And maybe they even lived to tell the tale, at least after a several month hospital stay)
@Kusalananda I wonder if it has better ways (other than the obvious answer, switch to Perl) of doing something like this...
groups=('-mtime -45' '! -mtime -45')
for group in "${groups[@]}"; do
find -maxdepth 1 -type d -not -name '.' \( $group \) -print0 \
| grep -xzvFf <(find -maxdepth 2 -type f \( -name 'WAITING' -or -name 'WAITING.*' \) | cut -d/ -f1-2) \
| xargs -0 du $du_args | sed -e 's!\./!!' \
| sort -nr \
| egrep -v '^[01]\s'
echo
done
(And yes, that gives a list of directories, with their sizes in front, excluding directories that have a WAITING or WAITING.* file in them, and excluding directories which are tiny)
Can anyone point me to a brief description of how pattern matching algorithms work? Like regexes, but more generally. I'm currently trying to deal with Lua patterns.
I can't remember my NFA's that well, but it seems to generate some unnecessary-looking states for some expressions. Ones where ε is the only way out, like with the re a(xx) (vs. axx) Is it just me?
If you put +reverse into the regular expression box you get a "Reverse" button that generates the FA that match the reverse string, so you can double-click that to get the minimal automata
@derobert User level. As background, I should say, for those who aren't aware, that my normal reaction to having to deal with regular expressions and their pattern matching cousins is to scream and run. But in the case of Lua, which doesn't have much by way of libraries, one had to handle things with patterns that other languages would offer libraries for.
In this particular situation, I'm trying to extract the basename of a file, given its full path. Python, for example, has a simple library for such things.
But to return to my question, Lua patterns, at least, don't appear exactly deterministic.
Based on the sketchy documetation I have at any rate. For example, how are conflcting requirements resolved?
Yes, in fact "([^/]+)$" is what I have to get rid of the forward slashes. But I also wanted to get rid of everything after the final ., including the dot, assuming there is one.
@sourcejedi I'm trying to be friendly to them. They want to reopen the question because they spent time on it. That is IMHO not a valid reason to reopen a dupe. Also, the text of the question is opinionated throughout ("not in a positive mood" as you say), which is another reason I'm not going to reopen it.
Lua patterns are just not-very-good regular expressions, and regular expressions are deterministic in a "does this match" sense, but they can match multiple different subsequences and the particular library might choose a different one to give you first/at all
I don't have lua in front of me now to try out ([^/]+)([.][^./]+)?$
PEG are a fundamentally different complete parsing system and it feels like overkill for this problem, but I'm not sure you can do it in one step with the regular expressions
PEGs are essentially a way of representing a whole context-free grammar (your friend Chomsky!) and then you try to apply the whole grammar against an input to get back a yes/no and a syntax tree if it worked. Internally they're very complicated but from the outside, if you're already thinking in terms of a grammar, they just work magic
They are just different
PCRE is regular expressions plus a bunch of non-regular extensions and syntactic shorthands that make them much more pleasant to write
It turns out Debian has the Lua Rex libraries. E.g. lua-rex-pcre. How out of date, I don't know. Some of its Lua libraries are years out of date. I don't know why.
LPEG does look like it has more muscle than the Lua patterns. I'll put a pin in it and check it out further when the next pattern matching event comes up.
The super-short version of the paper is that it's very complicated and all the answers have major problems, but here are some names to describe those anyway
@Kusalananda I don't expect there's anything you could have said they would have taken as friendly :-). My last comment was hard-disengage and just see if I could make them feel a bit better. 'cos having your question closed sounds bad at first, but (from my POV) having your question closed as duplicate with valid answers is actually positive.
@Kusalananda I assume "all links get redirected" was a complaint about the post notice. I didn't think the site does a weird instant redirect to the duplicated-against question, in any situation.
I'm surprised at the idea Google does anything useful with "Duplicate questions that use different words to describe the same problem", if we serve Google a redirect to the dupe target. I would have guessed Google would ignore the original, either assuming it's some sort "we;re redirecting you now" message, or as some weird SEO trick.
