« first day (787 days earlier)      last day (52 days later) » 

8:18 AM
what did they say?
unhealthily curious
 
They said, "Brother, what happened. Isn't it working?". And that seems to be in very surprised tone which I think is a troll.
 
I am in UK on my other laptop
 
Just for reference: I commented, "If that worked, why would the OP post a question here? Have some common sense"
 
wow it's so nice to have working a, d and left shift keys
 
@Zanna Great. Experiencing the weather of UK after so many days?
 
8:24 AM
"common sense" in English haha
@Kulfy it's like 9 degrees C. I'm freezing D: but Autumn is beautiful. Nice to see that
 
I forgot what we call that in Hindi :P
@Zanna And here in north India it's about 25°C
 
@Kulfy haha all languages get littered with useful phrases from other languages isn't it
@Kulfy it got about that low in the night in Chennai this week. Monsoon arrived. But still about 30 in the day time
 
Indeed. So I was neither using Hindi nor English. I used "Hinglish". 😅
 
I enjoy these mixtures of languages a lot
that's why I commented on it
 
@Zanna South doesn't get as cold as the North.
 
8:29 AM
I watched some Hindi movies on the flight. Lots of English bits haha
 
And the accent 😅
 
@Kulfy a few Chennaites have said to me "here we have three seasons: hot, hotter and hottest"
 
LOL
 
@Kulfy I enjoy all accents hahaha I love the variation in the ways people speak. Music
 
There was a dialogue in some Telugu movie, "Don't trouble the trouble. If you trouble the trouble, trouble will trouble you. I'm not the trouble, I'm the truth" :P
 
8:35 AM
hahaha I wonder what "trouble" is in Telugu
have to ask my friend in Hyderabad
 
On a side note, can you please move this conversation to the island?
 
ooh it's in English lol
28 messages moved from Raiders of the Lost Downboat
 
Thanks
 
8:59 AM
no problem!
the context of the start of that conversation might not be very clear
for future readers, they relate to this message
 
Just out of curiosity, are people really interested in reading "off-topic" conversation?
 
9:33 AM
I don't think it's unimaginable that someone will have some reason to read it in some future time
 
 
8 hours later…
5:49 PM
Hey @EliahKagan!!! I need your help in understanding something.
 
@Kulfy I'm intermittently AFK but I'll do what I can to help.
 
OK I'll post here. If you get time, do answer even if I'm offline since it's already 23:20 here and I'm about to sleep anytime soon.
 
Sounds good.
I'll be at my keyboard for at least the next several minutes, so definitely please do go ahead.
 
Today I answered a question. But after pLumo's comment, that my answer would fail if there are whitespaces along the path or filename.
The question is, "How to concatenate files found by the find command in bash"
And the command I thought of was
cat `find . -name "configuration_dev.txt"` > testing.txt
I though may be enclosing the output of find in quotes or escaping the whitespaces would remove the limitation.
So first I tried: find . -name "Tes*" | sed "s/^/'/g" | sed "s/$/'/g"
which gave me output
'./Test File 2' './Test File 1'
And I expected that it would parse the same output to the cat.
But when I run
cat `find . -name "Tes*" | sed "s/^/'/g" | sed "s/$/'/g"` > temp
I get:
cat: "'./Test": No such file or directory
cat: File: No such file or directory
cat: "2'": No such file or directory
cat: "'./Test": No such file or directory
cat: File: No such file or directory
cat: "1'": No such file or directory
So why find is returning './Test, File, 2',... as different file names?
Why enclosing in quotes didn't work?
Neither escaping whitespaces worked.
 
6:09 PM
@Kulfy find isn't doing that--bash is. Specifically, it performs word splitting (which POSIX calls field splitting).
Depending on what characters find output, the shell may also perform globbing (POSIX), which is also called "filename expansion" or "pathname expansion" and which you also don't want here.
The reason this happens is that word splitting and globbing are performed on the results of some (though not all) kinds of shell expansions, when the expansions are not quoted.
The kinds of expansions that are immediately subject to word splitting and globbing (when not quoted) are parameter expansion, command substitution, and arithmetic expansion.
 
So would changing value of IFS work?
 
@EliahKagan This is to say that the expansion that are subject to word splitting and globbing are the ones that can be denoted with a syntax that includes a leading $ sign.
You may or may not find that mnemonic useful: note that command substitution is subjected to word splitting and globbing both when it is denoted by backticks, as you wrote it, or by $( ), which is the preferable way to do it (because that form nests less confusingly--people say the backticks form is deprecated, which is not actually true, but is still best avoided in favor of the $( ) syntax).
@Kulfy Setting IFS, for example to hold just a newline, would make your technique work in more cases, but it would still fail in some cases. I do not recommend that.
 
@EliahKagan Yeah I have heard people saying ` is deprecated.
@EliahKagan So, do you have any idea to fix that command? I'll cite your message as source.
 
@Kulfy Though I strongly believe that it is inaccurate to say it is deprecated -- usually that designates an official status, and the backtick form of command substitution has not been officially deprecated by the Austin Group or otherwise -- I do nonetheless agree urge against its use (unless there were some unusual and very compelling reason to use it).
@Kulfy Yes. :)
(I'll get to this. I feel I've not yet answered all of your question, especially the part about why quote-pasting didn't work.)
@EliahKagan Changing the value of IFS doesn't affect globbing, so if you were to proceed that way, you would also want to disable globbing, which could be done with: set -f
However, you would still have the vexing problem that the output of the -print action of find (whether specified explicitly or triggered as the default action) is actually ambiguous and thus poorly suited for parsing, because it is possible for a filename to contain an newline, and this does actually happen in real life (usually by accident, but someone might also be trying to attack your script).
 
6:24 PM
@EliahKagan I didn't find any official documentation which specifically mentions that it is going to be deprecated.
@EliahKagan No problem. I might wake up to read an excellent explanation. :)
 
@Kulfy Yeah, in addition to not being officially deprecated, I suspect it will not be. Anyway, that something may in the future be deprecated is rarely a reason not to use it. The reason to prefer the $( ) syntax is mainly that it nests more cleanly. Nesting backticks does work properly in modern Bourne-style shells including bash (I've heard rumors it used to not work right in some shells, but I am not sure about that).
But it involves a lot of backslashes and it tends to be confusing to people who right and read scripts. The $( ) syntax is also readily visible at a glance.
 
Agreed.
 
Anyway, what I've said so far is quite abstract. To concretely test what (and how many) words an expansion produces, I recommend adjoining it to the end of something like:
printf '[%s]\n'
In particular:
ek@Kip:~$ mkdir tmp
ek@Kip:~$ cd tmp
ek@Kip:~/tmp$ ls
ek@Kip:~/tmp$ touch 'one-word'
ek@Kip:~/tmp$ touch 'two words'
ek@Kip:~/tmp$ find .
.
./two words
./one-word
ek@Kip:~/tmp$ printf '[%s]\n' `find .`
[.]
[./two]
[words]
[./one-word]
And, adding the pathological case of a filename containing a newline character:
ek@Kip:~/tmp$ touch $'two\nlines'
ek@Kip:~/tmp$ find .
.
./two?lines
./two words
./one-word
ek@Kip:~/tmp$ find . | cat
.
./two
lines
./two words
./one-word
ek@Kip:~/tmp$ printf '[%s]\n' `find .`
[.]
[./two]
[lines]
[./two]
[words]
[./one-word]
This shows something else important about parsing the output of find -- the output you see on a terminal is not necessarily the same as the output you would be parsing, because find checks if its standard output is a terminal and, when it is, replaces weird characters with ?.
When it is not a terminal -- a situation I produce artificially via the quite useful technique of (what I would call) "postfix UUOC" -- it does not perform that replacement.
In contrast, if command substitution on find (and the ensuing word splitting) had produced one argument per command, then you would see something like what you get with a glob:
ek@Kip:~/tmp$ printf '[%s]\n' ./*
[./one-word]
[./two
lines]
[./two words]
@Kulfy Actually, I'll say what my thoughts are about getting around the problem before I talk about why quote-pasting does not work. (That does not obligate you to stay awake, though -- my messages should still be here when you get back. :)
 
6:43 PM
No problem. I'm not that sleepy right now.
I'm trying to read the docs and your messages again and again to understand in a better way. I'm little stubborn :P
 
GNU find has a -print0 action. The null character cannot appear in a filename, so this makes it safe to parse the output of find. That approach is used in pa4080's answer.
You can also make find run cat one or more times and pass the filenames to it. One way to do that is the approach used in aleksander_si's answer, which I'd say is the best way...though the placement of > testing.txt is extremely confusing because it creates the false impression that it is part of the command find is running.
(Redirections can appear anywhere in a simple command, even between arguments, and that's what's happening here. find never sees any part of > testing.txt. That just redirects the output of the whole find command--including its subprocesses, since they inherit its stdout--to that file.)
 
@EliahKagan Yeah I understand that. find is just parsing its output to cat. Its cat which is handling redirection.
 
@Kulfy Neither find nor cat see > testing.txt or are in any way involved in the redirection. The shell applies that redirection before it runs find, and find runs cat. The cat command(s) run by find just write to standard output, which it/they have inherited from their parent find, which find inherited from a shell process, which happened after that shell process opened the file on standard output (i.e., performed the redirection).
 
7:00 PM
Ah!! I forgot that.
 
Yeah, the way the command is written in that answer is super confusing. I presume we'll eventually delete comments but since a couple of them are relevant to this conversation I'll onebox them here:
This probably fails with a large number of files because cat > testing.txt {} will be called more than once, overriding the previous contents of the file each time. — danzel 14 mins ago
@danzel That's not a problem, it just looks like it would be, due to the confusing placement of > cat testing.txt. As pLumo says, it's clearer to put it at the beginning or end of the whole command. The key is, > cat testing.txt is not actually part of any command run by find. find never sees > cat testing.txt. The shell applies that redirection to the command find . -name "configuration_dev.txt" -exec cat {} +. No matter how many times find runs cat--as you say, with enough files this happens multiple times--testing.txt is opened only once, when the shell opens it. — Eliah Kagan 6 mins ago
(The code formatting is easier to see with the CSS on the main site than here, though, so you may prefer to read them there.)
 
I see. The only issue with the questioner's answer is representation of command. Right?
 
@EliahKagan That is another flaw in the technique you presented in your answer, though: because it builds a single cat command and cat is an external utility rather than a shell builtin, it will fail -- albeit safely and with a comprehensible error message from the shell -- if the command being built is too long. In contrast, find with exec ... + and (with its default behavior) xargs automatically run a command multiple times as necessary.
@Kulfy Yes.
Regarding the problems with word splitting and globbing, I should clarify that sometimes we use the term "filename" to mean "last path component," and in that sense, the OP's specific use case doesn't cause problems with word splitting or globbing from the filename configuration_dev.txt. But that probably doesn't significantly mitigate the problem, as find outputs paths with other components (i.e., directories leading up to the filename), which may trigger word splitting or globbing.
 
So my answer isn't generalized. It works only for that specific case. (Since in question OP didn't have spaces in filename/path)
 
At least as I understand the OP's question, it doesn't necessarily work for their specific case, because find could output something like
./tmp/config files/configuration_dev.txt
which would be split at the space in config files.
I don't think I know of a good way to really fix your answer that wouldn't effectively be replacing the method it currently recommends with a different technique. For example, you could change it to recommend using a recursive glob, which would still fail with a huge number of files, but that's really probably not a big issue here... but that would be a quite different method from what you've recommended. It wouldn't use find at all.
It would be similar in spirit, though, in that it would be a way of using a shell expansion to build an argument list for a cat command which the shell directly runs.
But if you were to modify your answer to suppress globbing and to make it split only on newlines, and disclose prominently that it breaks in the corner case of filenames (including directory names leading up to the file) containing newlines, then I think that would fix the answer in the sense that the answer would no longer be misleading and the technique it recommends would be one that readers would be interested in and that would be acceptable in a number of real specific cases.
Or you could do that and also add another method to it.
Or you could delete it, though in general I prefer improvement to deletion when there is a way.
Or you could leave it as it is, but I don't recommend that.
 
7:58 PM
@Kulfy So anyway, the reason pasting quotes onto the filenames doesn't work is that quoting characters that result from command substitution (or parameter expansion) are not treated specially by the shell, regardless of whether or not the command substitution (or parameter expansion) is itself quoted.
It is good that quoting characters are not treated specially in this context, because if they were, then that would be one more weird thing you usually wouldn't want and would have to take care to suppress or work around much of the time.
@Kulfy For demonstration purposes I'll use this command, which is shorter and (in my opinion) more readable, but which has the same effect as your sed pipeline:
find . -name 'Tes*' | sed "s/.*/'&'/"
Suppose I'm in a directory with those two test files, i.e.:
ek@Kip:~/tmp$ ls
'Test File 1'  'Test File 2'
ek@Kip:~/tmp$ printf '[%s]\n' *
[Test File 1]
[Test File 2]
You can paste quotes on the edges:
ek@Kip:~/tmp$ find . -name 'Tes*' | sed "s/.*/'&'/"
'./Test File 1'
'./Test File 2'
But it is not typically useful to do so.
ek@Kip:~/tmp$ printf '[%s]\n' $(find . -name 'Tes*' | sed "s/.*/'&'/")
['./Test]
[File]
[1']
['./Test]
[File]
[2']
The quoting characters do not perform quoting in the shell when they arise as a result of any of the $ expansions -- nor when they arise as a result of command substitution written with the backticks syntax.
ek@Kip:~/tmp$ printf '[%s]\n' `find . -name 'Tes*' | sed "s/.*/'&'/"`
['./Test]
[File]
[1']
['./Test]
[File]
[2']
If it did work, that would be bad, because the same characters that could perform quoting often occur in contexts where they either don't signify quoting or don't mean to signify it at the level of processing.
ek@Kip:~/tmp$ touch "Tessa's bar and grill"
ek@Kip:~/tmp$ printf '[%s]\n' $(find . -name 'Tes*' | sed "s/.*/'&'/")
['./Test]
[File]
[1']
['./Tessa's]
[bar]
[and]
[grill']
['./Test]
[File]
[2']
Quotes are syntactically meaningful to the shell in code the shell actually runs. They are not meaningful in the output commands the shell runs, even when the shell processes that output, unless the shell is actually running that output.
You should almost never actually do this, but you can make the shell run arbitrary text that is generated at runtime with the eval builtin. If you do, it is only safe to paste quotes in situations where you know the exact text you are pasting them on, i.e., in situations where you neither need nor significantly benefit from eval in the first place (and even then, the resulting code is immensely confusing).
In situations where the power of eval actually brings something to the table, you would need to use a more robust method if you wanted to ensure something was quoted. Bash provides two mechanisms that help with this (though one of them was only recently introduced). Note that I am absolutely not advocating that you should use this, or any technique involving eval, in that particular answer. I'm just mentioning it because it is topically related and interesting.
One is the %q format specifier for printf:
ek@Kip:~/tmp$ printf '%s\n' *
Tessa's bar and grill
Test File 1
Test File 2
ek@Kip:~/tmp$ printf '%q\n' *
Tessa\'s\ bar\ and\ grill
Test\ File\ 1
Test\ File\ 2
 
8:35 PM
In cases where that applies, you can also hand-roll something that performs robust quoting with sed, based on the insight that you can wrap the text in single quotes if you handle just the single quotes that it contains.
ek@Kip:~/tmp$ printf '%s\0' * | xargs -0 printf '%s\n'
Tessa's bar and grill
Test File 1
Test File 2
ek@Kip:~/tmp$ b=\\
ek@Kip:~/tmp$ q=\'
ek@Kip:~/tmp$ printf '[%s]\n' "$b" "$q"
[\]
[']
ek@Kip:~/tmp$ printf '%s\0' * | sed -z "s/$q/$q$b$b$q$q/g;s/.*/$q&$q/" | xargs -0 printf '%s\n'
'Tessa'\''s bar and grill'
'Test File 1'
'Test File 2'
ek@Kip:~/tmp$ file 'Tessa'\''s bar and grill'
Tessa's bar and grill: empty
The other mechanism bash provides to help with this is the new ${parameter@operator} form of parameter expansion. In which the Q operator performs quoting.
ek@Kip:~/tmp$ f="Tessa's bar and grill"
ek@Kip:~/tmp$ printf '%s\n' "$f"
Tessa's bar and grill
ek@Kip:~/tmp$ printf '%s\n' "${f@Q}"
'Tessa'\''s bar and grill'
In summary, pasting quotes at the edges of text (a) does not typically cause the quotes to be syntactically significant, and (b) would not typically be guaranteed robust enough to perform correct quoting in situations where the quotes were syntactically significant.
 

« first day (787 days earlier)      last day (52 days later) »