When you pair the *globstar and the pathname separator you're saying, shell I want an argument list amounting to any or all directories contained within this one
sh -cx 'ls */'
+ ls one/ three/ two/
one/:
three/:
two/:
@Tim - see that + bit?
That's actually the shell's evaluation of my command.
That's what actually happens.
So doing ls */ is the same in this case as if I had typed ls one/ two/ three/ because the shell expands that out before ever calling on ls at all.
You don't get the / at the ends because that's actually a quality of default settings - which are ignored when you specify -d because that option is specifically for treating all files as a directory.
Typically when people talk about null they refer to the ASCII 0 byte. It's a long sordid story, but it goes back to punchcards. If you were processing data on punchcards, you could make filler space by just not punching and holes - that's a null byte
it is \000
There's also the concept of a null string - which is an empty expansion. so printf %s ''
printf gets an argument there, but it is empty - the null string.
So I can do: printf '%s this still gets printed' ''
But a null byte, nowadays, is not a blank punchcard, it's an actual byte. Anyway, it is still forbidden in filenames. Only it and the pathname separator / share that distinction.
A lot of things, but by default ls distinguishes by filetype.
Ok... welll... this goes deeper because it's about dentries.
So files are not the names you cal them.
files are inode numbers.
Hang on...
touch file
ls -i
397399 file
A directory is special type of file - it's a file's file - i guess.
It stores information about files.
When you move a file from directory to directory you directly affect the file - you affect which directory stores information about it.
The filename is stored in the directory as a quality of the file's directory entry - its dentry.
The filename refers to an inode - which is how the kernel refers to a file.
A hardlink is another dentry for the same inode - just different listings for the same file. It is not a quality of the file, but a quality of the directory listing it.
ls does dentries.
That's what it does - it reads the directory and reports on its listings.
It doesn't do too much with the files within unless you tell it to - with -d.
That's when it treats each file as a directory and reads it.
(1) "which are ignored when you specify -d because that option is specifically for treating all files as a directory." Do you mean treat a dir as a file instead?
(2) with -p, dirs are output with / at the ends. without -p, no / at the ends of dirs.
No, vice versa. Like I said, ls does dentries - so when it does a listing it typically only has to stat the containing directory. When it treats each of its arguments as a directory in its own right, it stats each one.
Well the -p is an argument to ls. It says ls please append the pathname separator to dir types.
The */ glob is an instruction to your shell which calls ls.
It says, shell, before calling ls, expand this glob out to represent every directory within the current one, and then hand that list to ls as arguments
Because ls read directories - not the files described by their dentries, it doesn't amount to much of a difference at all.
However, if you did ls -dp * and ls -p */ you might see the difference.
Yes, definitely. They're how any programmer has to translate a filename to an inode number and back again. Which is needed when handling files with relation to the kernel.
I wouldn't know why the */ thing would be covered in one and not the other.
Well, you have to link through it. But still - it's there. It's number three:
Specified patterns shall be matched against existing filenames and pathnames, as appropriate. Each component that contains a pattern character shall require read permission in the directory containing that component. Any component, except the last, that does not contain a pattern character shall require search permission. For example, given the pattern:
/foo/bar/x*/bam
search permission is needed for directories / and foo, search and read permissions are needed for directory bar, and search permission is needed for each x* directory.
If the pattern matches any existing filenames or pathnames, the pattern shall be replaced with those filenames and pathnames, sorted according to the collating sequence in effect in the current locale. If the pattern contains an invalid bracket expression or does not match any existing filenames or pathnames, the pattern string shall be left unchanged.
There's also the pattern matching notation thing there.
Still, the spec can be a little difficult to swallow sometimes.
I mean, it's a bit like a stereo manual.
You can also just do man sh or man bash to learn more. That's probably a good place to start.
Your ${pathname##*/} is a parameter expansion. It means strip from the head of this $pathname variable any and all characters up to and including the last / occuring within its value
so like:
var=aabbccdd
echo ${var##*c} dd
But the weak form only matches as little as possible:
Of course, this can't happen because you strip them all, but if you had a literal / stored within a variable and expanded it can expand again to a pathname - because variable expansion happens first
In that case it's literal at first, and then it isn't.
@mikeserv Certainly. Short version: I was creating a bunch of files, and loading it into a db in parallel. The only problem is that I was creating and loading each file in pieces and loading each pieces of each file into the db directly. So, the runtime was dominated by the loading process.
This is sort of textbook moron. I really have to wonder what I was thinking. Ironically, the slower version is also significantly more complicated. Usually the optimized version is more complicated.
@mikeserv The optimization came from (a) not breaking up each file into pieces. That doesn't make much sense to start with, and (b) waiting to the end to load the files into the db.
@mikeserv Well, dbs can handle parallel loads. That's not the problem. The problem is that doing repeated loads thousands of times is incredibly expensive. For one thing, one has to drop and load constraints, which of course gets more expensive as the db gets bigger.
@mikeserv Well, I'll try it with a pipe. There is a link somewhere. One sec.
In linux pipe the output of the files listing to psql. Make copy use the standard input:
cat /path_to/ys*.csv | psql -c 'COPY product(title, department) from stdin CSV HEADER'
Look for the equivalent in other OSs
Apache Hadoop is an open-source software framework for storage and large-scale processing of data-sets on clusters of commodity hardware. Hadoop is an Apache top-level project being built and used by a global community of contributors and users. It is licensed under the Apache License 2.0.
The Apache Hadoop framework is composed of the following modules:
Hadoop Common – contains libraries and utilities needed by other Hadoop modules.
Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster...
@mikeserv This is one of the reasons that working with other people is a good idea. The most useful function of a collaborator is to say - what the !^&&** are you doing, you incredible idiot?
One thing that strikes me listening to dbs chatter is how complicated the whole thing is, considering we are basically just talking about a bunch of tables. Wrt relational dbs, of course.
@mikeserv Formal universities are surprisingly dysfunctional places. The major advantage is the contacts you make. Also, they sometimes have good libraries, and journal access.
A US teenager once remarked to me that they don't make Hollywood films about good students. Which I suppose is true. It might not be particularly exciting watching people study.
I think it is natural to like learning, but schools are good at beating it out of you. Of course, that was India.
The whole testing / artificial competition thing is particularly unhealthy. Good for teaching deference to authority, though. Which is what the people who run things like.
@mikeserv Well, it's nice you had one teacher you had positive feelings about. I had an art teacher I got on quite well with, though she was quite eccentric. Though maybe part of the reason I liked her was because she was eccentric.
@Gnouc - You're already pretty well setup to gather it on a branch test anyway, since you've got the label. So... hang on, maybe I can come up with something if I stare at it....
@Gnouc - maybe that's not even the best way, maybe mine's too complicated... This could be a very good candidate for a paired address - a range address.
sed '/[[:alpha:]]/,/[^[:alpha:]]/{/[[:alpha:]]/{N;s/\n/<tab>/;};}'
No. That does the opposite of what it should... damn...
ok, the branch thing...
@Gnouc:
sed ':a
/[^[:alpha:]]/{N;/\n[^[:alpha:]]/!ba;s/\(.*\)\n/\1\t/}
' <<\DATA
NAME_A
NAME_B
NAME_C
12,1
NAME_D
21,2
DATA
NAME_A
NAME_B
NAME_C 12,1
NAME_D 21,2
No.I
That's wrong - I don't know why it worked - I guess just because of the second address.
Anyway - the first one needs to not complement
/^[[:alpha:]]/{$!N
Anyway, for a line that starts with a alphabetic character, it pulls in the next if it's the last line, then if it can address a newline followed by a non alpha it selects everything in pattern space up to the last newline there and does a substitute on it - changing the last new line to a tab
Anyone ever notice that even though the votes on a migrated question go with it, if one of those is yours, you can still upvote it again on the site it was migrated to?
@VolkerSiegel You've got 3 different people telling you to use SIGTERM. If you're such an expert on signals, why did you even ask the question? — Patrick30 secs ago
I mean they believe it to be best for them and are quite vocal that it must therefore be best for everyone else
if you want an example of that behavior with other software just throw emacs and vim into a sentence together.
I don't often use either screen or tmux, but when I need something like that I just need a detached remote session, and screen fits the bill better than tmux since I don't need tmux's ability to multiplex