Cuonglm and slashes

cuonglm

May 25, 2016 10:33

@StéphaneChazelas @Gilles Would you mind looking at my answer unix.stackexchange.com/a/285343/38906

I found the busybox find behavior is weid, it treats basename of file contain / is legal

Is there any documentation for this behavior?

terdon

May 25, 2016 10:54

→ 15 messages moved to discussion room

cuonglm

@terdon: The room is frozen, can you open it?

terdon

Done

Stéphane Chazelas

May 25, 2016 11:13

@cuonglm find's -name matches against the basename. For / with busybox or heirloom, it seems the basename is the empty string, while it should probably be / (GNU find will give that warning if you use -name / but will still match). For a/ the basename is '' for heirloom, a/ for busybox. For a//, it's still '' for heirloom but / for busybox. Just bugs. Those implementations have simply not considered those corner cases.

Gilles

@cuonglm matching / in a file name is a bug in busybox find

busybox is meant for small systems, so it's unsurprising that it has a few bugs in corner cases

GNU find treats / as the basename of the root directory (so find / -name \? matches the root directory). That's a bug, less excusable.

POSIX is buggy too (as of susv4): it defines “basename” as “The final, or only, filename in a pathname”, but the pathname / does not have any filename at all (the pathname / is decomposed as a beginning slash followed by zero filenames).

So according to these definitions / does not have a basename. But a lot of text in the spec assumes that all pathnames have a basename.

A basename is a filename, and a filename consists of 1 or more bytes that aren't /. That doesn't leave room for anything sensible for the root directory when it's expressed as / (or /// etc.).

I guess . would make sense. The basename of /. is ., and /. is the same file as /.

But it would mean that find / -name . should match the root directory, which I suspect no find implementation does, and probably other similar consequences.

cuonglm

May 25, 2016 11:41

@StéphaneChazelas @Gilles As POSIX defined

> In the context of a pathname, each filename shall be followed by a <slash>

As I understand

in /a/b/c/d/

the basename is d/

Using basename() pubs.opengroup.org/onlinepubs/9699919799/functions/…

to compute, it return d because trailing slashes were removed

pubs.opengroup.org/onlinepubs/9699919799/basedefs/…

terdon

Yes, but the point is that file names cannot contain /

Stephen Kitt

That spec explicitly states

> If the string pointed to by path consists entirely of the '/' character, basename() shall return a pointer to the string "/". If the string pointed to by path is exactly "//", it is implementation-defined whether '/' or "//" is returned.

terdon

pubs.opengroup.org/onlinepubs/9699919799/basedefs/…

cuonglm

Read http://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html, the 10th example, the doc said
> because trailing <slash> characters are ignored when computing the basename of the file under evaluation

terdon

@cuonglm Yes, but apparently busybox find doesn't. That still doesn't change the fact that slashes are not allowed in file names.

cuonglm

May 25, 2016 11:47

@terdon: Isn't d/ in /a/b/c/d/ the context of pathname?

terdon

Yes, I guess

cuonglm

@terdon: they are allowed in context of pathname

terdon

@cuonglm Yes, but not in file names.

@cuonglm your answer states:

> And a filename can contain trailing slashes

And that's explicitly forbidden by POSIX.

> A sequence of bytes consisting of 1 to {NAME_MAX} bytes used to name a file. The bytes composing the name shall not contain the <NUL> or <slash> characters.

cuonglm

@terdon: Sorry, missing the context path, added

terdon

The filename will never contain a slash (with the exception of the root dir). The pathname can, but not the file name.

@cuonglm You are still incorrectly stating that a file name can contain a slash. It can't.

cuonglm

May 25, 2016 11:50

The filename in context of pathname can contain slash, can't it?

terdon

@cuonglm No. In the context of a path name, the file name will be followed by a slash.

> A sequence of bytes consisting of 1 to {NAME_MAX} bytes used to name a file. The bytes composing the name shall not contain the <NUL> or <slash> characters. In the context of a pathname, each filename shall be followed by a <slash> or a <NUL> character;

cuonglm

Isn't followed mean contain?

terdon

@cuonglm No. followed means "comes after" and contains means "is a part of"

For example, Tuesday follows Monday but Tuesday does not contain Monday.

cuonglm

@terdon: Does string "a/" contain "/"?

terdon

Yes

Gilles

May 25, 2016 11:53

@cuonglm no, the basename is a filename and it is followed by a slash. The slash is not part of the filename.

terdon

@cuonglm But the file name a is followed by a /.

cuonglm

@Gilles: So in /a/b/c/, which is its basename?

Gilles

@cuonglm c

cuonglm

So what is the point of POSIX definition of filename when mentioned in pathname context, filename shall be followed by /

And also, the 10th example in find spec said when computing basename but not against basename

> Except for the root directory, and "//" on implementations where "//" does not refer to the root directory, no pattern given to -name will match a <slash>, because trailing <slash> characters are ignored when computing the basename of the file under evaluation. Given two empty directories named foo and bar, the following command

Gilles

@cuonglm I think this sentence is redundant: it's implied by the definition of pathname

cuonglm

May 25, 2016 11:59

@Gilles: Well, it's said basename of the file under evaluation

terdon

@cuonglm It means that when a filename is used in the context of a pathname, then and only then, the filename will have an extra trailing /. So the file/dir foo becomes foo/ when used as a pathname.

Gilles

@cuonglm that sentence is reminding the reader that a basename cannot contain a slash, except for the root directory

it seems to assume that the basename of / is /

terdon

Also, I think we have already established that busybox find isn't actually conforming to the standard here.

cuonglm

@Gilles: I think no, because the example following it does not mention anything about /

> find foo/// bar/// -name foo -o -name 'bar?*'

Gilles

The definition of the basename utility explicitly defines its behavior on /, //, ///, etc. without refering to the basename concept

@cuonglm the first part of the first sentence only makes sense in English if the root directory's basename contains /

In susv4 the only problematic reference to basename is in find

Stéphane Chazelas

May 25, 2016 12:06

@Gilles No the basename of / is /, so find / -prune -name / should match as -name is meant to match against the basename of the file. GNU find is compliant in that it matches but gives that irrelevant warning.

cuonglm

Hmm, I'm still confuse a lot, I just changed my answer to cite the doc explicitly

Gilles

@StéphaneChazelas how do you get that the basename of / is / (according to POSIX, not to historical behavior of existing implementations)?

cuonglm

@Gilles: Yes, see the 10th example in find spec, it allowed / to be matched by ?

Gilles

@cuonglm no it isn't

cuonglm

@Gilles: Like I said, the find spec said when computing basename , not against basename

Stéphane Chazelas

May 25, 2016 12:08

@Gilles 3. If string consists entirely of <slash> characters, string shall be set to a single <slash> character

pubs.opengroup.org/onlinepubs/9699919799/utilities/…

Gilles

@StéphaneChazelas that's the basename utility. Which doesn't exactly compute the basename of its argument...

cuonglm

@Gilles: Here the example said
> Except for the root directory, and "//" on implementations where "//" does not refer to the root directory, no pattern given to -name will match a <slash>, because trailing <slash> characters are ignored when computing the basename of the file under evaluation. Given two empty directories named foo and bar, the following command:

In case of /, the pattern give to -name can match a slash

Gilles

The definition of the basename utility is equivalent to the definition of the basename concept, except that the definition of the concept doesn't give a meaning to the basename of a pathname containing only slashes.

@cuonglm yes, but ? wouldn't match it

cuonglm

So:
> find / -name \?

can match /

terdon

And all this simply because an OP asked what the \ in \? means. All the poor fellow needed was an explanation of escaping and instead he got two answers delving into the depths of the POSIX specs for file name expansion.

cuonglm

May 25, 2016 12:10

@Gilles: It would match, see the example in my answer
> find / ! -path / -prune -o -name \? -print

Gilles

@cuonglm no, you're right, if the basename is / then ? would match it

The problem is that nowhere does it say that the basename of / is /

> The primary shall evaluate as true if the basename of the current pathname matches pattern using the pattern matching notation described in Pattern Matching Notation. The additional rules in Patterns Used for Filename Expansion do not apply as this is a matching operation, not an expansion.

The word “basename” refers to the definition of the concept, not to the basename utility.

cuonglm

@Gilles: Yes, that's the reason make me confuse

Gilles

“The additional rules in Patterns Used for Filename Expansion do not apply” means that wildcards are allowed to match a slash.

Stéphane Chazelas

@gilles, pubs.opengroup.org/onlinepubs/9699919799/basedefs/… has the definition of basename which is a bit vague "The final, or only, filename in a pathname" (would still not allow the basename of / to be the empty string)

The "except for / and //..." quote @cuonglm gave is clear about the intent anyway

Gilles

@StéphaneChazelas exactly: basename (the concept) is not defined for a pathname that does not contain any filename.

Stéphane Chazelas

May 25, 2016 12:15

IMO, there's little ambiguity that find / -prune -name / should match

Gilles

@StéphaneChazelas yes, because the concept is in fact only used by find. The definition of pax refers to the utility, and in other places they refer to files that can't be directories or have a non-slash character in their name.

cuonglm

@StéphaneChazelas: Would you show your opinion about the definition of basename in the spec

As I understand now, basename is "The final, or only, filename in a pathname"

And filename "In the context of a pathname, each filename shall be followed by a <slash> or a <NUL> character"

So basename of a file can contain trailing slashes

terdon

@cuonglm No! followed != contains

Stéphane Chazelas

The POSIX definition of basename when it comes to / or // or ///+ is unclear to me. Now, If / has a basename, the only possible values are "/" or "". "" doesn't make sense, so it can only be "/". The basename of a/// or a/ is "a", that at least is clear

terdon

This is a parenthesis followed by a slash: ()/ and this is a parenthesis containing a slash: (/) the two are very different.

Gilles

May 25, 2016 12:19

@terdon not a parenthesis containing a slash, a pair of parentheses containing a slash

and it's a different meaning of “containing”: “enclosing”, not “comprising”

cuonglm

@StéphaneChazelas: As I understand, by the definition, basename of / or // or ///+ is /

because it's the last filename (in context of pathname) of pathname.

terdon

@Gilles No, parenthesis can be used to refer to either one of the punctuation marks ( or ) or to both of them. See, for example, merriam-webster.com/dictionary/parenthesis

cuonglm

The process to compute the basename, using basename() function will remove the trailling slashes and also have corner case for / and //

terdon

The common mistake is saying The word (text) is in parenthesis instead of in a parenthesis or in parentheses.

cuonglm

If my interpreting is not true, then I have the same opinion with @StéphaneChazelas, the spec should be clarified the definition for basename of /, // and ///+

Gilles

May 25, 2016 12:25

@cuonglm No: the pathname / does not contain any filename.

@cuonglm Yes. The current definition does not apply at all to a pathname containing only slash characters.

Stéphane Chazelas

@cuonglm, yes I agree. I think we all agree. The intent is that -name / should match on / or /// or ////, but as specified it's unclear. As per the definition, / has no basename, so what you'd need to pass to -name so it matches on / is unspecified (even if further down in the APPLICATION USAGE section (not normative) it makes it pretty clear that -name / should match)

Looking at the latest draft, the basename definition has been clarified: _For pathnames containing at least one filename: the final, or only, filename in the pathname. For |
pathnames consisting only of <slash> characters: either ’/’ or "//" if the pathname consists of |
exactly two <slash> characters, and ’/’ otherwise._

cuonglm

@StéphaneChazelas: can you give us the link to latest draft

Stéphane Chazelas

It's on the opengroup site, you need to register to get access to it. issue 7 TC2 is going to come out soon.

See austingroupbugs.net/view.php?id=653

So find / -prune -name / or find / -prune -name '?' are meant to match and the next technical corrigendum of the POSIX spec will make it clear/

cuonglm

@StéphaneChazelas: +1

And thanks all you guys for useful discussion

/dev/chat

Participants