oh yes?

I would like to know what the error means!

When the OP ran rename -n 's/\.DAT/\.dat/' ** with globstar, they got bash: /usr/bin/rename: Argument list too long. But I'd guess this was because of the inclusion of numerous paths not ending in .DAT, and that if the globstar pattern **/*.DAT had been used instead, then it would have worked. It's possible for there to be so many .DAT files that the same error would occur, of course, but I think that method is worth presenting. (It does match even files in the current directory.)

I don't recommend removing the command you did show, but I suggest also showing it that way and explaining the difference.

In a situation where both the commands you showed work but the find command is faster, that's also (probably) the reason: the find command avoids passing all paths to rename, and just passes the ones it should rename. However, you can make that find command even faster by ending it with + instead of \;. This is another area where I suggest keeping what you have but also adding the faster alternative and explaining it. Readers may not know that + isn't always suitable.

ah ok... I didn't use + because I thought it would make the argument list too long for them

I vaguely thought about the issue with including too many non-matching files, but I didn't even know it was possible to avoid that, much less with such a simple change!

@Zanna When you use + with find's -exec (or -execdir) action, find will split it into multiple invocations of the command, with different arguments, as necessary, in order to make sure that it succeeds even if the argument list would be too long if done in one command. That is to say that + makes find behave like xargs (which does that too), which is why the ability to use + in find makes xargs less necessary (in connection with find).

This is one of the factors users should keep in mind when using find's -exec or -execdir with +: not only is it necessary to use it only with commands that actually accept multiple pathname arguments, but it is necessary to use it only either (a) with commands where splitting the list and calling it two or more times will still do the right thing or (b) more rarely, in cases where that isn't known to be so but where one is absolutely sure that the argument list will never be too long.

@Zanna Well making the glob match fewer paths will not always be sufficient, because one might actually have a huge number of files with .DAT suffices under wherever one is searching, and then dotglob-enabled pathname expansion will still generate an argument list that is too long to pass to an external command successfully. But I'd say it's worth trying -- in many cases, it's the only change needed. (Of course this only applies when one does not actually need all the paths, as here.)

I suspect OP did have a huge number of filenames with that suffix. But definitely good to mention it

Well, unlike my **/*.DAT suggestion, find with + will work even if there are too many filenames that have to be passed to rename for it to do it in one rename invocation. This may also be considerably faster, because there is startup cost each time find forks and execs a new process.

take the knife out of the drawer, cut a slice, wash the knife...

Probably still markedly faster than a shell loop, but yes. :)

Actually maybe not much faster, because the Perl interpreter has its own startup time even after it has started running.

I think it's reasonably fine to use a shell loop to rename files because doing stuff with files is something shells are designed to do (unlike text-processing), I think. But I think the point of rename is that you won't need to use a sloowww shell loop because you can use fancy text-processing-like tools to do the job. So a method that requires calling rename many times is lame.

(irrelevantly) is there really an owl dancing extension?

reminded me of this poem by Sherman Alexie (because otherwise I don't have any familiarity with what the poem refers to)

I was actually not familiar with that poem!

this poem is one of my favourites, so thanks for reminding me of it in a very unexpected context haha

@Zanna I agree that a shell loop is fine. As for rename, it seems to me that the main point of rename is its versatility -- and its excellent -n option, which apparently rename.ul has not analogue to, at least going by the rename.ul manpage.

I think of rename's performance as a secondary and only sometimes relevant benefit -- I don't have any problem with calling it many times in a loop. If there are a huge number of files, then doing it that way may be too slow. But often there are not enough files for it to matter.

@Zanna You're welcome! :)

Btw, I have mostly completed a draft answer to that question, fully explaining what is going on with the "Bareword not allowed" errors, and also explaining exactly what the shell is doing with the original commands, i.e., explaining what the rename command actually sees.

Although there's quite a bit of material in my draft that I like, overall I... hate it. It's too long and confusing! I am usually quite at peace with writing enormous answers that cause people to say, "That answer by Eliah is enormous!" so when I feel that I have written something overly verbose and cumbersome, I usually take heed of that feeling. I say "usually" but this has actually only happened once before (and I did not post the answer).

sometimes I don't like my answers but I post them anyway if I think there is anything valuable in them

Well there's something valuable in this one...

I think I should probably move the stuff about that renaming question to the island. How do you feel about this?

Also, do you want to see my draft?

@EliahKagan yes, makes sense

@EliahKagan yes!

@Zanna That will make all the other stuff being talked about here way easier to understand. I will move the messages shortly.

I better edit my answer... feeling pretty lazy today

Your answer to that same file-renaming question?

@EliahKagan yes. I am editing it, but I need to link to your chat comment. Wondering if the permalink changes when the message gets moved (it wouldn't be a permalink if it did...)

← 36 messages moved from Raiders of the Lost Downboat

@Zanna I'm pretty sure that would've worked even before I moved these messages. As far as I have observed, when a chat message is moved, links to the specific message still work, and they point to the message in whatever transcript currently has it.

Ah, you are here! Welcome to the Island, @BhargavRao!

thanks :)

@Zanna Here's the draft (or here it is in a pastebin, if you prefer):

Other answers have addressed one of the two major aspects of this question: that of how to successfully carry out the renaming operation you needed. The purpose of this answer is to explain why your commands did not work, including the meaning of that weird "bareword not allowed" error message in the context of the `rename` command.

###When `rename` gives you weird error messages, add "Perl" to your search.

In Debian and Ubuntu, the `rename` command is a [Perl][1] script that performs file renaming. On older releases--including 14.04 LTS, which is still supported as of this writing--it wa…

Wow! There really is effectively no limit to chat message length when the message contains at least one line break!

lol crazy

Anyway I'm guessing you'll want to paste that into some kind of editor that renders it (or that puts syntax highlighting on the Markdown). But now that it's here, it's CC-BY-SA. I do intend, if possible, to turn it into an answer that works -- unless someone else wants to -- but if not, at least it will be here and usable for the future (though admittedly somewhat obscure).

@EliahKagan that was a good design decision...

@EliahKagan After this many edits, I can more or less render Markdown mentally XD

:)

@EliahKagan I don't see anything bad about that answer... it seems very useful

the last section especially is interesting (maybe that only seems most interesting to me because I wanted to know what the error message meant)

I fear one must already know a significant amount of Perl to understand it, and also that perhaps nobody wants that level of detail in connection with the rename command. Also, when I started writing the answer, I did not anticipate that the ideas in it would take that long to expound on.

I think you should definitely post it

@BhargavRao welcome to the island! (belatedly)

The island seems spooky to the non technical folks :P

Spooky, you say?

Kinda :D

"Non-technical folk" sounds like a genre of music. Like folk music, but somehow less technical. :)

Soon there'll be non-technical rap music

@Zanna I feel like my answer can't make up its mind about what background knowledge readers are expected to have. At the beginning, it gives links for concepts like variables. Later, it refers to "type," "built-in function," and "subroutine" with no explanation.

hahaha

maybe you should add some explanations of those things. But maybe by the end the answer was getting worried about its length

I may be able to add links.

More links will hopefully help.

On the other hand, if someone who starts out totally unfamiliar with the material follows all the links, they might then spend most of their day with my answer!

if I don't know what a built-in function or a subroutine is, I can still understand that they are things the program already knows about

well, it would be a productive day if they wanted to learn about this topic...

Are you sure you don't? Built-in functions in Perl (and yes, I'm adding that link to the draft answer too) include things like print, while a subroutine is something you define with:

sub whatever_name_you_want {
    # some code should go here
}

I think I do have some idea of what those things are, I was just trying to imagine that I didn't

@EliahKagan ah :)

Oh.

Gotcha.

sorry

?

for creating confusion as usual

I am unsure of the terminology "strictures."

I have seen it but I don't know if it's considered correct or, even so, if it is recommended.

Originally I had written "strict semantics" but that would be a very poor term (also, that appears to have just been my mistake; I cannot find other sources that have used that term for strictures in Perl).

strictures sounds a bit forbidding XD

but it is short and makes sense

perldoc strict just calls them "restrictions." That's also short and makes sense. Is that less forbidding than "strictures"? :)

yes it sounds a bit less unpleasant to me haha

I want to split the What the Perl interpreter means by "bareword not allowed" section into two sections. But I didn't see a good place to do it. Do you have any thoughts on this?

I should possibly not call it strictures.

@Zanna There was no ambiguity there, btw. I just failed utterly to actually read what you had said properly. My eyes glazed right over the "if"! :(

@EliahKagan I think that, if that section cannot be split in two, then the material in it--and, more broadly, in the answer as a whole--probably cannot be absorbed by readers who don't already know most of it.

Similarly, if it can be split in two but doing so would require some rewriting, then that might actually be excellent, as it might fix problems.

@EliahKagan ah, agreed

I don't really see why that section needs to be split into two sections... What needs to be separated out?

Well, the section is very long, and also very dense. Also, it covers at least two discernibly separate topics, though the topics are related, and I am not sure there is a clear break between them in the draft. It covers how strings are usually represented in general-purpose programming languages (and how that is different from the BAM!: A Thing And Its Name Are Not Different Concepts After All way of shells). Then, it introduces sigils and barewords in Perl, which is really a third topic!

I don't really understand what you meant by

> the main purpose of quoting in Perl is not to protect strings, but to create them in the first place

@Zanna How do you feel about this, for that paragraph and the next?

In a shell, we quote our strings to protect them from unintended shell expansions that would otherwise transform them into other strings automatically (see the section above). [Shells are special purpose programming languages that work very differently from general-purpose languages][29] (and their [very weird syntax and semantics][30] reflect that). But Perl is a [general-purpose programming language][31] and, like most general-purpose programming languages, the main purpose of quoting in Perl is not to *protect* strings, but to mention them at all. This is actually a way most programming …

or rather, restrictions

yes that is much better, I see what you mean now

Yeah, I will probably change "strictures" to "restrictions"... or to something.

@Zanna Can you easily see where all those reference-style hyperlinks in my Markdown document point to. That is, when you're reading, are you able to see the URL to get a sense of how I am supporting what I am saying? (I am not asking you to follow all the links, I'm just asking if you can see about where they go.)

yes I can see that in your draft

I really mean to ask about how easy it is to check them.

well, it's the same as usual... I guess I don't know what you mean!

I guess that section maybe shouldn't be broken up. The first section is about the relationship between rename and Perl and how the code argument is used. The second section is about how the shell performs expansions--specifically globbing-- to construct an argument list. The third section is about what is going on in Perl code that gives "Bareword not allowed" errors. The fourth section is a summary of all the steps that take place between entering the command and getting the error.

What do you think?

And if that does make sense, then is there some way for me to include the meta-insight of what I just said in the post itself?

the sections as they are feel right to me

you can add your meta insight to the introductory paragraph of your answer I think...

Good point.

@Zanna Thanks for your help! I have posted the answer.

I don't think I did anything! But great news that you posted it