/dev/chat

1:56 AM

semi-idle curiosity, for anyone who has >= 25k fake internet points; do the site analytics here show the kinds of browsers that hit U&L? The help page doesn't say explicitly. A random yahoo article indicate it is/was available. I'm working on a Meta post and I'm wondering what the impact would be.

cas

4:34 AM

@JeffSchaller I've never even bothered to look at the site-analytics before...but it doesn't seem to have any User-Agent stats. It has two tabs. The "History" tab has three graphs, for Posts, Votes, and Traffic. The "Traffic Sources" tab has a pie chart for Traffic Source (divided into referrer, search engine, and direct), another pie chart for Top 10 Referrers, and some lists of Referring Sites and Search Engines.

Michael Homer

The site analytics page has virtually no useful information. It's basically just "which search engines are popular" and some question trends

cas

The number one referer for U&L is duckduckgo.com, with stackoverflow.com a distant second.

user9303970

10:30 AM

Someone wants to review the organization/architecture of my Bash program in GitHub? I invested so much in this, asked so many questions here, I was shouted upon on so many times. I feel this is quite ready (it might have a code problem I'll fix asap) but I think my organization of the file system (and some files) could improve a bit. If you want to give a few words on the code, please do.

This is basically a Ubuntu-Nginx-WordPress environment bootstrapper.

github.com/benqzq/uncwe

Jeff Schaller

10:44 AM

Thank you, cas and Michael!

Kusalananda

@user9303970 The conf/assignments.sh script write to the user's .bashrc. This may be a bit surprising for the user. The local/imb.sh script dumps all databases on the local machine. This may be result in a possible breach of privacy. It is really hard to follow the various scripts as they all use variables defined elsewhere. The scripts themselves also have unintuitive names.

local/tdb.sh kills a tmux session, but it's unclear why (this too may be a bit surprising for a user using tmux).

crontab/crontab.sh replaces the crontab of the user with its own crontab.

crontab/weekly.sh deletes mail?

And log files!

terdon

And all of this needs to be run with root privileges and passes the -y ("say yes to all the things") option to apt, so can cause all sorts of compatibility issues. . .

@user9303970 I would suggest you just keep this for yourself. It's a great way to teach yourself about this sort of thing but is absolutely not something you want to be distributing to others. You know how you want your system set up, so you now have something that does it for you (although I would strongly urge you to remove the -y from the apt commands), so yay!

Kusalananda

install.sh does rm -rf ${repo}/.git but the repository is located in ~/$repo/

Kusalananda

11:04 AM

@user9303970 Do not take the above in the wrong way. I'm fairly used to reviewing all sorts of code and I'm a picky reviewer at the best of times. I do appreciate your questions on the site as they are usually well written. The scripts in the repository, however, presents some issues, mostly for whoever runs them blindly (as root) without knowing what their side effects are.

Suggestions for improvements include collecting some of the small scripts into larger scripts carrying out several tasks (maybe just one script, possibly utilizing local shell functions), and making sure that side effects (such as replacing any crontab for the user and deleting mail and log files) are avoided, or at least are clearly documented.

Also, restricting the operations that needs to be carried out by root somehow would be a good thing.

user9303970

Thanks @terdon I totally agree this suites to be kept to myself. This is indeed meant, by principle, for personal usage. @Kusalananda Much thanks! I wanted all variables to always be available (I sometimes use them for aliases or other purposes and cannot know when I'll need one. the tdm alias is to (temporarily) unlock phpmyadmin hence usage in tmux and killing of tmux session after 2 hours of using phpmyadmin.

I delete all logs and mail each week just not to have accumulative data that will grow and grow (I've seen mail files of thousands of lines and by principle I want to keep the SSD free as possible).

Fixed rm -rf mistake.

Kusalananda

@user9303970 Log files are best rotated by logrotate or similar tool.

user9303970

11:29 AM

@Kusalananda do you think my install.sh can run in a better way? I feel a bit frustrated from running it line by line because it worked fine in the last times I checked (before creating the mistake I know fixed).

I think something in install.sh can be done better.

terdon

11:51 AM

for one thing, what's the point of having assignments.sh as a separate file? it's only 4 lines!

for another, each time you run this it will re-add the same lines to bashrc

So if you run it 3 times, then, each time you open a terminal, it will re-source all those scripts 3 times. The more you do it, the more your system will slow down.

user9303970

Yes, you are right about assignments.sh. Seems I went too-modular there. Sorry for not mentioning I used to run it with curl but I'm having many problems running that remote script with curl.

terdon

To be honest, the entire thing is too modular. You have I don't know how many separate scripts sourcing each other and setting up variables. That just makes it very very hard to follow and understand and introduces a whole level of complexity you don't need.

I especially don't understand the point of setting up aliases that you only use inside the same script. Just use the command directly, or save it in a variable or something. Aliases are useful when doing things interactively, what's the benefit of using them in a script?

Or will you be sourcing the assignments.sh file from your terminal at other times as well?

user9303970

@terdon I no longer use assignments.sh (deleted it after reading what Kusalananda said. I know do the source form install.sh.

About aliases in scripts --- I know it is not possible, I don't do that... Where did you see me run an alias in a script?..

terdon

@user9303970 github.com/benqzq/uncwe/blob/master/templates/aliases.sh

user9303970

I just declared them for later use as I generally use all of them interactively.

terdon

12:00 PM

OK, but then you have things like this:

#!/bin/bash
cat ~/${repo}/templates/nginx_default > ${s_a}/default

That's your uncwe/conf/nginx.sh

user9303970

Why is it bad? This saves me of changing the default Nginx file by hand each time.

terdon

So you have a separate script that runs one command, why? And you are using two separate variables that aren't defined there, so you have no way of knowing where they are defined, where they come from, what happens if either of them isn't set or is set to a bad value?

You also have most (all?) of your variables unquoted which is a very bad idea.

205

Q: Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some file names). I encountered a problem such as the following: I have a file name containing a spa...

user9303970

They are exported in the variables file that gets exported each time I start using the OS.

terdon

148

Q: Security implications of forgetting to quote a variable in bash/POSIX shells

If you've been following unix.stackexchange.com for a while, you should hopefully know by now that leaving a variable unquoted in list context (as in echo $var) in Bourne/POSIX shells (zsh being the exception) has a very special meaning and shouldn't be done unless you have a very good reason to....

bash shell shell-script security quoting

@user9303970 Yes, but when you come back to this in 3 months time, you won't remember and will have to track everything down again.

Wait, each time you start the OS?

How?

user9303970

Oh, you mean to ${myVar} instead "$myVar"?

terdon

12:04 PM

@user9303970 Yes. Or "${myVar}" if you like, but never $myVar or ${myVar} unquoted.

user9303970

I believe I'll remember, honestly... This is just a small personal environment I maintain regularly (as you said, it's best for personal usage and not to be shared with anyone, I'll usually work differently for a non-personal project).

Each time I start the OS because the variables file is sourced in ~/.profile.

terdon

@user9303970 Even for personal stuff, you really want to get into the habit of always quoting your variables. I strongly urge you to read the two questions I pasted above.

@user9303970 Then it is read each time you log in, not each time the OS is started.

user9303970

Oh there is a misunderstanding here @terdon I do intend to quote them. And thanks!

Oh sure, each time I login.

Sorry...

terdon

Anyway, the single most important thing you really must fix is the -y option for apt. That is an awful idea. It means it will silently say yes to any prompt and you absolutely don't want that when installing things like this since it can lead to a broken system.

Think "Package A conflicts with packageB, remove packageB?" and you say yes automatically but need packageB.

user9303970

@terdon I install an extremely small number of well commnual packages...

terdon

12:09 PM

Oh, also, these lines:

user9303970

Why should I not automate that process?...

terdon

@user9303970 Yeah. That's what we all think until one of them breaks our system.

@user9303970 Because things change. And you are using PPAs which can stop being maintained at any time.

And these lines then:

printf "\n%s" "source ~/${repo}/templates/variables.sh" >> ~/.bashrc
printf "\n%s" "source ~/${repo}/templates/aliases.sh" >> ~/.bashrc

Apart from being added over and over to bashrc each time you run this, they will result in a bashrc that doesn't end with a newline. You want printf "%s\n" not "\n%s"

user9303970

Only certbot, I don't plan to use any other PPA besides that.

I run install.sh only ocne.

once*

I'll fix this.

Only certbot, I don't plan to use any other PPA besides that, on this machine*

terdon

You could also do something like grep "source.*variables.sh" ~/.bashrc || printf "%s\n" "source ~/${repo}/templates/variables.sh" >> ~/.bashrc

That way at least it will only be added if it isn't there.

@user9303970 So? What happens if certbot causes a conflict? Package conflicts are a common problem.

user9303970

I assume that I'll just recreate it's installation script or use different software.

terdon

12:24 PM

No, with the -y option it will just do whatever apt thinks is best which isn't always a very good solution.

user9303970

12:41 PM

@terdon Okay, I'll see what I'll do then, in such case. Maybe I'll use Ansible by then. Either way, it's unlikely with Certbot which is the only and will 99% keep being be the only PPA I'll install here. Thankfully a VPS rebuild is a matter of a few seconds (I also usually have external weekly backups and I also tend to make immediate ones).

Again I emphasize that I'll usually not conduct this way with any other PPA and especially in a non-personal project and I agree with the notion that when working with PPAs, -y is usually should be avoided.

Kusalananda

(sorry for not being able to take part in the discussion, I'm busy with work)

Transcript for