Beginner level introduction to command line (part 2)

Zanna

Oct 18, 2020 15:01

meanwhile we better open a terminal

just shout out if you are stuck!

last week we made a directory for playing around in

nobody

Am I late ?

Zanna

looks like you are right about on time :)

αғsнιη

wait, wait just arrived

jokerdino

wonderful crowd

Zanna

So last week I made a new directory in my home for playing around. If you were here, you might have done the same and still have the directory, and if so you can just cd into it

Otherwise, make sure you are in your home directory and then create a new directory and then cd into it, like

technastic_tc

Oct 18, 2020 15:04

cd playroom ?

Zanna

cd
mkdir playroom
cd playroom

@technastic_tc perfect, that's exactly what I'm going to do

so now my prompt is like zanna@peach:~/playroom$

none of those commands output anything

technastic_tc

ok..

Zanna

unless they fail...

but last week we ran some commands that produced output!

user3140225

I'm here

Zanna

yay

when you get some output from a command it appears in the terminal

(obviously)

But we can also send the output to a file using redirection. The redirection operator is this thing >

Since we are in this playroom directory where we can create files and clean up later without making a mess of our home or accidentally overwriting anything, we can safely redirect output from commands to create some files to play with. For example

echo $USER > name

We didn't see the output in the terminal because it was redirected elsewhere.

Now run ls here and see that it shows the file name

nobody

Oct 18, 2020 15:08

what if output is a stream?

technastic_tc

@Zanna name

jokerdino

yes same

Zanna

@nobody hmm we can try it out in a minute maybe...

If the target of redirection doesn't exist, it creates a file with the name (the path) of the target (even if there is no output to redirect - then we just get an empty file).

Now see what is in the file name by running cat name.

jokerdino

it's my username

technastic_tc

@jokerdino same

Zanna

Oct 18, 2020 15:11

cool

try this

technastic_tc

can we use less name?

Zanna

echo randomness > name

jokerdino

it got replaced!

with randomness

Zanna

@technastic_tc yes, definitely, but it's a very short file, and then we have to hit q to escape, so unless you really don't want the contents of the file in the terminal, it's easier to cat it

@jokerdino :D

technastic_tc

@Zanna ok

Zanna

Oct 18, 2020 15:13

@jokerdino yes, the file got overwritten. We have to be careful with redirection!

To prevent redirection from overwriting files, you can make it append to the file instead by writing >> instead of >. This also works whether the file exists or not:

echo oregano >> tomato

jokerdino

aha

user3140225

can we add some cheese?

Zanna

Do you have a file tomato" with the text "oregano" in it?

αғsнιη

bash: syntax error near unexpected token `>'

@Zanna I get error

technastic_tc

So echo command types text in a file?

Zanna

Oct 18, 2020 15:15

@αғsнιη hmm did you run exactly that command

αғsнιη

@Zanna there should not be any space between first & second > ?

terdon

@αғsнιη You probably had a space between the two >.

jokerdino

@user3140225 yum

i ate the cheese, nothing to add

terdon

terdon@tpad ~ $ echo oregano >> tomato
terdon@tpad ~ $ echo oregano > > tomato
bash: syntax error near unexpected token `>'

Zanna

@technastic_tc echo just outputs its input. Output redirection sends command output to files. So echo didn't create the file, it was the shell

αғsнιη

Oct 18, 2020 15:15

@terdon ahh, yes, I had one space between > >

terdon

The '>>' is a single token, it isn't two times '>', it is one time '>>'

technastic_tc

@Zanna ok..

user3140225

@jokerdino hmmmm

Zanna

Let's add something else to see if we can keep the oregano

@user3140225 you can add whatever you like :)

nobody

Is the command tee -a also working

Zanna

Oct 18, 2020 15:16

echo cheese >> tomato

@nobody yes, we are coming to that

nobody

sorry

user3140225

@Zanna Thanks!

Zanna

:D

We have just been using echo, but redirection is really useful for storing the output of any command.

In general the output you see in the terminal isn't saved anywhere - when you close the terminal it's just gone.

If you want to save the output of a command you need to store it in a file, which you can do with redirection. Try this:

uname -r > kernel

Now we created a file kernel. What is inside?

technastic_tc

@Zanna 5.4.0-51-generic

αғsнιη

@Zanna 4.4.0-18362-Microsoft

Eliah Kagan

Oct 18, 2020 15:19

4.15.0-112-generic

jokerdino

@αғsнιη same as this

Zanna

The uname command gives information about the version of the kernel. The -r option makes it output only the version string. This is the first Linux command I learned, so it's a friend of mine :)

Artur Meinild

5.4.0-51-generic

nobody

5.4.0-51-generic

karel

5.4.0-51-generic

Zanna

Oct 18, 2020 15:20

same for me

jokerdino

same not for me

Artur Meinild

Just checking if ppl has updated their kernel :-D

jokerdino

so we learned cool stuff about redirection and smartly redirection

Zanna

Now sometimes you want to see the output of a command in the terminal as well as send it to a file. For that there is the tee command as mentioned by @nobody , which we can use with another useful thing, a pipe:

uname -a | tee kernel

nobody

not tee -a ?

jokerdino

Oct 18, 2020 15:22

this tee overwrites the stuff in my kernel

αғsнιη

Linux XXXfilteredXXXX 4.4.0-18362-Microsoft #1049-Microsoft Thu Aug 14 12:01:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

Zanna

yes, it overwrites stuff too, just like redirection

user3140225

Linux HP 5.4.0-51-generic #56-Ubuntu SMP Mon Oct 5 14:28:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Zanna

but you also got the output in the terminal, right?

jokerdino

and yeah, it also thinks I am on PST

whereever that is

Zanna

Oct 18, 2020 15:23

hahaha

jokerdino

@Zanna yes it did

Artur Meinild

Pacific :-)

αғsнιη

@jokerdino I don't what that PST mean too

Zanna

I got UTC

Artur Meinild

UTC for me too

technastic_tc

Oct 18, 2020 15:23

UTC

jokerdino

yoU Take Care

αғsнιη

@jokerdino maybe Please Stop Terminal output? :D

Zanna

To make tee append instead of overwriting, use the -a (append) flag, as @nobody said

lol

Artur Meinild

hehe

jokerdino

@Zanna nice trick

nobody

Oct 18, 2020 15:25

I try to slow down

jokerdino

thanks @nobody

Zanna

the punning part of my brain has been switched off for the duration of the lesson I think

I'm relying on y'all

This long vertical line character | is the pipe. What a great thing it is!

It takes the output of the command on the left of it and turns it into the input of the command on the right of it.

So, like a water pipe makes water flow on further, a shell pipe makes command output flow on further...

jokerdino

is that what people mean when they say pipe it down?

Zanna

hahaha

jokerdino

btw, the time part did not change at all

maybe it be like that

Zanna

Oct 18, 2020 15:27

@jokerdino I think that means mouth-a close

@jokerdino when what? I think I have missed something

jokerdino

5 mins ago, by αғsнιη

Linux XXXfilteredXXXX 4.4.0-18362-Microsoft #1049-Microsoft Thu Aug 14 12:01:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

terdon

@jokerdino The time refers to the time the kernel was originally compiled at. It won't change until you change your kernel.

Which is also why the time zone isn't necessarily your own.

jokerdino

That makes sense

Zanna

I don't think I have ever bothered to read all that uname -a info before

nobody

hahahaha

Zanna

Oct 18, 2020 15:30

Anyway, we can use piping in, like, a zillion other ways. For example, if the output of a command is long, we can pipe it into less to make it more readable, just like we can use less to read files.

I am worrying about people missing the session last week where we talked about less

I guess it's OK

jokerdino

it's ok

nobody

yep

αғsнιη

I never liked '|' askubuntu.com/q/564380/283843

Artur Meinild

I know less - wait that doesn't sound right.. :-D

Zanna

Anyway last week we used the command printenv (we were having fun with that)

its output is pretty long, so you can try running printenv | less

I have mostly LS_COLORS

jokerdino

Oct 18, 2020 15:33

yep

user3140225

@Zanna So much fun!!!!

Zanna

how do we get out of less?

@user3140225 :D

jokerdino

you reboot the computer?

αғsнιη

q

Artur Meinild

@jokerdino haha

Zanna

Oct 18, 2020 15:34

@jokerdino hahahaha I thought that was how you get out of vim

terdon

@αғsнιη That's... something you really should get over. Pipes are one of the most basic, fundamental tools of the shell. The <() alternative you were given there is also often useful, but it isn't portable (not all shells support it) and it is far more complicated than a pipe for no real benefit (if you're just using it to avoid pipes).

Zanna

pipes are the best

Artur Meinild

@αғsнιη embrace the pipe

Zanna

The important thing is not to get confused between redirection and pipes. Output redirection* causes the output of a command to be written to a file. Pipes cause the output of a command to become the input to another command. That command might write a file (like `tee` does), but piping itself doesn't create or overwrite or append to files like redirection does.

* we have been using output redirection, but we can also redirect input from a file to a command.

2

user3140225

Speaking of pipes: youtube.com/watch?v=CpSgveTDtso

Zanna

Oct 18, 2020 15:36

lol unavoidable 5 minutes break XD

jokerdino

5 minute break it is

Artur Meinild

@user3140225 Or blue man group

user3140225

@ArturMeinild Cool!

jokerdino

ok so we can pipe to get input from an output and use content of a file too?

what else is there??

Artur Meinild

@jokerdino Nothing else, 'cos everything is a file :-D

jokerdino

Oct 18, 2020 15:39

Everything is a file.

Artur Meinild

hehe

αғsнιη

@ArturMeinild yeah, I am, like a pipe way |:|

Zanna

@nobody mentioned redirecting streams

That's what I usually do at the beach, like, build some little sand fortifications and stuff

but I think that you are always redirecting a stream

jokerdino

yeah

Artur Meinild

Because the std output is a stream

Zanna

Oct 18, 2020 15:42

like, what we were doing echo something > file redirects the standard output stream

if you want to discard warnings and errors from a command you can redirect the standard error stream to the black hole /dev/null, like command 2> /dev/null

@ArturMeinild yeah

nobody

sometimes I'll try to redirect the output of a command and get an emty file.

Zanna

Anyway we already used most of the time!!

Artur Meinild

So just to elaborate on the above: > redirects stream 1 (std output) and 2> redirects stream 2 (std error)

Zanna

@nobody like, maybe you redirected the wrong stream?

@ArturMeinild exactly

nobody

I wished I could remember.

jokerdino

Oct 18, 2020 15:45

@ArturMeinild what if I want both?

Zanna

no worries... you can test it out later and report back :)

terdon

@nobody There's no other option. If the file is created (and simply '> file' with no command will create the file, or empty it if it exists), and is created empty, then you redirected something with no output.

Artur Meinild

@jokerdino Then you have to do some redirection between streams, and this gets a little more tricky

I'm sure I can find an askubuntu Q&A that covers this

terdon

@jokerdino redirect stderr to stdout and then redirect stdout to the file: command > outFile 2>&1

Eliah Kagan

@Zanna How shall we use the remaining ~15 minutes?

Artur Meinild

Oct 18, 2020 15:46

@terdon Right on

Zanna

I will hopefully come up with some fun piping stuff later

jokerdino

@Zanna oops

Eliah Kagan

What I find to be nice about pipes is that one can chain as many commands as one needs:

printenv | nl | less

terdon

'&1' is "stream 1" (OK, it's actually the file descriptor for stream 1, but never mind). So 2>&1 will redirect stream 2 (stderr) to stream 1 (stdout)

Zanna

@EliahKagan yess

@EliahKagan was going to talk about some expansion. Do you want to talk instead?

terdon

Oct 18, 2020 15:49

Here's some further reading on some of the things Zanna mentioned:

Pipes and redirection operators: https://askubuntu.com/q/447772/85695
More detail than you ever wanted on all control and redirection operators: https://unix.stackexchange.com/q/159513/22222
Redirecting stdout and stderr: https://askubuntu.com/q/625224/85695

Eliah Kagan

@Zanna No.

I mean I don't want to talk instead.

jokerdino

we got another 10 minutes I think XD

Eliah Kagan

Please proceed with expansions! :)

Zanna

I was going to talk about brace expansion which has nothing much to do with any of the foregoing haha

try this

echo {a..z}

αғsнιη

it prints a to z

jokerdino

Oct 18, 2020 15:51

O_O

Zanna

also try

mkdir {01..04}

and ls to see the directories we created

αғsнιη

01 02 03 04 kernel name

jokerdino

I have a tomato as well

2

Zanna

me too hahaha

yum

Artur Meinild

@jokerdino Nice :-D

Zanna

Oct 18, 2020 15:52

Brace expansion works for anything the shell can identify as a sequence. You can also use brace expansion with commas for things that Bash can't recognise as sequences:

terdon

@Zanna No, we only do Ubuntu here, no SuSe or anything else.

3

Zanna

echo {one..three}

doesn't work

echo {one,two,three}

works

@terdon lol

that wasn't very useful... I mean it was more typing than echo one two three

jokerdino

can I use this to find a series of prime numbers?

Zanna

er... probably yes

Artur Meinild

@terdon Yum is the package manager in Redhat, in case anyone missed that ;-)

Zanna

Oct 18, 2020 15:54

@jokerdino you can do that for homework

jokerdino

@ArturMeinild what about dnf?

terdon

@jokerdino for i in {1..100}; do echo $i | awk '$1%2!=0'; done

jokerdino

woah what

terdon

You asked!

jokerdino

that is probably in part 24 of this lesson

terdon

Oct 18, 2020 15:55

:)

Zanna

hahaha

αғsнιη

: )

Artur Meinild

@jokerdino :-D

terdon

That wasn't primes though, just odd.

Zanna

Anyway the comma thing is actually useful... see how we can attach some common part to the brace

echo {fi,wi,ga}dget

Artur Meinild

Oct 18, 2020 15:56

@Zanna Cute example ;-)

αғsнιη

@terdon Ohh, I found an easy way for that echo {1..100..2}

jokerdino

are these real words?

@αғsнιη O_O_O_O

terdon

That means "print the numbers between 1 and 100, in increments of 2"

Zanna

@jokerdino I think so

Artur Meinild

@jokerdino They are

Zanna

Oct 18, 2020 15:57

@αғsнιη cool, I didn't know that would happen :D

If you put a space between the brace and the common part, it doesn't work:

zanna@peach:~/playroom$ echo {soy,oat,almond} milk
soy oat almond milk

jokerdino

mmmhm

Zanna

but you can escape/quote the space if you need one, as usual:

zanna@peach:~$ echo {soy,oat,almond}\ milk
soy milk oat milk almond milk

jokerdino

lottsa milk

Zanna

haha yes, we got extra milk for our typing

One nifty use for this is to avoid typing the whole path of a file twice

try this

echo /etc/default/grub{,.old}

Eliah Kagan

/etc/default/grub /etc/default/grub.old

Zanna

Oct 18, 2020 16:00

yeah :)

We can use that with the cp command to make a backup, for example. We'll use cp another time!

technastic_tc

shell-tips.com/bash/seq-brace-expansion

Zanna

nice!

jokerdino

@technastic_tc cool thanks

Artur Meinild

I use this one: devhints.io/bash

Zanna

very nice!

technastic_tc

Oct 18, 2020 16:02

@jokerdino welcome!

Zanna

we ran out of time again

Eliah Kagan

@ArturMeinild Nice!

user3140225

@ArturMeinild Hey! That's beautilful! Thanks!

terdon

@ArturMeinild Please don't get into the habit of using CAPITALS for variable names though. That's a bad idea.

Eliah Kagan

I use gnu.org/software/bash/manual/bash.html#Brace-Expansion :)

Artur Meinild

Oct 18, 2020 16:03

@EliahKagan You read the manual - that's cheating

Eliah Kagan

True.

jokerdino

@EliahKagan what is this document?

user3140225

@EliahKagan Hardcore

jokerdino

@Zanna oh noo

Eliah Kagan

@Zanna Personally I would be okay with running over. But should the date/time for part 3 be announced/determined first?

jokerdino

Oct 18, 2020 16:04

but does it mean we are getting part 3 next week??

technastic_tc

@Zanna :-\

Artur Meinild

@terdon Agree - that part may be a little confusing, you're right

Zanna

Can we do 30 minutes earlier next week, so we finish 30 minutes before @ArturMeinild 's session?

αғsнιη

@Zanna it's fine for me

thank you

Zanna

:D

Eliah Kagan

Oct 18, 2020 16:05

@ArturMeinild People will tend to assume they are meant to be exported as environment variables. Also they are more likely to collide with same-named environment variables. Environment variables can be of any case, but they are most often named in shouty case (all caps).

jokerdino

@Zanna sounds good

terdon

@ArturMeinild It's only in a few of the examples, but it's a good idea to always use lower case variable names for your stuff since the global environment variables are in CAPS, so you can get unexpected results if there is a naming collision.

Artur Meinild

@EliahKagan Yep I know

terdon

So, @jokerdino, since you asked, here's a way to get primes (that actually prints primes):

Artur Meinild

@terdon I didn't make it, but I should gladly ask the author to correct it

terdon

Oct 18, 2020 16:06

$ for i in {1..30}; do echo $i | awk '{ prime=1; for(i=$1-1;i>1;i--){if ($1%i==0){ prime=0}}} prime'; done
1
2
3
5
7
11
13
17
19
23
29

Eliah Kagan

@jokerdino The bash reference manual. It's longer, and also more friendly, than the manpage. You can also read it locally if you have the bash-doc package installed, by running info bash. The best thing to bookmark for the bash reference manual (at least what I find best) is gnu.org/software/bash/manual because that links to multiple formats.

user3140225

@Zanna Should be fine for me too.

technastic_tc

@terdon cool..

Eliah Kagan

@Zanna Yes, this works for me.

Zanna

great!

terdon

Oct 18, 2020 16:07

Of course, the {1..30} is only incidental, but still.

jokerdino

ok round of applause for @Zanna for another successful lesson!

Eliah Kagan

Thank you @Zanna! I look forward to next week.

Zanna

the next bit is related, but also pretty long, so I am OK with finishing here and trying to pick it up next time

Ask Ubuntu Classroom

Participants