Hi everyone!

hello

Hi :)

welcome everyone to week two of Ask Ubuntu 10th anniversary classroom event.

:)

:)

:)

:)

(:

oops

O: xD

I'm just going to ping the people who are faded..

lurkers

@Theo @gparyani @user3140225 @terdon @CookieWeeb ping

meanwhile we better open a terminal

just shout out if you are stuck!

last week we made a directory for playing around in

Am I late ?

looks like you are right about on time :)

wait, wait just arrived

wonderful crowd

So last week I made a new directory in my home for playing around. If you were here, you might have done the same and still have the directory, and if so you can just cd into it

Otherwise, make sure you are in your home directory and then create a new directory and then cd into it, like

cd playroom ?

cd
mkdir playroom
cd playroom

@technastic_tc perfect, that's exactly what I'm going to do

so now my prompt is like zanna@peach:~/playroom$

none of those commands output anything

ok..

unless they fail...

but last week we ran some commands that produced output!

I'm here

yay

when you get some output from a command it appears in the terminal

(obviously)

But we can also send the output to a file using redirection. The redirection operator is this thing >

Since we are in this playroom directory where we can create files and clean up later without making a mess of our home or accidentally overwriting anything, we can safely redirect output from commands to create some files to play with. For example

echo $USER > name

We didn't see the output in the terminal because it was redirected elsewhere.

Now run ls here and see that it shows the file name

what if output is a stream?

@Zanna name

yes same

@nobody hmm we can try it out in a minute maybe...

If the target of redirection doesn't exist, it creates a file with the name (the path) of the target (even if there is no output to redirect - then we just get an empty file).

Now see what is in the file name by running cat name.

it's my username

@jokerdino same

cool

try this

can we use less name?

echo randomness > name

it got replaced!

with randomness

@technastic_tc yes, definitely, but it's a very short file, and then we have to hit q to escape, so unless you really don't want the contents of the file in the terminal, it's easier to cat it

@jokerdino :D

@Zanna ok

@jokerdino yes, the file got overwritten. We have to be careful with redirection!

To prevent redirection from overwriting files, you can make it append to the file instead by writing >> instead of >. This also works whether the file exists or not:

echo oregano >> tomato

aha

can we add some cheese?

Do you have a file tomato" with the text "oregano" in it?

bash: syntax error near unexpected token `>'

@Zanna I get error

So echo command types text in a file?

@αғsнιη hmm did you run exactly that command

@Zanna there should not be any space between first & second > ?

@αғsнιη You probably had a space between the two >.

@user3140225 yum

i ate the cheese, nothing to add

terdon@tpad ~ $ echo oregano >> tomato
terdon@tpad ~ $ echo oregano > > tomato
bash: syntax error near unexpected token `>'

@technastic_tc echo just outputs its input. Output redirection sends command output to files. So echo didn't create the file, it was the shell

@terdon ahh, yes, I had one space between > >

The '>>' is a single token, it isn't two times '>', it is one time '>>'

@Zanna ok..

@jokerdino hmmmm

Let's add something else to see if we can keep the oregano

@user3140225 you can add whatever you like :)

Is the command tee -a also working

echo cheese >> tomato

@nobody yes, we are coming to that

sorry

@Zanna Thanks!

:D

We have just been using echo, but redirection is really useful for storing the output of any command.

In general the output you see in the terminal isn't saved anywhere - when you close the terminal it's just gone.

If you want to save the output of a command you need to store it in a file, which you can do with redirection. Try this:

uname -r > kernel

Now we created a file kernel. What is inside?

@Zanna 5.4.0-51-generic

@Zanna 4.4.0-18362-Microsoft

4.15.0-112-generic

@αғsнιη same as this

The uname command gives information about the version of the kernel. The -r option makes it output only the version string. This is the first Linux command I learned, so it's a friend of mine :)

5.4.0-51-generic

5.4.0-51-generic

5.4.0-51-generic

same for me

same not for me

Just checking if ppl has updated their kernel :-D

so we learned cool stuff about redirection and smartly redirection

Now sometimes you want to see the output of a command in the terminal as well as send it to a file. For that there is the tee command as mentioned by @nobody , which we can use with another useful thing, a pipe:

uname -a | tee kernel

not tee -a ?

this tee overwrites the stuff in my kernel

Linux XXXfilteredXXXX 4.4.0-18362-Microsoft #1049-Microsoft Thu Aug 14 12:01:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

yes, it overwrites stuff too, just like redirection

Linux HP 5.4.0-51-generic #56-Ubuntu SMP Mon Oct 5 14:28:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

but you also got the output in the terminal, right?

and yeah, it also thinks I am on PST

whereever that is

hahaha

@Zanna yes it did

Pacific :-)

@jokerdino I don't what that PST mean too

I got UTC

UTC for me too

UTC

yoU Take Care

@jokerdino maybe Please Stop Terminal output? :D

To make tee append instead of overwriting, use the -a (append) flag, as @nobody said

lol

hehe

@Zanna nice trick

I try to slow down

thanks @nobody

the punning part of my brain has been switched off for the duration of the lesson I think

I'm relying on y'all

This long vertical line character | is the pipe. What a great thing it is!

It takes the output of the command on the left of it and turns it into the input of the command on the right of it.

So, like a water pipe makes water flow on further, a shell pipe makes command output flow on further...

is that what people mean when they say pipe it down?

hahaha

btw, the time part did not change at all

maybe it be like that

@jokerdino I think that means mouth-a close

@jokerdino when what? I think I have missed something

@jokerdino The time refers to the time the kernel was originally compiled at. It won't change until you change your kernel.

Which is also why the time zone isn't necessarily your own.

That makes sense

I don't think I have ever bothered to read all that uname -a info before

hahahaha

Anyway, we can use piping in, like, a zillion other ways. For example, if the output of a command is long, we can pipe it into less to make it more readable, just like we can use less to read files.

I am worrying about people missing the session last week where we talked about less

I guess it's OK

it's ok

yep

I never liked '|' askubuntu.com/q/564380/283843

I know less - wait that doesn't sound right.. :-D

Anyway last week we used the command printenv (we were having fun with that)

its output is pretty long, so you can try running printenv | less

I have mostly LS_COLORS

yep

@Zanna So much fun!!!!

how do we get out of less?

@user3140225 :D

you reboot the computer?

q

@jokerdino haha

@jokerdino hahahaha I thought that was how you get out of vim

@αғsнιη That's... something you really should get over. Pipes are one of the most basic, fundamental tools of the shell. The <() alternative you were given there is also often useful, but it isn't portable (not all shells support it) and it is far more complicated than a pipe for no real benefit (if you're just using it to avoid pipes).

pipes are the best

@αғsнιη embrace the pipe

Speaking of pipes: youtube.com/watch?v=CpSgveTDtso

lol unavoidable 5 minutes break XD

5 minute break it is

@user3140225 Or blue man group

@ArturMeinild Cool!

ok so we can pipe to get input from an output and use content of a file too?

what else is there??

@jokerdino Nothing else, 'cos everything is a file :-D

Everything is a file.

hehe

@ArturMeinild yeah, I am, like a pipe way |:|

@nobody mentioned redirecting streams

That's what I usually do at the beach, like, build some little sand fortifications and stuff

but I think that you are always redirecting a stream

yeah

Because the std output is a stream

like, what we were doing echo something > file redirects the standard output stream

if you want to discard warnings and errors from a command you can redirect the standard error stream to the black hole /dev/null, like command 2> /dev/null

@ArturMeinild yeah

sometimes I'll try to redirect the output of a command and get an emty file.

Anyway we already used most of the time!!

So just to elaborate on the above: > redirects stream 1 (std output) and 2> redirects stream 2 (std error)

@nobody like, maybe you redirected the wrong stream?

@ArturMeinild exactly

I wished I could remember.

@ArturMeinild what if I want both?

no worries... you can test it out later and report back :)

@nobody There's no other option. If the file is created (and simply '> file' with no command will create the file, or empty it if it exists), and is created empty, then you redirected something with no output.

@jokerdino Then you have to do some redirection between streams, and this gets a little more tricky

I'm sure I can find an askubuntu Q&A that covers this

@jokerdino redirect stderr to stdout and then redirect stdout to the file: command > outFile 2>&1

@Zanna How shall we use the remaining ~15 minutes?

@terdon Right on

I will hopefully come up with some fun piping stuff later

@Zanna oops

What I find to be nice about pipes is that one can chain as many commands as one needs:

printenv | nl | less

'&1' is "stream 1" (OK, it's actually the file descriptor for stream 1, but never mind). So 2>&1 will redirect stream 2 (stderr) to stream 1 (stdout)

@EliahKagan yess

@EliahKagan was going to talk about some expansion. Do you want to talk instead?

Here's some further reading on some of the things Zanna mentioned:

@Zanna No.

I mean I don't want to talk instead.

we got another 10 minutes I think XD

Please proceed with expansions! :)

I was going to talk about brace expansion which has nothing much to do with any of the foregoing haha

try this

echo {a..z}

it prints a to z

O_O

also try

mkdir {01..04}

and ls to see the directories we created

01 02 03 04 kernel name

I have a tomato as well

me too hahaha

yum

@jokerdino Nice :-D

Brace expansion works for anything the shell can identify as a sequence. You can also use brace expansion with commas for things that Bash can't recognise as sequences:

echo {one..three}

doesn't work

echo {one,two,three}

works

@terdon lol

that wasn't very useful... I mean it was more typing than echo one two three

can I use this to find a series of prime numbers?

er... probably yes

@terdon Yum is the package manager in Redhat, in case anyone missed that ;-)

@jokerdino you can do that for homework

@ArturMeinild what about dnf?

@jokerdino for i in {1..100}; do echo $i | awk '$1%2!=0'; done

woah what

You asked!

that is probably in part 24 of this lesson

:)

hahaha

: )

@jokerdino :-D

That wasn't primes though, just odd.

Anyway the comma thing is actually useful... see how we can attach some common part to the brace

echo {fi,wi,ga}dget

@Zanna Cute example ;-)

@terdon Ohh, I found an easy way for that echo {1..100..2}

are these real words?

@αғsнιη O_O_O_O

That means "print the numbers between 1 and 100, in increments of 2"

@jokerdino I think so

@jokerdino They are

@αғsнιη cool, I didn't know that would happen :D

If you put a space between the brace and the common part, it doesn't work:

zanna@peach:~/playroom$ echo {soy,oat,almond} milk
soy oat almond milk

mmmhm

but you can escape/quote the space if you need one, as usual:

zanna@peach:~$ echo {soy,oat,almond}\ milk
soy milk oat milk almond milk

lottsa milk

haha yes, we got extra milk for our typing

One nifty use for this is to avoid typing the whole path of a file twice

try this

echo /etc/default/grub{,.old}

/etc/default/grub /etc/default/grub.old

yeah :)

We can use that with the cp command to make a backup, for example. We'll use cp another time!

shell-tips.com/bash/seq-brace-expansion

nice!

@technastic_tc cool thanks

I use this one: devhints.io/bash

very nice!

@jokerdino welcome!

we ran out of time again

@ArturMeinild Nice!

@ArturMeinild Hey! That's beautilful! Thanks!

@ArturMeinild Please don't get into the habit of using CAPITALS for variable names though. That's a bad idea.

I use gnu.org/software/bash/manual/bash.html#Brace-Expansion :)

@EliahKagan You read the manual - that's cheating

True.

@EliahKagan what is this document?

@EliahKagan Hardcore

@Zanna oh noo

@Zanna Personally I would be okay with running over. But should the date/time for part 3 be announced/determined first?

but does it mean we are getting part 3 next week??

@Zanna :-\

@terdon Agree - that part may be a little confusing, you're right

Can we do 30 minutes earlier next week, so we finish 30 minutes before @ArturMeinild 's session?

@Zanna it's fine for me

thank you

:D

@ArturMeinild People will tend to assume they are meant to be exported as environment variables. Also they are more likely to collide with same-named environment variables. Environment variables can be of any case, but they are most often named in shouty case (all caps).

@Zanna sounds good

@ArturMeinild It's only in a few of the examples, but it's a good idea to always use lower case variable names for your stuff since the global environment variables are in CAPS, so you can get unexpected results if there is a naming collision.

@EliahKagan Yep I know

So, @jokerdino, since you asked, here's a way to get primes (that actually prints primes):

@terdon I didn't make it, but I should gladly ask the author to correct it

$ for i in {1..30}; do echo $i | awk '{ prime=1; for(i=$1-1;i>1;i--){if ($1%i==0){ prime=0}}} prime'; done
1
2
3
5
7
11
13
17
19
23
29

@jokerdino The bash reference manual. It's longer, and also more friendly, than the manpage. You can also read it locally if you have the bash-doc package installed, by running info bash. The best thing to bookmark for the bash reference manual (at least what I find best) is gnu.org/software/bash/manual because that links to multiple formats.

@Zanna Should be fine for me too.

@terdon cool..

@Zanna Yes, this works for me.

great!

Of course, the {1..30} is only incidental, but still.

ok round of applause for @Zanna for another successful lesson!

Thank you @Zanna! I look forward to next week.

the next bit is related, but also pretty long, so I am OK with finishing here and trying to pick it up next time

Thanks for this session all!

Thanks @zanna

and thanks everyone for making this week's lesson a great one @ArturMeinild @αғsнιη @EliahKagan @karel @user3140225 @terdon @technastic_tc @nobody

:)

thank you so much everyone for participating and being awesome!

:)

(phew I was just pinging everyone as well)

@Zanna We can't help it

Bye and thanks .

Thank you @Zanna and everyone!!!

Thanks for another interesting lesson.

UTC - cya

thanks a lot for joining @karel

@jokerdino For redirecting both stdout and stderr, Bash has: &>

It can also be written as: >&

@ArturMeinild haha likewise

(So long as you do not put a number immediately after the & with no space, since that has a different meaning.)

@Zanna XD

Bye!

@EliahKagan 😐

@EliahKagan I have a question regrading &> and >& differences

is it fine to ask here? or we can move to general room?

I think it's okay to ask here.

The answer there should give you everything you need (and probably more).

@EliahKagan { ls nonexistancefile existancefile; } &> >(sed 's/file/----/g')

sed 's/file/----/g' <(ls nonexistancefile existancefile >& )

sed 's/file/----/g' <(ls nonexistancefile existancefile &> )

What are you trying to do with these?

It looks like you want to pipe, but are instead using process substitution.

considering these 3 commands the first command having "&>" works just fine with process substitution, but the other two doesn't either ">&" or "&>"

but this is working

sed 's/file/----/g' <(ls nonexistancefile existancefile &> /dev/stdout )

The first command is works (in Bash), though you should write this instead:

ls nonexistancefile existancefile |& sed 's/file/----/g'

or evene

sed 's/file/----/g' <(ls nonexistancefile existancefile 2>&1 )

so what I understood after ">&" operator there must be always a file to redirect? or it's something related to process substitution ?

@αғsнιη you are not actually redirecting the output anywhere. The &> HERE means redirect output to HERE, but your command has no HERE

Why would you even do this though? Just use a pipe, this is what they're for.

If you must use the far more cumbersome and less portable process substitution, you need something like this:

$ sed 's/file/----/g' <(ls nonexistancefile existancefile 2>&1 )
ls: cannot access 'nonexistance----': No such ---- or directory
existance----

@terdon got it

@αғsнιη Yeah, the redirection operators that don't end in &n (where n is some number) need to have a filename to the right of them.

thanks @terdon @EliahKagan

Process substitution is not an exception to this. With ... as some command, Bash expands <(...) or >(...) to a path.

@terdon it's for a lesson I just wrote these examples how to pass stdout/stderr to process substitution

$ echo <(true)
/dev/fd/63
$ echo >(true)
/dev/fd/63

(It won't always be /dev/fd/63, of course. For example, if you have two separate process substitutions in the same command, their FIFOs will have different file descriptors, and at most one will be 63.)

hmm

Process substitution is really for the situation where you want to take input from the output of a command, or give output as the input to a command, but where you need to do so in a situation that doesn't directly support doing that but does support reading from or writing to a named file. Process substitution is a complicated way of using pipes, not an alternative to pipes. (Though it's true that it's not using a shell pipeline with |.) The path names a pipe.

(There are only a few situations where both a shell pipeline and process substitution will both work, yet process substitution has an advantage. I agree with terdon that is is best to prefer shell pipelines in cases where they are applicable.)

What's the question?

@EliahKagan nothing, seems I was doing something wrong and found the problem

thanks again both for your answers

:)

@nobody I wonder if the situation you were referring to is when you try to overwrite a file with the output of a command that reads the same file. Since the shell opens files for redirection before running a command, and > redirection truncates the file to zero bytes when it opens it, that will first erase the contents of the file, even before the command attempts to read it. This is a common mistake, and one that often, unfortunately, causes data loss (when the file is not backed up).

ek@Kip:~/playroom$ echo 'hello, world' >greeting
ek@Kip:~/playroom$ cat greeting
hello, world
ek@Kip:~/playroom$ sed 's/.*/\U&/' greeting
HELLO, WORLD
ek@Kip:~/playroom$ sed 's/.*/\U&/' greeting >greeting  # Very bad, erases the file!
ek@Kip:~/playroom$ cat greeting
ek@Kip:~/playroom$

@terdon thanks for this!

you're welcome

May I request one of us consider changing their room name?

if not, it's ok because

we all know that

:-)