@Fabby good stuff! I actually discovered EDLIN and used it a couple times for fun.

@derobert hah! How could I miss that. Memory banks, remind me next year.

@Fabby Interesting article. I'll need to re-read that again, as I looked at it in the morning. I've been saying for years that ETs exist, just not how most people think about them.

OK, I'm going to ask a probably stupid question, but why does buffering exist ?

Like what's the benefit of buffered reads/writes, instead of doing everything byte-by-byte ?

There's gotta be a reason why this has become a thing for most apps (and probably due to libc) and you need stdbuf to turn that off

@SergiyKolodyazhnyy It's many times faster.

@derobert I suspected that's the case, but it seems counter intuitive. Reading in buffers of 1024 is faster than reading every single byte ?

Yes. Each read or write is a syscall, so it costs a fair bit of overhead. If you use buffers (and 1024 bytes is tiny — they're normally larger), you get far fewer syscalls

Alright, so each syscall has also overhead of things happening behind the scenes, and that's why it's simpler to make one large read, than many smaller ones. Did I get that right ?

Faster, not simpler. The code to do it is often more complicated.

(And not a small amount faster, character-at-a-time read will probably not get above the tens or hundreds of KB/sec range; 64–512K at a time will probably hit the hardware limits)

Yeah, I've already realized that long ago with python (the more complicated code part, not hardware limits)

@FaheemMitha That user about the ISP question, both does not know how TCP/IP, outing, traceroute, and private/public IP addressing works...it is a quite vague and broad netowrking question dressed as a Unix question.

@RuiFRibeiro Not sure what question you're talking about.

@FaheemMitha Your previous comment about a router upgrade? hmmmm it puts me off you forgetting context.

@RuiFRibeiro Sorry, still not sure what you mean. And I never claimed to have a good memory. Anyway, I have a cold, and didn't sleep properly.

When referencing a question, it always helps to link to that question.

@SergiyKolodyazhnyy I asked, but did not get a response yet...

@JeffSchaller :D :D :D Thtat's only 20 years ago since I used that one!

Hi does anyone know how to disable SELINUX on grub?

Is speed the only reason for buffering?

@SergiyKolodyazhnyy Because of Floppy disks.

On a floppy, you need buffers.

On an NVMe, it's kinda pointless.

@Fabby Thanks.

YW

Interesting how nowadays with SSDs and tons of RAM, those techniques are becoming sort of pointless, but they still have good reasons behind them

Yet still with SSDs and multi-core CPUs you still have to know how to take advantage of them

@SergiyKolodyazhnyy Yup: buffered on an SSD will still be faster as RAM is still faster than said SSD.

And that's why databases like Redis run from RAM :)

Speaking of which I managed to install it couple days ago, and a python module for that

Now I just need to come up with some sort of project that will make use of Redis

So, Mercurial currently closes sufficiently old bugs as "RESOLVED ARCHIVED". I described this practice as deranged in a post to mercurial-devel. And one of the developers responded. He seemed to take exception to that word. I take exception to people closing bugs as RESOLVED when they have not, in fact, been resolved.

Happily, this practice seems to be rare. But I hope it does not gain in popularity.

This regarding bugs as a irrelevant nuisance strikes me as reminiscent of Microsoft.

In fact, Bill Gates once said that users don't care about "unimportant" bugs. For some value of "unimportant", presumably.

See this interview

> No! There are no significant bugs in our released software that any significant number of users want fixed.

> The reason we come up with new versions is not to fix bugs. It's absolutely not. It's the stupidest reason to buy a new version I ever heard. When we do a new version we put in lots of new things that people are asking for. And so, in no sense, is stability a reason to move to a new version. It's never a reason.

It's so gratifying that this man is a multi-billionaire. It warms my heart. It really does.

Well, when bugs cost money, companies listen. Otherwise it seems that developers have to close bugs just do make it manageable, while management breathes into their necks

@SergiyKolodyazhnyy Cost money in the sense of causing problems that cost money to fix?

Or so is my understanding. So far bug reports in my app indicators are mostly fixed and no management involved :) It's not like I know anything about it anyway

Note that Mercurial is (supposedly) a community project, not a corporate or proprietary product. Though in practice this seems less true than it was.

@FaheemMitha In the sense that cost money and may lead to company loosing customers

@SergiyKolodyazhnyy Costs some company money?

I think I will kill my computers and then kill my brain processes. Nothing is working today.

@RakibFiha Put it off for a while and approach it calmly next day. Or at least next hour. Now, there was one answer before, but for some reason author deleted it, so they might undelete it once the answer is polished. Among other things, the sudo error you're getting is specific to the system - you don't have sudo there. (You've probably already realized that commands passed to ssh are executed on target system, and whatever you have locally doesn't apply)

So you either have to install sudo on the host, or log in as root ( very bad idea, and root login should not be allowed on ssh in the first place)

@SergiyKolodyazhnyy ?

Command '.sudo' not found, did you mean:

  command 'sudo' from deb sudo
  command 'sudo' from deb sudo-ldap

Try: sudo apt install <deb name>

Oh, they've a dot there !!!

@RakibFiha ^

@RakibFiha please edit your question and remove all the obsolete stuff. We only need to know what the problem is now, not what it was before. Reading your question, I am confused about what part is fixed and what part is still an issue.

By the way, since the problem is the #, why didn't you just remove the #?

@terdon Quick question: can you spot a problem with unix.stackexchange.com/a/503723/85039 ?

@SergiyKolodyazhnyy You don't seem to be answering the question, really. You just explain what $- is not when it is set.

I think the OP wants something like "The variable is set automatically when you launch a new shell"?

I've edited with direct reference to OP's question. Really the answer is "it depends". Am I missing something ?

I'll probably book some time tomorrow to polish the answer, if the question or answer are still there.

@SergiyKolodyazhnyy I think the answer look good.

The question lacks an actual issue related to knowing where the variable is set.

@Kusalananda Thanks. I've dug through a few old comments to add more content to it.

@SergiyKolodyazhnyy I am the dumbest man here, I learnt that inside EOI, the tab is ok but spaces (which sometimes looks like tab) are considered as dots. I removed them and now working fine.

@RakibFiha Believe me, I've had quite a few moments like this.

Sometimes I can latch onto solving a problem, without realizing I'm solving something completely different.

@RakibFiha What?

terdon@tpad ~ $ ssh -tt localhost <<EOI
   echo "foo"
  echo 'bar'
 echo "baz"
EOI
Last login: Tue Nov 13 16:38:30 2018 from ::1
   echo "foo"
  echo 'bar'
 echo "baz"
Unable to connect to X server
[terdon@tpad ~]$    echo "foo"
foo
[terdon@tpad ~]$   echo 'bar'
bar
[terdon@tpad ~]$  echo "baz"
baz
[terdon@tpad ~]$

How are spaces considered dots?

     The following redirection is often called a “here-document”.

           [n]<< delimiter
                 here-doc-text ...
           delimiter

     All the text on successive lines up to the delimiter is saved away and made available
     to the command on standard input, or file descriptor n if it is specified.  If the
     delimiter as specified on the initial line is quoted, then the here-doc-text is
     treated literally, otherwise the text is subjected to parameter expansion, command

That's from dash's manual, though the same applies

I think the last line is the source of confusion. And confusingly enough, it's not even dot, it's a dash

Well, it's "Oh, no, I stayed till morning again" o'clock. Good times, good times. Have a good rest of your day, everyone !

@terdon I have no idea but it was considering spaces as dots, I was using vim

@SergiyKolodyazhnyy Thank you for giving me some hope.

@RakibFiha Wait, what? How does vim come into it?

Because, sometimes copying and pasting codes from computer to computer mah give issue

@RakibFiha Vim can be made to show tabs as e.g. dots.

Don't copy and paste cod to transfer it metween systems.

Just scp orc rsync files if they need transferring.

I'm guessing you dots are in fact visual queues in you editor, picked up by copying and pasting.

Spaces in the shell are never arbitrarily turned into dots.

Is GNU Hurd on-topic on U & L?

@Pandya I don't see why not however I did see a recent question about an OS under development closed with the reason that bugs should be reported to the developers and not here

So I guess it also depends on what the question would be

@Jesse_b ok. I've downloaded Debain Hurd Iso and looking for a way to burn USB stick or say make a bootable USB stick of it.

@Pandya I would probably just run a VM off of the ISO. But I assume you're looking to run it on bare metal?

@Kusalananda: Hopefully the third time will be the charm :p

@Jesse_b I've voted to close as unclear until it's not.

"What commands did you use?". "Emptied of all files except this file" (sigh)

@Jesse_b He seems to want to delete all files in that directory. Get him to run rm -rf on the directory. If that does not work, it will be a permission issue or something that will require fsck.

They're using a funny group name. I don't know if that ^ means anything special.

@Pandya It isn't technically Unix, is it?

@FaheemMitha It gives a Unix-like user experience, and that's what matters.

Just like WSL does.

There's also a hurd tag.

(you'll need to listen for at least 2 minutes, that's when it starts getting interesting)

@Kusalananda I wasn't aware that was the criterion.

@Pandya You might find posting on a more specialist forum more productive. Though there might be Hurd experts here I don't know about.

@FaheemMitha No it's not.

Apparently still on topic here, though.

ok

I just found that Hurd doesn't support USB

so, I need to burn ISO to DVD.

Do we have Hurd expert here?

@Pandya If you need serious help, I'd probably take it to the mailing lists.

There really aren't many people using Hurd, and some of them are probably reading the lists.

Let it be known, to all and sundry, that @StephenKitt used the word "sensible" in the context of systemd.

@terdon Sensible is a relative term.

You can't do indirect expansion with this sort of parameter expansion huh?

$ var1=one
$ var2=two
$ test=var1
$ echo ${!test:-var2}
one
$ unset test
$ echo ${!test:-var2}
-bash: test: invalid indirect expansion

hop-step in the middle with a itest=${test:-var2}; echo ${!itest}

> If the first character of parameter is an exclamation point (!), and parameter is not a nameref, it introduces a level of indirection. Bash uses the value formed by expanding the rest of parameter as the new parameter; this is then expanded and that value is used in the rest of the expansion, rather than the expansion of the original parameter

Thanks, yeah I was just wondering if I was missing something that could make it possible in the single step :P

Can probably figure out a way to make it easier with ed...

I use the editor that doesn't waste my VALUABLE time.

If you use ED, you are on THE PATH TO REDEMPTION.

So glad I'm generating this command from a script. xterm -class upgrade-em-all -title kunikida.gcloud.metrics.net -e ssh -t kunikida\.gcloud\.metrics\.net sudo\ \-\-\ bash\ \-c\ if\\\ command\\\ \\\-v\\\ apt\\\ \\\>\\\ \\\/dev\\\/null\\\;\\\ then\\\ […]

(That goes on for 1171 characters total)

script-generating scripts are the bomb

yo dawg, I heard you like scripts...

I re-wrote the shell script in perl. Works so much better now, and I get to write the remote shell script in normal form, not something inside a shell string.

Someday will no doubt switch to Chef/Puppet/etc., but... until then...

chsh -s /usr/bin/perl ?

lol, there is a Perl shell you can do that with somewhere.

I almost tried it.

gnp.github.io/psh (haven't tried it, and looks rather abandoned)

use Shell

(link: perldoc.perl.org/Shell.html)

@JeffSchaller I think that every time I see anyone suggesting using eval as a "good solution" to a shell scripting problem.

@Kusalananda which kind of bomb? youtube.com/watch?v=jQE66WA2s-A or unix.stackexchange.com/a/436251/117549 ? :)

@JeffSchaller I did not read the context to your comment, but now I did. Generating scripts is totally fine. Executing data as code is different.

The difference is not big.

@JeffSchaller Read this answer and cringe...

But then: It did what needed to be done at the time I needed it...

It's all about use-case...

@Kusalananda Well, executing data as code is sometimes needed too, just you ought to realize what a scary thing you're doing before you do it.

(Strictly speaking, if you write a shell script, you're executing data from your text editor as code...)

@derobert Ooh! I like that one! Starred!

@Kusalananda What's wrong with eval?

23:09 where he lives

he's probably asleep.

@FaheemMitha Code injections are possible with eval, so security reasons. If the script is locked with permissions and only evaluates hard-coded (and not user-input stuff) it iss sort of OK

It's also hard to do right. So even when you're running a script for yourself (e.g., not across a security boundary), it's easy to have input that you'd expect to work that doesn't. At best, you get an error. At worst...

That ^ Especially when quotes are involved

@SergiyKolodyazhnyy Ah, so this one is good then??? :-)

@Fabby Sort of alright :) Of course, there could be a foobar if $0 has been unlinked or moved or replaced with malicious script during execution . . . I think. $0 should remain in memory of the shell that is running , so whether or not path specified by $0 is still valid is up for a chance

@SergiyKolodyazhnyy :O :O :O

How can you modify $0

@Fabby It's not $0 that gets modified, not the variable (although it's possible - that's how shellcode injections work). But do this experiment:

If someone can modify or unlink your script out from under you, then you have bigger issues than eval. E.g., they could have just replaced the script before you hit enter to run it.

Make a script myscript.sh with contents as so:

@SergiyKolodyazhnyy Oh. Obviously you don't want to evaluate user input.

#!/usr/bin/env bash

while true ; do
    stat $0
    sleep 1
done

Launch it in one terminal tab

In another terminal tab either rm the script or mv the script to some other name

  File: ./myscript.sh
  Size: 66              Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 397653      Links: 1
Access: (0775/-rwxrwxr-x)  Uid: ( 1000/  adminx)   Gid: ( 1000/  adminx)
Access: 2019-03-01 22:32:11.165050405 +0000
Modify: 2019-03-01 22:32:08.917050405 +0000
Change: 2019-03-01 22:32:08.929050405 +0000
 Birth: -
stat: cannot stat './myscript.sh': No such file or directory
stat: cannot stat './myscript.sh': No such file or directory
stat: cannot stat './myscript.sh': No such file or directory

You'll see that the script running from memory still remembers path to file set in $0 , even though it's not there yet

@SergiyKolodyazhnyy even more fun, modify the file w/o unlinking it.

Now imagine if I overwrite myscript.sh with maliciou.sh

@SergiyKolodyazhnyy Ah!

@derobert Beaten me by a second :) Exactly that !

Got it...

@derobert thanks...

That's not a problem with eval though.

well, I think that by now I have no more of my GOTO scripts left...

@Fabby This is also why setuid scripts are bad. There's apparently a small window before interpreter loads and opens the script to read instructions. So in that window you could inject malicious code

Try it without eval. Shell only reads the lines in as it executes them. So you can modify lines it hasn't read yet.

@derobert so is my goto workaround that insecure?

(running a grep on / to see whether itùs still used)

It's still defined though... (prettyu sure about that)

Problem largely because shell itself has not many mechanisms to ensure security. Scripts are meant for automation, and not so much for secure application development. Though one can still do make small apps in bash. I've done it multiple times combining zenity, gsettings, and whatever else I could use

I think if someone else can write to your shell script, then you've already lost.

Same if they can unlink it and put a new one in place.

@derobert only root can.

I put all my tested scripts in /usr/local/bin.

I believe in global things instead of local things...

grep finished. Not used any more.

only defined in /etc/bash.bashrc

That eval doesn't look too bad, I'm not sure it actually works entirely ... but at least I doubt its a security issue.

if someone has root on my machine, I'm in deeper trouble than an eval()

@derobert I had a better man than me look at the original and modify it to be more robust...

some guy called @SergiyKolodyazhnyy

>:-)

What's good about compiled executables (think C language) is that 1) there's memory layout randomization, so it's harder to break program's memory and 2) if it's compiled, malicious user can't just edit the file directly

Provided, of course, that no (lesser-trusted) security context can write to the script. But if that were true, you'd already have lost.

@SergiyKolodyazhnyy Hah!

@Fabby Sounds familiar, I wonder where I've seen him before ;)

@SergiyKolodyazhnyy (2) is not true. It's a little harder, but just a bit.

I started on Z80 ASM, then C, then PASCAL.

I disassembled one of my C OBJ files to read the ASM and optimised that.

@derobert OK, hex editor can do that admittedly. Still, compared to scripts that's a little better against script kiddies

Disclosure: that was in the early 90s.

@SergiyKolodyazhnyy Sure. Unless the just happen to have their virus installer on hand. I mean, only one person has to write it, then all the kiddies can run it.

Well, that comes with the bag of problems called "system access". If attacker has access to the system, in worst case root or sudoer account, they can install things.

Computer Security is a fun topic about which I don't know enough, and probably could've/should've studied. Sadly no such major in my university has been available at the time.

@SergiyKolodyazhnyy Physical access = root access.

That's why we keep our DCs locked, our cages locked and our racks locked...

Essentially, yes. And full disc encryption could help a bit

@SergiyKolodyazhnyy Meh, physical security is easier...

Full disk encryption is a recovery nightmare.

On the same topic, I've just got simple xor encryption script written and working. Might be posted at some point on AU and Code Review :)

Better encypt folders you want to keep secure.

@SergiyKolodyazhnyy XOR 10101010 was my first encryption algorithm. CP/M days: 8bit

third one was based on e ('2.7182818...) up to 64K (Win16 386 days)

Second one was actually a compression algorithm

:) Well, I could do something along those lines as well. I've been looking for projects lately.

Pi and e are random.

Actually, I've plently of things I wanna learn, do , rewrite and improve existing code. I just need to figure out a way to manage my time

OK, I'll be out for like 40 mins. In case some of you guys are gonna be heading to bed, have a good night.

in 360° hindsight: if you knoew the algorithm, it was easily decodeable...

Good night!!!

@FaheemMitha The biggest issue is using eval on unsanitized input, like strings read from files or inputted interactively by a user. There's nothing wrong with using eval in itself, but I feel it's too often the go to solution and the failure conditions are very seldom even mentioned.

@Fabby Not asleep yet, but nearly. Was playing games...

>:-) which game?

I was playing gzdoom STRG.wad earlier today...

@Fabby Factorio.

Watching trailer...

@Fabby The trailer is old. The graphics is very different in the new 0.17 release. This gamer is good at playing it and I often watch her YouTube stuff: youtu.be/el9Ih1gg3i4

@Fabby Somewhat, yes, possibly. I haven't played StarCraft though.

Old StarCraft was the best game ever...

@Fabby false

@Jesse_b I don't like on-line games...

@Fabby Command and conquer red alert was the best RTS ever. Startcraft probably isn't even top 10

Never played that one...

warcraft 2 was the best rts blizzard released

I think I've played about 1100 hours of Factorio, but I've never "finished" the game.

But now I'm off to bed.

Take care.

@Jesse_b That was OK...

@Kusalananda Good night homeskillet!

I'm more of an SF than a Fantasy guy though...

Going to sleep here too!

Good night!!!

:-)