/dev/chat - 2018-05-16

« first day (2776 days earlier) ← previous day next day → last day (2161 days later) »

7:05 AM

@terdon Have you ever run the Ensembl VEP predictor?

2 hours later…

8:39 AM

@Kusalananda Yes, a few times. Mostly because I needed to see how it works while writing up an article about varsome.com which does a similar job and I needed to compare the two.

Haven't run the command line version though.

But what do you need?

@terdon I'm just running the command line version of it for the first time. I haven't used the VEP at all previously. I just wondered, but I guess it's hard to say, whether using its --everything flag would be overkill, and whether you knew what it would do "by default".

I stopped working for Ensembl at around the time when the VEP was first written. I have had no use for it until now. I have ta add VEP annotations to a VCF file...

Species?

Hooman

Well, if this is something you will be doing often, I strongly recommend our API.

varsome.com/information/varsome-api-info

Of course you do :-)

8:50 AM

But if this is a once-off thing, you may as well stick with VEP

And no, I can't really help. I've played around with the online version, but that's about it.

It's something that we'd expect our customers to have done, but this particular customer didn't, so we're doing it instead.

Oh well. I'll run and I'll se what it looks like in our browser once it's done.

2 hours later…

11:07 AM

General question: is it reasonable to have text in an email message go out as far as 97 char? Let's round up and say 100? And if not, then how much is reasonable? Are there any general guidelines.

97 chars in a line? Most email clients (i.e. MS stuff) don't seem to insert hard newlines anyway, so it doesn't really matter what you send...

11:21 AM

@Kusalananda I tried echo "hello !!". You were right. Salute

! is also special (by itself) at the start of a pipeline, it inverts the exit code. Though that's not really something that would get confused with anything. I don't think anyone's been insane enough to create a command called ! yet.

Anyway, I originally quoted it without thinking about it too much. Too much quoting seldom does any harm in things you want to send to the command literally.

11:35 AM

@ilkkachu You mean it will wrap?

Might get confusing. Emacs by default seems to wrap around 70.

I didn't doubt it

@ilkkachu I wanted to know why he wrote ! is safe in shell. Then he told me about it. That's it. I didn't doubt I just asked

@FaheemMitha why Linux mint demands passwd for everything. For every small thing it asks for passwd.

@FaheemMitha, mm, I mean that anyone who uses an email client that doesn't handle overly long lines, is in for trouble anyway, since they're going to get messages with even longer lines from other sources.

@FaheemMitha The guideline used to be to wrap at 72 characters (that is, you wrap it there). The MUA would probably not touch the contents once it's submitted.

@ilkkachu That's me.

Well, not the client, but my terminal window...

@PrabhjotSingh Sounds like a configuration issue. But nobody can help you without more information.

@Kusalananda 72 sounds reasonable.

I was just going to say that with today's GUI's and screen size, it shouldn't be too hard to stretch the terminal to at least 120 characters...

11:45 AM

@PrabhjotSingh You will need to provide your password for anything that requires root permissions, even if that thing is just changing a single character in a file.

@ilkkachu Hmm. Not everyone has large screens. Particularly in India.

but maybe even more importantly, there's probably quite a bit more variance in terminal widths, than there was when "everyone" had an 80-character terminal

How to install GNU health in Fedora 27

@ilkkachu Sure, but that makes texts even more unreadable :-P

My personal terminal is way more than 72. But that's not something that one can expect.

11:49 AM

@Kusalananda in CentOS i haven't seen that it demands passwd for everything. Even Fedora is good in this regard.

@PrabhjotSingh You really need to be more specific about "demands passwd for everything".

Provide at least one example of "everything".

@PrabhjotSingh If you're using sudo, then you can configure sudo to allow you to use it without a password (bad idea), or to remember your password for a certain amount of time. I'm sure this has been discussed in a Q/A on the main site several times.

@Kusalananda He hasn't actually given any examples of what he means.

When I tried to rename some files, install packages, even formating a USB flasdrive.

@PrabhjotSingh Using sudo, right?

11:54 AM

@PrabhjotSingh Rename what files? Did you own them?

Installing packages requires a password, yes.

Formatting a USB flash drive may or may not.

That's not surprising. All those things are administrational tasks (if the files are owned by root).

Slightly related: unix.stackexchange.com/questions/126721/…

It all depends on permissions.

As I recall, sudo "remembers" the password for a little while, but only per shell, or something. I'm not sure how that works, exactly.

Installing packages requires passwd. Agreed. But what is disturbing is even writing a USB flash or formating it requires passwd.

In CentOS or Fedora I haven't seen this.

Ni, no I'm not using sudo

@PrabhjotSingh Depends on permissions. What are the permissions? And what password is required?

I see in permissions.

@FaheemMitha Tell me how to install GNU health.

12:09 PM

@PrabhjotSingh Instructions and downloads are at health.gnu.org

Why sudo won't remember my password? That question is very close to my Linux mint question.

Glad someone has asked this.

Got it. Documentation at health.gnu.

org

So many easier question than Unix&Linux on SuperUser superuser.com/questions/tagged/linux I might consider making it my new home to finally get enough reputation to accomplish my evelish plans

Also for people who loves music I need help. I've heard this youtube.com/watch?v=L2kGNdE0b5Q And I'm sure the piano parts has been stole from an other group like supertramp but I can't manage to find exactly to what it makes me think. Any help will be rewarded with a nice Thank You

1 hour later…

1:37 PM

@Kiwy What evelish plans are those?

@FaheemMitha it comes and goes, I have thousands evil plan a day, but most of them end up impossible to accomplish. Also I'm too lazy to take over the world... way to much work IMO

2:17 PM

@Kiwy Then consider a smaller world.

@Kusalananda But the ultimate goal was to became the master of at least the milkyway

a nice master of the galaxy though. Not too much of tyranny

@Kiwy Persistence is the mother of Success.

2:34 PM

@Kiwy "Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon." Susan Ertz

woops, I meant to reply to @Kusalananda's line about "consider a smaller world"

There's also this -- Homer: "Lisa, the mob is working on getting your saxophone back. But we've also expanded into other important areas. Literacy programs, preserving our beloved covered bridges, world domination..." --Lisa: "World domination?" --Homer: "Oh ho, heh, that might be a typo." [thinks] Mental note: the girl knows too much.

2

2:55 PM

@FaheemMitha And Success has a severe identity crisis.

is it possible to integrate bash history to syslog so any command could be log ? I once again lost some command in my history and I hate that

Why do people insist on using ${var} for shell variable expansion? It's annoying.

@Kusalananda I do I tend to always do so I don't have to think about when I need to use ${} :D

Urgh

@Kiwy On a system I have access to, the default PROMPT_COMMAND string is: _siginttrap=trap -p SIGINT

; trap SIGINT; history -a >(tee -a ~/.bash_history | logger -p local5.info -t "$USER[$$] $SSH_CONNECTION"); eval "$_siginttrap"

Bah, backticks are wonky.

Anyway, the logger and the tee is the main bit you may be interested in.

I dont mind doing it locally it's a bit confuse to me, I will watch that carefully later. But thank. I think I'm going to do something like that soon enough

I can't bare loosing great idea that once cross my mind but can't remind of :D

@Kusalananda thanks

3:03 PM

The trapping is just for ignoring Ctrl+C during the actual logging operation, and then to restore it.

@Kusalananda No idea what that means.

@FaheemMitha Just that "Success" means different things to different people, and has a tendency to change for individuals over time too.

@Kusalananda Oh, I see.

3:23 PM

@Kusalananda I like the habit, in case I ever want to print a _ or alphanumeric next to a variable

What's annoying about it? Using it when it's not required?

3:36 PM

@JeffSchaller Some users seems to think that's how you quote a variable.

Or that it's somehow "safer"

@Kusalananda I though too until you learn me the truth,
Once I was blind , yet now I see

3:50 PM

@Kusalananda, I couldn't agree more about success. So starred.

@Kiwy Oh, be careful with my truths. They come with context.

4:05 PM

This is very strange. In May my ISP says that my daily usage (I have a monthly cap), has dramatically increased. E.g. for 12th May it said 23,590.22 MB. For other days in May it says between 10 to 15 GB.

I do watch video, but don't do any downloads. And I don't watch video that much - usually in the evenings. And often Amazon Video.

I'm running iftop, and the last line is showing as:

TOTAL:                 68.3MB           2.69Mb

But I'm not doing anything.

Now:

TOTAL:                 89.5MB            3.31Mb

Any idea what might be going on?

Is there any utility that can pinpoint net usage more precisely. By process, perhaps?

Hmm, a search found this:

141

Q: How do I find out which process is eating up my bandwidth?

I think I'm being the victim of a bug here. Sometimes while I'm working (I still don't know why), my network traffic goes up to 200 KB/s and stays that way, even tough I'm not doing anything internet-related. This sometimes happens to me with the CPU usage. When it does, I just run a top comman...

networking bandwidth

Um, that's odd. The top process appears to be ssh.

Apparently downloading at 200kb/sec.

How is that possible?

Oh, never mind. That's my backup process running.

Oh, maybe the increased usage is from borg. But why so much?

@FaheemMitha What did you do on the 12th of May? Did you reorganize large files that were picked up by borg?

@Kusalananda I don't remember. I don't think so.

It does look likely that Borg is involved. I see I changed my remote backups to back up much more stuff on around the 25th April. And that's when my usage skyrocketed.

But a incremental backup shouldn't be pushing so much stuff. What am I doing wrong?

I mean, I rarely change stuff. And borg is supposed to backup intelligently. Break stuff up into small chunks and so forth.

OS update? that could touch many files

Let me see if I can get anyone's attention on #borg. I'm a bit surprised my ISP hasn't cut me off already.

@JeffSchaller I'm only backing up personal stuff. No system stuff.

And the stuff isn't that large, at that.

I don't suppose borg has a log of what it's doing?

4:21 PM

Mind you, I'm doing it quite frequently, but that shouldn't make much difference. Because it's supposed to only transfer the difference, if any.

@JeffSchaller Yes, it does.

Ok, the other weird thing is that the downloads have massively increased, not the uploads.

But of course a remote backup is an upload, by definition.

@FaheemMitha Are you doing consistency checks as well, at the same time as the backup?

@Kusalananda Yes, I am.

maybe do that less often?

Every time.

@Kusalananda Is that causing the problem?

That could account for downloads... possibly.

I do hourly backups, but only one verification run per day.

4:26 PM

@Kusalananda Hmm. Any idea of any easy way to check for that? How do you handle it?

@Kusalananda Separate cron job?

We talked about this recently I think, or a couple of weeks ago. I have a script that takes a backup or check command line option. I schedule the backing up hourly and the check just after midnight in two separate cronjobs.

@Kusalananda So you did.

If it's not too terrifying complex, could I take a look at how you are doing it?

Your hypothesis seems likely. Ssh has incoming around 200-250 kb/sec. I can't imagine what else could be causing this.

I'm using restic though, not borgbackup...

@Kusalananda I recall. But the structure is what is relevant.

I guess I need to do that now. At some point my ISP is going to notice, if they haven't already. Sigh.

@FaheemMitha The gist of the script is

# environment variables here

case "$1" in
    backup)
        # command for backing up
        ;;
    check)
        # command for verification
        ;;
esac

4:31 PM

And I had lots of exciting plans for this evening. Just kidding.

@Kusalananda Does that suffice for a command line argument?

Ah, right, $1 is the first argument.

Does that come from C, or did C take it from shell?

Sure. Just ./script backup to back up, and ./script check for verification.

@FaheemMitha What? The $1? I don't actually know where it comes from.

@Kusalananda Ah. Could be an interesting historical question.

And yes, the $1.

Actually, I don't think C uses that exact thing.

@FaheemMitha argv[1]

@FaheemMitha No, it has an argv array of argc strings.

@StephenKitt Yes, right. Thank you.

@Kusalananda Yes, so not the same thing.

4:35 PM

@FaheemMitha Well, actually, the $1, $2, ... may well come form the indexing into argv.

@Kusalananda Indexing? Does shell have argv too?

@FaheemMitha it's just called $@ :)

@JeffSchaller Ah

@FaheemMitha No, I meant that $1 etc. might have gotten their names form the indexes used in C to index into the argv array.

s/form/from/

@Kusalananda Ah, ok.

4:42 PM

I'm uncertain of when command line parsing (using the positional parameters) was first introduced though. C did not exist until 1973, and Unix was rewritten in C then. What the shell was capable of at that point, I don't know.

It may have existed in Algol 68 before that, or Pascal. I dunno.

There are probably people here who know.

Boot image=/vmlinuz4.16.7-200.fc27.i686+PAE just crashed

I'll ask on the site, so as to avoid fiddling with my backup scripts for a few minutes...

@Kusalananda Feel free to improve, edit, or otherwise modify - unix.stackexchange.com/q/444201/4671

5:07 PM

@FaheemMitha Did a modification. I hope I did not destroy anything.

The syntax of the shell as a whole comes from Algol 68 (or Algol 68C to be more precise). That fact is pretty much established.

@Kusalananda Looks good.

2 hours later…

7:02 PM

@Kusalananda Your case statement would require the script to be written in two parts, backup and check.

While that is fine, I wondered if there was an obvious way to still keep the backup and check sections togther, for each volume to be backed up.

7:16 PM

@FaheemMitha You have multiple volumes (disks?) to back up?

@Kusalananda They're not multiple disks. Just multiple directories. In one case, just a file, really.

Wouldn't you be able to do borgbackup create somerepo /dir1 /dir2 /dir3?

@Kusalananda I suppose. But it seems easier to keep them separate.

I'm someone here has the answer to this delicious zsh question :D

1

Q: Why does a value '-' (hyphen) in "tag-order" style prevent matching of the "executables" tag even if that tag is explicitly specified?

With the code below, everything works as expected: When I write c and hit the TAB key, I get the appropriate completions in correct order. (Image) When I write ./ and hit the TAB key, I get the completion for the executables tag. (Image) # Always use menu selection when using unambiguou...

zsh autocomplete tab-completion

@Kusalananda Actually, I've forgotten what a repo is, exactly.

7:29 PM

@FaheemMitha Sorry, the manual uses /path/to/repo::Monday as an example, so I said "repo".

I was looking at borgbackup.readthedocs.io/en/stable/quickstart.html

@Kusalananda Yes, actually I think that's an archive. And a repository holds multiple archives.

@FaheemMitha Duh, words.

Which you can see by using borg list reponame.

@Kusalananda Yes, words make things complicated.

7:58 PM

@Kusalananda I run my remote backup every 4 hours.

I'm now doing the check once every 24 hours.

I wonder if I should do it less.

8:11 PM

@Kusalananda I wonder if these two lines are the problem:

mapfile -t mailarchives < <(borg list --short 'faheem@ramnode:/mnt/backup-Mail')
borg extract -n 'faheem@ramnode:/mnt/backup-Mail'::"${mailarchives[-1]}"

I forget what that mapfile lines does.

I don't even know what mapfile does.

Oh, this mapfile command appears to be reading the entire list of archives from the repository from the remote. Which actually doesn't seem like it should generate so much traffic, but appears to be.

Unless it's the command after that. Which is extracting an archive from remote. I wonder how much my mail directory is.

@FaheemMitha It's the same as readarray. The first line creates an array, mailarchives, whose values are the directories that you have backed up, or something.

Hmm, 3.4 GB. Every 4 hours, that's 6 times a day.

The second line extracts the last directory.

3.4 * 6 = 20.4

I think that's my bandwidth right there.

I wonder, is it actually necessary to run the extract locally?

Thoughts?

What is this code snippet part of?

8:18 PM

@Kusalananda Actually, it's the list of archives. Which even if thousands, should not take more than a second or two.

@Kusalananda My backup script.

The extraction is only done on the last directory.

@Kusalananda The last archive.

That list is a list of archives. I think -1 is the most recent.

Ah, archive. Which is all mails?

I really should annotate my scripts.

@Kusalananda Yes, except for my INBOX, which is fairly awful.

So you're basically testing whether you can restore from backup.

8:20 PM

@Kusalananda For just mails, yes.

That's not something you need to do very often I think.

But do you think it's possible to do the extract remotely, but send the result of the extract back to the local machine?

Ugh, brain doesn't work.

That extraction is insanely expensive from the pov of bandwidth usage, even if only done once a day.

@Kusalananda Huh?

If you haven't solved this by tomorrow, ask me again and I'll look at it. I need to sleep now.

8:22 PM

@Kusalananda Ok. I doubt I'll solve it by myself. But it is a sufficiently clear question to ask on the site?

Or would I need to generalise it?

It's 29 degrees indoors and I'm melting. I think it's better that I melt in bed than into the keyboard.

@Kusalananda You don't have A/C?

@FaheemMitha I think it may be a good question.

@FaheemMitha This is Sweden, we deal with -20 degrees. We're not used to +30.

@Kusalananda Those are the best kind. But should I put it in terms of borg? Or something simpler?

Or more general?

@FaheemMitha No, it should probably be a specific borgbackup question. That would make it easier to answer and it would help you more.

8:24 PM

@Kusalananda I don't think I'd enjoy -20 C. Celsius, right?

I think I'd rather take 29 C instead.

@Kusalananda Except that everyone who is never used Borg, which is basically everyone, would have no answer how to answer it. That would probably leave, well, you.

But sure, I'll ask it. Why not?

@FaheemMitha Cold is better, you just put more clothes on and crank up the radiator. With too hot, you take off clothes, but when it's still too hot then there is a problem.

@Kusalananda 29 C isn't really that bad.

Depends on the humidity.

@FaheemMitha There is a borgbackup tag... and a backup tag.

@Kusalananda There is? Huh. But maybe I created the former.

I should check.

@FaheemMitha Luckily the humidity is low.

8:27 PM

@Kusalananda Luckily.

Ok. I'm off to bed.

Yes, I probably created that tag. I wrote the first question with that tag.

@Kusalananda Have a good night.

I don't understand

1

Q: Can I change ownership of the data directory content in a borg backup repo?

I've been using borg-backup for the past few weeks (on Ubuntu 16.04). Every time I need to mount a backup I've used sudo (sudo borg mount ...); if I don't, I end up with a permission error and the borg filesystem is not mounted. The problem is that afterwards, the mounted filesystem requires su...

sudo backup ownership borgbackup

What's he on about?

I guess I should read it.

Actually, the poster doesn't seem to be around, so maybe not very useful.

I just got this weird error page: unix.stackexchange.com/error?aspxerrorpath=/users/260540/…

What's a good generic terms for the amount of bits downloaded off the net? Data seems common, but is a good general term?

1 hour later…

9:59 PM

@Kusalananda unix.stackexchange.com/q/444250/4671

« first day (2776 days earlier) ← previous day next day → last day (2161 days later) »

Transcript for

May '1816

/dev/chat

General discussion for unix.stackexchange.com. If you have a q...

join this room
about this room

00:00

06:00

12:00

18:00

all times are UTC

site design / logo © 2024 Stack Exchange Inc; legal

mobile