12:01 AM
@derobert hey thanks for the reply! Yeah but there might be better approach, I am searching since 1 hr. Other devices also pass absolute coordinates how do drivers work in that case.

14 hours later…
1:49 PM
In my actual data file, only about 4% of the lines contain a colon at all.

@MathematicalOrchid But do dupes always have one?

For every line that ends :FOO, there's always a dupe on the line before.
There shouldn't be any other dupes.

OK, there's something weird going on with the /:
$printf 'Foo/\nFoo/Bar\n' Foo/ Foo/Bar$ printf 'Foo/\nFoo/Bar\n' | uniq -f1
Foo/
Ah! OK, it looks like uniq takes any non-alphanumeric string as a delimiter.
Dang.
That's not what the documentation says though:
Fields are sequences of non-space non-tab characters that are
separated from each other by at least one space or tab.

Ouch! That's not fun...

Yup.
Let me see if I can think of a workaround.

1:55 PM
Maybe this is just too complex for a shell script... (?)
It looked simple, but it seems it isn't.

Oh, it's a simple awk or perl one liner anyway.

I don't grok awk. Maybe someday...
You want to comment your answer to help the next poor soul?

I'll fix it up as soon as I get a working solution.

@MathematicalOrchid how big are these files?
Would something like this be OK?
$grep -hFxv "$(grep ':FOO' file | cut -d: -f1)" file
red.7
green.2:FOO
blue.6
yellow.9:FOO
/Foo
/Foo/Bar/
/Foo/Bar/Baz
It needs two passes of grep (and also probably needs GNU grep)
Hey @steeldriver, nice sed solution. it really would benefit from an explanation though, sed labels and commands look like line noise to the uninitiated.

2:26 PM
What's the context?

2

I've got a knotty problem that I can't figure out how to solve. I have a text file containing a few million lines of text. Basically I want to run uniq, but with a twist: If two lines are identical but for a :FOO suffix, drop the line that lacks the suffix. But only if the lines are otherwise id...

2:46 PM
@terdon Ok. Thank you.

1 hour later…
4:04 PM
@StephenKitt What would a version just to exclude non-English locales look like?
The first part would presumably be:
# Drop locales except English
path-exclude=/usr/share/locale/*
path-include=/usr/share/locale/en/*
path-include=/usr/share/locale/locale.alias
And the second part:
# Drop translated manpages except English
path-exclude=/usr/share/man/*
path-include=/usr/share/man/man[1-9]/*
path-include=/usr/share/man/en*/*
Is that right?

@FaheemMitha wow, you working on embedded to care that much about <1G space?

If anyone else is interested, this is a reference to unix.stackexchange.com/a/281549/4671
@derobert No, but space adds up. Why not save it?
Especially if it is a one-time thing.

@FaheemMitha Configuration is not free (time to learn the syntax, write the config, merge your changes on upgrade, etc.) Neither is running non-defaults (more likely to hit bugs).

@derobert I agree that running non-defaults is more likely to hit bugs.
But Stephen provided the syntax. Do you see a problem with it? Or do you think localepurge is safer?

Looks fine... though somewhat pointless, unless you have a tiny disk.
/me notes his entire /usr is 18G, fairly small—and that's on a pretty bloated install.

4:13 PM
Parenthetically, what's the most efficient way of finding out the space used in a directory? I use du -hs, but it isn't exactly fast.

I don't think you're going to find anything faster.
Unix filesystems track the size of each file... they don't track it per-directory. So you have to stat each file, and total them up.

@derobert Hmm. All the filesystems?

@FaheemMitha Maybe there is an exception. But there isn't AFAIK any API to get that information out of the kernel, so even if there were a filesystem that did...
Many can track quotas. So you could get space per-user quickly.
Or per group.

@derobert I see.
I get du -hs -> 59G. But that's wrong, because /usr/local/src is separate but included. I need to exclude that. It is possible to tell du to not cross filesystems?
Yes, -x.
du -hsx -> 15G

yes, and 3TB is ~\$100, \$200 if mirrored.
(bloody mathjax)
so, about a dollar. For the whole install.

4:29 PM
@FaheemMitha That looks correct, yes.

@StephenKitt Ok. Thanks.

:29492834 Sorry, I was trying to fix up my message — Enter sends the message...

@StephenKitt No problem.
@derobert I'm not sure if I can get those prices here.

@FaheemMitha Right, I guess your install probably costs ₹300 (Google says before)
and my \$1 figure was mirrored. Because who doesn't mirror? :-P

@derobert real users use raid0 and then ask a question on Unix.SE when they lose their data ;-)

4:34 PM
@StephenKitt LOL, but only after trying to fix it themselves with mdadm --create a few times.

@derobert preferably with a patched mdadm...

@derobert I don't follow.

@FaheemMitha Can't be more than 4.5x as expensive, can it?

@derobert That's true, it can't.
Assuming you mean 4 or 5 times.
Even Indians can't jack up more than that.

I used 4½ times, because that gave a nice round number :-P

4:37 PM
Actually most electronic/computer stuff here is within 2x of the US values. Sometimes less.
@derobert What did you use for rupee/dollar conversion?

4:49 PM
@derobert ok
Rs 300 goes further here than in the US. There are plenty of people in this city who live on that daily. Or less.
I wonder if one could use a db to solve unix.stackexchange.com/q/281513/4671

@FaheemMitha Well, they should definitely avoid apt install kde-full, then....

@derobert That's unlikely to happen. Because they don't have computers.
In India, at least, being able to connect to the net requires a certain minimum income, I 'd say.
You also need a place to put your computer.

5:14 PM
unix.stackexchange.com/questions/281554/… ... does my answer here look sane?

@derobert I'm behind the times on PC booting restrictions; grub's able to handle booting a partition that's >9 Gb from the top?

@JeffSchaller yes... for quite a while. Early 2000s, maybe?

@derobert that'd be the timeframe that I stopped booting PC's! Thanks :) Seems like a reasonable answer to me, otherwise!

@derobert Looks reasonable. I generally recommend LVM to people too.

@JeffSchaller Anything with LBA mode could do more than 9GB. LBA32 gets you to 2TB.
LILO did LBA32... I forget when that was added.

5:23 PM
@derobert yeah, looks fine. If you like, you could also suggest a simpler solution: create a new partition and then move /usr and /var to it, replacing them with links in /. Those are usually the biggest dirs in / and that would allow the OP to sort out their space problems without too much pain.

Yeah, I was thinking of doing so but thought that yours was close enough. If you don't, I'll post one in a bit.

I'm guessing all the space is in /usr. At least, that's where it winds up on mine.
Well, except /var/cache and /var/tmp... which of course you clean up if you're out of space.

@derobert Yeah, but since those can grow, I'd move both over.

@terdon Right, but the problem is if its all in /usr ... then OP will just be out of space on /usr.

5:26 PM

(plus, separate-/usr is barely supported if at all in a lot of distros)

@derobert even with manual partitioning? or do you mean that the guided partitioning won't offer a separate /usr?

@derobert Why? Things in /usr needed before non-/ partitions are mounted?

@terdon yep. Often /usr/lib or /usr/bin.
It's far too easy to accidentally have your boot scripts try to use a library or binary out of /usr

Fair enough. Not to mention that many distros these days have /bin -> usr/bin

5:29 PM
Some distros have given up and now demand /usr be mounted from initramfs. My guess is the rest will do that soon, too.

guess the days of read-only or centrally-mounted /usr are long-gone!

Well, might still be here if you mount it from initramfs.

good point

Is anyone here a fan of Ken MacLeod, by the way?

You can wind up needing a lot of stuff for separate /usr. You can, for example, need a full network stack (including up to NetworkManager) and GSSAPI authentication stack to mount an NFS share. That's a lot to make sure it doesn't need to access anything in /usr/lib.

5:34 PM
@terdon had to google him to make sure, but - nope, haven't heard of him

Haven't heard of him, either.

Oh well. He's very good, certainly worth a read. Especially his early, political SF. Anyway, he answered a question on ELU. I love it when that happens!

6:07 PM
Does this make sense to anyone?
0

Is a POSIX context switch well-defined? Is it the same thing as switching threads in C? Can the C compiler generate everything for a context switch or is assembly programming still needed for a routine that switches the threads or switches the "context"? Is there even defined what is meant by "co...

6:17 PM
@terdon It reads like the OP needs a good book on kernels.
Or a course.

Since that's also true of me, I didn't feel up to passing judgement on it. Dunno if it makes no sense because of my ignorance or the OP's.

7:15 PM
"The Rapture of the Nerds" is a funny (and true) way of discribing that Singularity nonsense.