It's aggravating when people build things for Ubuntu and not for Debian - launchpad.net/~sile-typesetter/+archive/ubuntu/sile

Get your priorities in order, people!

@FaheemMitha Considering that Ubuntu is several times more popular than Debian, you could argue that they have gotten their priorities right :)

Wait until you hear about all the things that are only developed for RHEL

@terdon Not so, but far otherwise.

Anything packaged for Debian will percolate to Ubuntu by definition.

It's the natural starting point.

So, on the subject of my ongoing memory travails, I had to reboot my machine again just now.

"available" bottomed out at around 20 or 30. Then the memory killer kicked it, and it jumped to 788, but I still rebooted. Here is what free and top looked like at the time.

gist.github.com/d306fc74dda3c6b7e16bb6c317a51799 and gist.github.com/6598707b86340cb2e6abd3e6e8a01084

If anyone has any idea what is going on, let me know. Thanks.

@FaheemMitha Some things will, not everything. But Debian is in many ways a niche system, it isn't particularly popular on the desktop. So I can understand why someone would make something for Ubuntnu and not bother with Debian.

@terdon No, Ubuntu takes everything wholesale from Debian. They call it universe, I believe.

With a pledge of zero maintenance. If something breaks, tell Debian about it.

Well, other than the subset of packages they actually maintain, of course.

Universe is the repository of, well, everything. They'll have all sorts of things there. But I don't think they take everything from Debian.

I don't track Ubuntu, so things may have changed. But I doubt it.

@terdon As far as I know, they do.

But of course, I could be wrong. Or out of date.

Universe has community-maintained packages. I don't think it ever had a specific connection to Debian.

From the beginning Ubuntu was a Debian derivative. Which means they take a snapshot of unstable and then add their patches on top.

Yes, exactly. But that doesn't mean that packaging something for Debian is a good way of getting it into Ubuntu.

@terdon Other than the packages having been created by Debian?

@terdon Of course it is. Because it's automatically part of Debian. Unless Ubuntu decides to exclude it for some reason.

@FaheemMitha Well exactly. What gives you the idea that Ubuntu take everything?

Ubuntu has diverged very significantly from Debian.

@terdon Because it does, as far as I know. Since 2004.

And I don't see anything that would suggest that there is any link between Universe and Debian. There's even an article on debianadmin.com describing how to add Ubuntu repositories to Debian. That would be completely pointless if the Universe were simply a Debian repo.

Ah, this suggests you're right:

> Ubuntu syncs from Debian every six months (weeks before every releases). Any package that doesn't have any Ubuntu specific changes in Ubuntu (i.e. the previous version was also from Debian) or isn't in Ubuntu already, gets synced into Ubuntu's Universe (free and third-party).

@terdon Ubuntu has other things that are not part of Debian. Like the PPAs.

If so, yes you're quite right and a package made for Debian will make it into Ubuntu. I guess the other issue is that it is hard to get a package into Debian. Which is why they just made a PPA and added it there since they're presumably targeting Ubuntu/Mint anyway.

It's probably close to a superset of Debian if one counts things like PPAs.

@terdon It's not that hard, unless you run afoul of the FTP-masters, who can be a bit uncooperative at times. If they get an idea in their heads it's hard to get it out.

DDs can upload packages with little fuss, usually. So you need to be sponsored by a DD if possible.

Exactly: hard. You can't just go and do it yourself unless you're a DD

It's good to be part of Debian, because then people are more likely to try your package. Including the derivatives.

@terdon Depends whether you consider getting DD sponsorship a high bar.

Caleb is a high rep user here, though I haven't seen him for a while. And it seems he was involved in creating those packages.

@FaheemMitha Certainly higher than making a PPA, yes. You can have a privately hosted PPA and you don't need anyone but yourself, I think. There may be higher bars to pass if you want an official PPA hosted on launchpad.

@FaheemMitha your MEM column adds up to 103.7%, so presumably there's no leak in the kernel or some other shenanigans. Chromium has 59.2% memory, firefox at least 24.9%. That leaves 17% for anything else. Bash has 3%, skype has 3%, plasma and Xorg together have 2%, sql has 1%... the browsers are just easting too much memory.

@AndrasDeak Does it look within normal parameters? And should that happen within just 2 days?

Usually I run out of memory within 2 or maybe 3 days. This time is was 3 days.

That really depends on your browsing habits, and I've never used chromium. On firefox I've seen imgur (presumably due to infinite scrolling plus tab history) eat up a lot of memory very fast. And I saw livefeeds eat up a lot of memory gradually over time a while back, but I haven't seen that in recent times.

I have like 60 tabs open in Chromium. Nothing particularly extravagant. Though I just noticed that the suspend add-on or module or whatever it's called no longer seems to be working, for some reason. Something to investigate.

You should check how much memory is used by a fresh chromium and firefox instance. If that's lower: consider restarting the browsers more often.

@AndrasDeak It's definitely lower, but how to find out the total amount. Just add up all the rows? How about overlap?

I have 422 tabs in firefox right now, but it's pretty good at not loading unused tabs. It's using 22% of my 16 GB RAM after startup (because luckily my Debian froze half an hour ago).

@AndrasDeak What's your uptime?

@FaheemMitha I just summed them up. 'chromium' for chromium, 'firefox*' and 'Web*' for firefox.

@AndrasDeak Oh. And there is no significant overlap?

Why would there be? Are many processes shared by firefox and chromium?

There has to be some overlap, of course, if you look at the 103% total

@AndrasDeak I mean the memory usage across the chromium processes is probably not distinct. There must be some sharing. Obviously it's hard, possibly impossible to determine how much.

@FaheemMitha just shy of 2 hours

@AndrasDeak Oh, you just rebooted? What's your typical uptime? Mine used to be measured in months. Though that was before the era of world-eating browsers.

@FaheemMitha ah, yes. But since the 103% total figure came pretty close to your actual 95% use, it should be a good approximation

@AndrasDeak OK. Fair enough.

@FaheemMitha I mean I explicitly said so... do you read every message in full?

So you see no obvious signs of hardware dysfunction?

I normally have uptimes of a couple of months

@AndrasDeak I try to.

I have to restart firefox every few weeks if I'm running out of memory

@FaheemMitha I'm not one to answer that, I don't even know what the %MEM column exactly refers to, and what the various columns mean.

but "browsers eating all memory" seems like a fairly mundane explanation

@AndrasDeak OK. Would anyone else care to comment?

@AndrasDeak This didn't use to happen.

Well, did you update chromium and/or firefox?

Unless it's the streaming, which is relatively recent.

@AndrasDeak Update? It's whatever Debian stable ships with.

It changes all the time. With every release, at least. But usually Debian updates browsers much more frequently than that, since I think they've mostly given up on patching it.

Maybe that volatile thing.

It's possible that Hotstar is very leaky. Possibly also Amazon Prime. Though, since I currently run them both on Firefox, because they don't currently work on Chromium, I don't see how they would account for Chromium's memory usage.

Would anyone else like to comment?

@FaheemMitha so has "whatever Debian stable ships with" changed before the memory issues started?

@terdon In the case of SILE, which is what I was talking about, it would make sense for them to try to get it into Debian, because that would immediately increase the number of people who would try it. Assuming the SILE community is sufficiently large, it should have enough contacts to get a DD interested.

@AndrasDeak I don't remember how long these issues have been going on, sorry. And anyway, Debian regularly changes the browsers, even in stable, as I already mentioned.

@FaheemMitha Yeah... I think of Debian as something very few people use so I don't see why but OK.

@FaheemMitha my assumption was that the versions regularly change. Otherwise I wouldn't have asked if the version changed.

SILE looks quite interesting, though I've never used it. There's an open PR which looks like it is being actively worked on, which adds math support. Obviously experimental.

But is has the usual problem of lack of resources. Plus getting traction with TeX in the picture is going to be tough.

I'm asking "has the oil been changed since the car's been making this bad smell?"". You're saying "motor oil is changed regularly".

well, yes, that's why I'm asking

@terdon It depends on your definition of very few. But I wouldn't describe it as like that, personally.

Because, you know, there's either a hardware reason, a software reason, or no particular reason.

@AndrasDeak I have no idea how often the browser versions have changed. But I do know that the problem has been there consistently for quite a while. A couple of years, possibly.

Your memory has been running out every other day, for a couple of years?

I could get a sense of how often the browser has changed by looking at the apt and dpkg logs, I suppose.

@AndrasDeak When you say it like that, it does sound insane. But yes, possibly. I'm poor at tracking things, especially when there are no signposts.

I see

It might be a year. I would have said at least that. Is there some easy way of tracking reboots?

After about six months I may have considered doing a memtest to rule out hardware issues

That would be informative.

@AndrasDeak I tried that the other day. My temperature alarms promptly started going off. Probably the processor.

but if free sees as much memory as it should, and top reports all that memory being full, then this doesn't sound like a hardware issue

@FaheemMitha are you sure it was a memtest rather than a bitcoin mining script?

@AndrasDeak It was mem86 at the console.

I tried running it from the GRUB menu, but all the memtest links were broken. I don't know why, and I don't have the energy to investigate.

Actually, last shows logins going back to March. Is there some way to get it going back longer?

Is it March 2021?

Perhaps not, because it sources something called /var/log/wtmp, it seems. Perhaps I can adjust something to make the system keep those records longer.

Mine goes back to December 15, but it's 2019

@AndrasDeak wtmp begins Sun Mar 21 00:02:37 2021

OK

That's what last says at the bottom.

yes, that's what I asked

@AndrasDeak Huh, that's odd. Did you change the defaults?

No, but it's not a lot of entries

I guess one would need to adjust the rotation.

138 lines

reboot   system boot  5.10.0-1-amd64   Sat Apr  3 22:34 - 17:54 (91+19:20)

I have:

that's the record

# no packages own wtmp -- we'll rotate it here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    minsize 1M
    rotate 1
}

That's the contents of /etc/logrotate.d/wtmp.

I've got the same contents

How many lines do you have in wtmp?

@AndrasDeak Maybe you just reboot less.

@FaheemMitha yes, that was my point

last | wc -l
521

I see

I think it may have got worse recently. For example, it looks (from last), like the machine was up from March 21st to Apr 14th.

Exactly 3 weeks. But my usage certainly hasn't changed in that time.

Since May it looks like it's being either rebooted or crashing every few days.

Ah, a helpful answer here.

wtmp.1 begins Sat Apr  6 18:03:38 2019

shows the same pattern going back at least that far.

Rarely more than a week between reboots. At max, 2 weeks.