Stack Exchange
log in

mas

 /dev/chat

General discussion for unix.stackexchange.com. If you have a q...
2
mas
Aug 4, 2021 16:11
Sorry for the late reply
mas
Aug 4, 2021 16:11
@Kusalananda That makes a hell of a lot of sense.
mas
Aug 2, 2021 20:27
Yeah, I think I also think about it backwards, and I always have to look up to make sure I have it the right direction
mas
Aug 2, 2021 20:24
So I guess I'll just have to keep looking up what order to but the target/link every time I use ln for the rest of my life :-(
mas
Aug 2, 2021 20:19
I guess then you can't overwrite a dead link
mas
Aug 2, 2021 20:19
Is it insane to wonder why ln -sf isn't able to just determine the target and the link from context? Like, if the file doesn't exist for one of them, that's the link.
mas
Aug 2, 2021 19:43
It's on a vps, it seems like psql postgres://postgres@localhost does not work
mas
Aug 2, 2021 19:41
If one doesn't set a password for the postgres user in a new installation and opens the postgres server to external connections, no one can access that without sshing into the machine, right?
mas
Aug 2, 2021 19:24
I suppose it's rude to treat a community chat like an SMS thread with correction*
mas
Aug 2, 2021 19:22
No, 170mb over 66 days is 2.5mb/day which is about a gig a year.
mas
Aug 2, 2021 19:21
Or else I did my math really really wrong
mas
Aug 2, 2021 19:19
Sometimes storage space just boggles my mind. In 3 months I completely filled out all my inodes but it would take me 25 years to fill up a measly 25gigabytes
mas
Aug 2, 2021 19:18
I've been wanting to set up a VPS for this thing but I was initially worried about the storage, I've built a 170mb database in just 3 months, but it turns out that would still take like 25 years to fill up a 25gb digital ocean droplet.
mas
Aug 2, 2021 19:13
I'm not doing deployments, just scheduling jobs, apparently that creates eggs. So weird.
mas
Aug 2, 2021 19:12
lol, I don't think I'm doing anything outside of the docs!
mas
Aug 2, 2021 16:16
In the mean time I wrote a cronjob to just literally delete all the files in /root/.cache/Python-Eggs/ every hour.
mas
Aug 2, 2021 16:15
Nope, not a bug, a feature: https://github.com/scrapy/scrapyd/pull/394

I have no idea why this even makes sense. In 3 months you store 5million files to run a spider? Nuts.
mas
Aug 2, 2021 15:58
It has to be a bug in scrapyd.
mas
Aug 2, 2021 15:57
Every time crontab curls scrapyd's schedule.json route it creates a new copy of the entire project
mas
Aug 2, 2021 15:54
@AndrasDeak Not an hour later and there are 40 new files in egg
mas
Aug 2, 2021 15:00
But there were so many files I couldn't just rm -rf *. Damn thing said too many arguments!
mas
Aug 2, 2021 15:00
That sounds really unlikely since a deployment takes like 10 seconds at the extreme end.
mas
Aug 2, 2021 15:00
Maybe I can try a deploy and see if a buttload of files are created on every deploy too (maybe like 3 deploys results in millions of files?)
mas
Aug 2, 2021 14:59
I'm going to keep checking that directory to see when new files are created, that might give me some clue.
mas
Aug 2, 2021 14:57
But I have no idea why it would be packaging eggs except when I explicitly deploy.
mas
Aug 2, 2021 14:56
I nearly killed a raspberry pi with that.
mas
Aug 2, 2021 14:56
It must, I haven't touched it in like 3 months, but I remember finding a huge memory leak because I kept opening new instances of Chromium webdriver without closing them.
mas
Aug 2, 2021 14:55
scrapyd-deploy doesn't seem maintained anymore, I ended up forking it to fix a bug that's been sitting in the PRs for like 2 years.
mas
Aug 2, 2021 14:55
I'm pretty sure scrapyd-deploy packages everything as eggs and since I'm running the scraper as a service I think it's defaulting to root.
mas
Aug 2, 2021 14:54
@terdon I actually bought a duck and I never use it. I like to open a stack overflow question and because I know how much people hate malformed questions I usually end up answering my own just in better formulating the question.
mas
Aug 2, 2021 14:52
@AndrasDeak do you know why I'd be getting so many files in Python-Eggs? Bug with scrapyd or something?
mas
Aug 2, 2021 14:50
Thanks, guys, for letting me use the room as a rubber duck. :-)
mas
Aug 2, 2021 14:42
Alright, I think I found the problem, five million inodes in /root/.cache/Python-Eggs. What. The. Heck.
mas
Aug 2, 2021 14:12
Man, that's still just a drop in the bucket.
mas
Aug 2, 2021 14:11
Okay, by
```
sudo du --inodes | sort -k1,2 -n
```
in various directories I found that docker has 101k inodes. I uinstalled docker and dependencies (I'm on Arch) and it hasn't removed the files for docker in `/var/lib`. Is it safe to just delete them?
mas
Aug 2, 2021 13:58
I mean, there's over 6.2 million inodes used on this system, I don't get how the biggest inode directory is only 17k (man3).
mas
Aug 2, 2021 13:57
It uses chromium headless to scrape some sites, maybe chromium's cache is blowing up somehow?
mas
Aug 2, 2021 13:56
Maybe there's some service that's creating tons of files in separate directories.
mas
Aug 2, 2021 13:55
I'm convinced that something about my scrapyd software is eating all my inodes but I have no idea what and all these one liners to find the biggest inode dirs seem to not be giving me the results.
mas
Aug 2, 2021 13:53
Hey all, I'm running a little home server with a news web scraping service/web interface (scrapyd/flask) and I'm running out of inodes and trying to find where they're all going, I ran
```
du --one-file-system --inodes --separate-dirs | sort -rh | sed -n \
'1,50{/^.\{71\}/s/^\(.\{30\}\).*\(.\{37\}\)$/\1...\2/;p}
```
as suggested at https://unix.stackexchange.com/a/123052/288980 and there's 17,467 inodes in man3. Is that normal?
malan
Feb 1, 2019 15:36
Having a shell set to /sbin/nologin shouldn't preclude sftp login, right?
 

 Mathematics

Associated with Math.SE; for both general discussion & math qu...
5
mas
Jul 10, 2021 20:14
Well that didn't work
mas
Jul 10, 2021 20:13
Hi, I'm trying to prove

$\frac{-1+\sqrt{3}i}{2}$

is a cube root of 1 and I'm kinda stuck here:

$\frac{-1-3\sqrt{-3}}{8}$

Any suggestions?