« first day (2231 days earlier)      last day (2795 days later) » 

2:00 PM
pretty sure bb guns and such have been banned as long as anyone can remember....
 
@CausingUnderflowsEverywhere You love anything with two X chromossomes. ;p
 
oh and paintball
 
Dog
@ThatBrazilianHeadlessHorse Ewww
 
...unless I am mistaking users, in which case I apologize
 
females have two X chromosomes?
even if thats the case you are sadly mistaken
I love everyone and everything except for those scary things
I am about to consuming used dip!
yumm
 
2:10 PM
Someone said females and dip.. Had to post it.
d:-)
 
@JourneymanGeek Paintball hasn't been banned here.
See @dog? A reason to move to Brazil!
 
Bob
ATTACKATDAWN
 
ooo you live in Brazil?
 
2:14 PM
@CausingUnderflowsEverywhere What? Me? No, where did you get that crazy notion? I live at Leastasia. Always have.
 
can I live with you in leastasia
..but your ip address says otherwise
 
@CausingUnderflowsEverywhere It depends, are you named Winston?
 
YES
IM THE FOSSIL ARCHEOLOGIST
@ThatBrazilianHeadlessHorse , here is a picture of me on a good day.
 
Bob
@CausingUnderflowsEverywhere ...isn't that from maplestory
 
@Bob safety dog does not approve
 
Dog
2:35 PM
@ThatBrazilianHeadlessHorse here neither
@Bob wat I don't get it
 
Bob
@Dog Big rock held by thin stick. Stick break. Rock fall on head. Ouchies.
 
@CausingUnderflowsEverywhere oooᗤ
@Bob It's ok, they're wearing a protective hat.
@CausingUnderflowsEverywhere Why is pac-man innapropriate?
 
Gotta love that Number 13, 0x000000000000000000D
 
ᗣᗣᗣᗣᗣᗣᗣᗣᗣᗣ ᗤ
Funnily enough, both on my Android and on my Fedora PC, it shows as:
then who was unicode
¯\_(ツ)_/¯
 
2:45 PM
is a fedora PC a hat that is a pc, or a pc that is a hat?
 
Fedora was a fork of redhat
 
@JourneymanGeek Hatt-baby, hatt-baby!
 
Bob
2:50 PM
So, who recognises this one? :D
 
@Bob pile of hatz
 
@OliverSalzburg: Yeah i figured the smaller version would be easier to find fedora on it :)
 
SVG not working?
Ah :D
 
Bob
@OliverSalzburg tha's a 4 MB PNG o.O
and shitty quality too
SVG FTW
 
2:57 PM
@Bob Yeah, but it said "image not found" when I posted it here :(
The SVG also has links, but Chrome lacks proper search-in-SVG capabilities
 
Bob
@OliverSalzburg eh... un-onebox it?
not like you can read it at the oneboxed size anyway
 
You underestimate my user styles and desktop resolution ;P
 
Bob
@OliverSalzburg unless you're running 8k or something...
 
@Bob I can mouse-wheel while I hover over the image ;) The latter zooms to 100%
 
Bob
Hm... 50 GB miiiiiiiight have been overkill
suggestions to use 32 GB in 8 days (or 4 GB a day)? :P
 
3:18 PM
@Bob Is that transfer volume?
 
@Bob, download all the torrents
 
download a file using wget at 800megabits per second
COOL IM USING A DIRECT FORK OF GNU linux. hey MingW stands for minimalist general nuesance userface for windows
 
Dog
@Bob Speedtests
 
Bob
@OliverSalzburg ya
 
Dog
At 200Mbps, you can eat 32GB in less than an hour
 
Bob
3:26 PM
@Burgi nothing interesting... I have enough Linux ISOs :P
@Dog at 10-30Mbps...? :P
 
Dog
@Bob Ten hours? :P
 
Bob
Lol
 
Dog
Load the cell up to 100% capacity to set off the utilisation alarms at the provider to nudge them into upgrading the cellsite
 
Bob
That's assuming they care!
And don't just kick me off...
 
Dog
Hey you paid for the data, they can't kick you off for using it
 
Dog
3:51 PM
 
@bob, mr robot is good to watch
 
@Bob Download your entire steam library a single steam game? ;P
 
Bob
That would probably do it!
 
4:08 PM
hmmmmm
i can likely come hjere at work
not sure if bad idea :@/
 
@djsmiley2k You can likely what? (ಠ_ಠ)
 
come into the chat
 
You could have picked some less ambiguous phrasing, you know...
 
hehe
 
Sigh. Could they have made this damn email migration any more complex?
The Exchange server was at mail.team.workplace.tld. It had IMAP, and a webmail at the same address.
It worked, mostly fine, a few outages sometime but nothing too serious.
Nearly two weeks ago, the migration started. They sent an email with new info.
The email was sent to the mailboxes being deactivated.
That's not the first time something like that happens.
The text about the migration was confusing as hell, had to read it five times to get it
Turns out there are now two new different servers
ununtuitivename1.team.workplace.tld holds all the old messages, but won't send or receive anything.
ununtuitivename2.team.workplace.tld sends and receives, but usernames and passwords were changed.
Worry not, because the mail explaining everything was sent... to the deactivated server!
 
5:01 PM
@Bob no0pe
@ThatBrazilianHeadlessHorse haha sounds like one of my old bosses
 
> Based on our investigation, we learned that there was an issue with the battery cell. An overheating of the battery cell occurred when the anode-to-cathode came into contact which is a very rare manufacturing process error. We are working with multiple suppliers to ensure that a rigorous inspection process is conducted to ensure the quality of our replacement units and we do not anticipate any further battery issues.
> anode-to-cathode came into contact
In other words, a manufacturing error is producing cells that internally short themselves. Ouch.
 
@ThatBrazilianHeadlessHorse , I work in IT too!!
 
u heard they limiting chyarging to 60%
also not rare enough
@CausingUnderflowsEverywhere me 33333
 
5:19 PM
Wow, so many people here work in IT, what a coincidence! Who'd expect that!
 
:OP
not causation :P
 
I'm my own IT. One side of me breaks stuff, other side sighs and fixes it.
2
 
5:40 PM
hi
 
hi
 
I'd add another thing to the list of advice in the conclusion: Avoid visiting sites that promote illegal or amoral behavior, such as media and software piracy; content that is outlawed in most parts of the world; etc. These sites often contract with the least reputable advertising vendors, who make no real effort to filter the content of their "ads" at all, making it easy for criminals to inject your webpage with content that delivers malware or attempts to exploit your browser to gain access to your system. Sometimes even a good adblocker will miss this stuff. — allquicatic 58 secs ago
@BenN still an awesome answer though!
 
Dog
5:58 PM
!!ultraheaddesk
 
Dog
!!multiheaddesk
 
@Dog wouldn't that be heeeeeeeeeeaddesk?
 
LOLOLOL my cat just fell off the sside
 
6:17 PM
 
Is anyone else noticing weird things with MySQL powered sites since the last update?
 
nope
php/mysql ? or
 
ur going to have to define weird
 
yeah
 
Keeps crashing... Apparently there's ANOTHER new update on the Debian repos... Maybe that will fix it
EVERY EFFING SITE that I upgraded MySQL on yesterday was down this morning. Work and personal
 
6:25 PM
hmm wonder what var logs say
 
is it MySQL MySQL, or MariaDB?
 
/var/log/mysql/error.log shows no useful information. It's MySQL
 
what version of mysql?
5.6.33 ?
 
oh, because I haven't used MySQL since MariaDB was a thing
 
160914 14:24:08 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead.
160914 14:24:08 [Note] Plugin 'FEDERATED' is disabled.
160914 14:24:08 InnoDB: The InnoDB memory heap is disabled
160914 14:24:08 InnoDB: Mutexes and rw_locks use GCC atomic builtins
160914 14:24:08 InnoDB: Compressed tables use zlib 1.2.8
160914 14:24:08 InnoDB: Using Linux native AIO
160914 14:24:08 InnoDB: Initializing buffer pool, size = 128.0M
mysql Ver 14.14 Distrib 5.5.52, for debian-linux-gnu (i686) using readline 6.3
 
6:28 PM
heh, 32-bit, no wonder it crashes... probably runs out of virtual address space :P
 
dmesg?
 
could be indicative of a memleak
yeah, see if dmesg has an OOM kill on it
 
also debian
so its like 3 yrs ouuta date already?
;)
 
GOD DAMMIT
[87778.421247] Out of memory: Kill process 10469 (mysqld) score 88 or sacrifice child
[87778.422787] Killed process 10469 (mysqld) total-vm:343520kB, anon-rss:44848kB, file-rss:0kB
 
yeah, 5.5 isn't even the latest release series
 
6:29 PM
and there it is, ur welcome
 
@CanadianLuke ok, well at least we know the reason... the Debian update (which is heavily patched, as you can see from the .diff.gz file on packages.debian.org) or the upstream MySQL update introduced a memory leak
 
if it was mass effecting would expect to hear something
 
are you using jessie?
also can you find the exact package version for me
aptitude show mysql-server-5.5
is it 5.5.50-0+deb8u1?
 
Dog
@allquicatic Probably, but I created that a long time ago in my old workplace
 
@CanadianLuke see how big the VSS is in top about a minute after starting up mysqld; if it's already a couple gigs, it's not hard to imagine a patch that merely increased memory usage (even if it didn't actively introduce a leak) could push it over the top after a while of running
remember, your VAS is either a 2G/2G split (user/kernel) or 3G/1G
 
6:33 PM
It's on a $5 Digital Ocean server. 512MB of RAM
 
oh; but what about the ones at work? do they have more RAM?
 
Yea, 8GB at work. They both show 5.5.52-0+deb8u1
 
ok... and do the ones at work also show oom killer in dmesg?
(and are they 32-bit?)
also, I'm really confused by that package version number
either you're using a third-party apt repo, or a distro version I'm not expecting, because Debian Stable currently has 5.5.**50**-0+deb8u1
and unstable (Sid) has a much newer version
 
The main one at work did not have Aptitude installed. I'm waiting for that to install...
But they are all on Jessie, not sid
 
ok, maybe it's just some weird packaging thing, I'm really not sure
 
6:39 PM
5.5.50-0+deb8u1
That's the main work one, on Jessie 8.1 x64
5.5.52-0+deb8u1 <-- Jessie 8.1 x86
 
that's... weird
we're not even dealing with the same upstream release of mysql then, and it's still causing the same behavior.... I think, anyway
on your 64-bit box with 8 gigs of RAM, are you getting the OOM killer in dmesg?
 
Argh... And on the work server, dmesg does not show ANY memory issues
 
ahh
 
No, not on the 64 box
 
dmesg | grep -i segfault and see if it crashed? also check logs on the 64-bit box
I think there might be two separate issues; with the 32-bit Digital Ocean box, you just need to upgrade to a more expensive VM with more RAM, there's probably no two ways about that.
with the 64-bit boxes, I don't yet know.
 
6:41 PM
No to both
Checking logs on 64
Only log entry for MySQL:

160914 11:12:10 [Note] /usr/sbin/mysqld: Normal shutdown

160914 11:12:10 [Note] Event Scheduler: Purging the queue. 0 events
160914 11:12:10 InnoDB: Starting shutdown...
160914 11:12:12 InnoDB: Shutdown completed; log sequence number 4031247996
160914 11:12:12 [Note] /usr/sbin/mysqld: Shutdown complete

160914 11:12:13 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
160914 11:12:22 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
 
weird... are there any backup or older logs, perhaps gzipped, in the same directory?
 
64 bit seems to be staying up now...
There are, but they're 4.0k each zip... So I'm guessing not
 
you could run zless +G something.gz and check
if it's empty, well... yea
 
Blank
Checking 32
 
did you use apt-get or similar to perform a package update to a newer version of MySQL at the same time on all of these systems, or relatively close in time?
 
6:48 PM
Yesterday afternoon
I did another one this morning, and it seems to be stable now...
Weird... Maybe the update this morning fixed it
 
oh -- there was another update this morning, then?
 
they updated the upstream version of the package they're pulling down, to 5.5.52, just today
it could be that your problem was related to the old upstream version, 5.5.50
make sure that all of your boxen end up on 5.5.52 ASAP -- if you're using a mirror, it may take like 24h for some boxes to see the update
 
Weird... But very possible
 
the thing to keep in mind is that an upstream version will include more than purely harmless "security" content
it can contain bug fixes, regressions, and very small features in a minor release
so the fact they upgraded from upstream mysql tells me that you will be better off running 5.5.52 on all your boxes, and the one(s) that are still on 5.5.50 need to be updated
 
6:59 PM
Yet, the 5.5.2 one crashed again
 
just now?
 
on 64-bit?
run journalctl -xe
 
@CanadianLuke ah, yeah, that one is going to continue crashing until you either reduce the amount of data you're dealing with in those tables (or otherwise optimize MySQL's memory usage), or upgrade that DigitalOcean VPS with more RAM
 
7:01 PM
Sep 14 14:59:39 server.bcmainland.ca kernel: Out of memory: Kill process 15698 (mysqld) score 78 or sacrifice child
Sep 14 14:59:39 server.bcmainland.ca kernel: Killed process 15698 (mysqld) total-vm:343428kB, anon-rss:40060kB, file-rss:0kB
Sep 14 14:59:48 server.bcmainland.ca sshd[16038]: Connection closed by 188.212.104.164 [preauth]
Sep 14 15:00:37 server.bcmainland.ca mysql[16057]: Stopping MySQL database server: mysqld.
Sep 14 15:00:42 server.bcmainland.ca mysql[16102]: Starting MySQL database server: mysqld ..
 
I don't see anything indicative of a software problem with that little RAM
generally anything you do to reduce MySQL's memory usage, while it may improve stability, will hurt performance... so YMMV
 
You're just running out of RAM.
 
take it from that Canuck up there (.ca domain) and try to tune your MySQL a bit :)
 
That's what I'm trying
6.5MB Database size?!?
 
@CanadianLuke the size of your data on disk isn't necessarily very indicative of how much memory MySQL will use. If there are a lot of interrelations between data; indexes; and complex queries on that data, MySQL will create and then hold onto a lot of cached data, like maps between primary keys and rows; intermediate query results; etc. -- all in the name of performance, assuming a system with a reasonable amount of RAM instead of 512 MB
otherwise the CPU usage of MySQL for even a light workload would be insane
 
7:09 PM
Understood... Hopefully they don't balk at the price of upgrading...
 
also IIRC the data on disk is compressed
 
Awesome? Lol
 
the SE infrastructure guys told the community at one point that the entire StackOverflow Q&A database (excluding i.stack.imgur.com images stored on imgur's infrastructure) was something like a 50 GiB set of files in MS SQL Server's compressed on-disk format
databases store a lot of data efficiently and then use hideous amounts of RAM to maintain lots of different representations of that data in memory that they can easily process into low-latency query responses.
and that 50 GiB figure is fairly recent IIRC... 2015
 
Wow
Caching is working better... The site is loading faster
 
with the memory tunings?
 
7:17 PM
No memory tunings yet... Using some Wordpress Plug Ins. One for caching, one for "optimizing" the database. Nothing about memory yet, but it's not crashing right now
Up for over 20 minutes this time! Maybe that's all I needed
 
for now...
 
True...
 
Dog
...
 
there's still going to be a lot of memory pressure on a 512 MB system running wordpress and MySQL
 
You guys want to try to visit the site, and see if MySQL crashes again? I'm sure it should handle 5 people comforably...
 
7:23 PM
that's an underspecced amount of RAM, quite honestly
 
I know :( I may try lighttpd
Don't try to DDoS it, but browse it like a normal user, please
 
browsed
 
Dog
browsed
 
browsed
 
7:30 PM
Sweet, thanks guys. It seems to be holding up right now. I don't expect more than 10 would be on a time, at least not yet.
 
@CanadianLuke css errors: jigsaw.w3.org/css-validator/… :)
 
*posts the site to 4chan* *watches it crash*
j/k
 
@DavidPostill Not a clue how to edit the .CSS files directly... I don't want to learn themes, let alone tough them. Thanks though.
But the caching is definitely taking effect. It loads much quicker than it did even just last week
 
8:01 PM
@CanadianLuke "Error establishing a database connection"
 
DAMNIT. Thanks
 
.... as said, try the memory tuning config params first, and if that doesn't make it stable in the long haul, you are going to have to at least double the RAM
didn't think HTTP side caching would take you very far once mysql started caching a couple queries
 
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
	<title>Database Error</title>

</head>
<body>
	<h1>Error establishing a database connection</h1>
</body>
</html>
 
Version: '5.5.52-0+deb8u1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Debian)
Killed
160914 15:59:32 mysqld_safe Number of processes running now: 0
160914 15:59:32 mysqld_safe mysqld restarted
160914 15:59:32 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
160914 15:59:32 [Note] /usr/sbin/mysqld (mysqld 5.5.52-0+deb8u1) starting as process 18940 ...
160914 15:59:32 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and
 
> Killed
Something got terminated.
dmesg output?
 
8:05 PM
@bwDraco usually that's the OOM killer
we've already been over this...
read up please
 
I know, I know.
 
the more relevant error is at the bottom of the log, actually
> Unknown/unsupported storage engine: InnoDB
something is seriously broken there
 
> InnoDB: mmap(135987200 bytes) failed; errno 12
 
OH... I see what the problem is :D
 
You're out of memory again.
 
8:07 PM
it's trying to allocate 128 MB of RAM, then InnoDB's shared object (the InnoDB database engine) crashes with malloc returning FALSE
 
:(
Can I somehow force it to only use 128M of memory?
 
then it assumes that since InnoDB's initialization function returned FALSE that InnoDB "doesn't exist"
 
Even if it affects performance?
 
Can you drop it to 64MB?
 
8:08 PM
@CanadianLuke for like the tenth time, the tuning parameters in my.cnf from that guy's blog I linked you to WILL dramatically reduce MySQL's memory usage; if you've already applied them, let me know...
he even tells you what the minimum values are for each
if you set them all to their minimums and it still gives you that, that's different
IIRC the minimum should use way less than 128MB even
the problem is that it can't allocate a contiguous (linear in virtual address space) block of 128 MB of memory for InnoDB...
you probably have enough memory free, but it's fragmented so that it can't give it all to you in one chunk
> Setting Default Minimum
innodb_buffer_pool_size 128M 5M
so the default is 128 MB, which means that InnoDB will attempt to mmap() (allocate a linear amount of virtual address space in one big chunk) of 128 MB, but you can drop it all the way down to 5 MB if you want
that 128 MB is probably exactly the setting causing that OOM crash.
# /etc/my.cnf:
innodb_buffer_pool_size=5M
innodb_log_buffer_size=256K
query_cache_size=0
max_connections=10
key_buffer_size=8
thread_cache_size=0
host_cache_size=0
innodb_ft_cache_size=1600000
innodb_ft_total_cache_size=32000000

# per thread or per operation settings
thread_stack=131072
sort_buffer_size=32K
read_buffer_size=8200
read_rnd_buffer_size=8200
max_heap_table_size=16K
tmp_table_size=1K
bulk_insert_buffer_size=0
join_buffer_size=128
net_buffer_length=1K
innodb_sort_buffer_size=64K

#settings that relate to the binary log (if enabled)
 
I'd try 64MB or even 32MB. Database performance may suffer but at least it'll run.
What's the output of free?
 
if you edit your my.cnf.... remove any existing lines that contain any of those configuration variable names, and then insert the lines in that file I just posted (from the blog)
(you don't want the same variable to be given twice)
each of those settings should slightly reduce memory usage in its own way, but the biggie is innodb_buffer_pool_size I think
@bwDraco the site is mostly static, so performance should be fine with caching at the web frontend... all he really needs is for MySQL to keep itself from taking up too much memory; in the rare occasion that MySQL actually has to do any work, it'll be slower for the first person who accesses that page, then load instantly after that based on the HTTP server's caching.
I'd set it to the minimum, 5 MB, myself... no need to take any chances of future OOMs
 
I'm leaning towards 32MB.
 
free
total used free shared buffers cached
Mem: 512096 184404 327692 42244 10108 100732
-/+ buffers/cache: 73564 438532
Swap: 0 0 0
Trying those optimizations, MySQL won't startr
host_cache_size is not understood
[ERROR] /usr/sbin/mysqld: unknown variable 'host_cache_size=0'
 
@CanadianLuke ok, comment out that one
 
8:20 PM
Removed the line, trying it again
/usr/sbin/mysqld: unknown variable 'innodb_ft_cache_size=1600000'
 
keep doing it until it stops complaining, lol... commenting, that is
 
Hey folks, got a malware sample on my machine. It's in a ZIP attachment.
 
that tuning file was designed for MySQL 5.6, not 5.5
most of the params should work but some won't
 
What leaves me concerned is that the file wasn't caught by either Norton or Malwarebytes.
 
OK, it's up
 
8:23 PM
@CanadianLuke browsed
 
1:23 pm... Now to do more work
 
browsing
 
you can feel that little VPS churning, but it's not half bad
even a completely uncached page loaded in under 8 seconds
most users wouldn't give up on it
 
...uploaded to Symantec Security Response.
 
8:38 PM
@bwDraco where / how did you encounter it?
 
Spam email, purporting to be a girl wanting a date. Filename suggests an image but the ZIP file actually contains JS code.
 
@bwDraco interesting; did it get sent to Gmail? if so, did it show up in junk or in your inbox?
 
I use Yahoo Mail and fetch mail through Thunderbird.
 
ahh...
I'm not sure how good Yahoo is with their spam filters, but if they have a way for you to mark emails as spam (for their info -- similar to how Gmail does it), you should definitely do that
 
@allquicatic Damn package managers! Always messing with the world's infrastructurse!
 
8:42 PM
Yahoo itself probably caught it. Thunderbird does its own filtering.
I don't have the tools to safely test the code, which is obfuscated.
Because Windows Defender recognizes it, I've temporarily enabled it and will use it to scan the machine.
 
Is it intended for HWMonitor to not show my CPU fan RPM?
 
That's for the actual Candi.js file inside the zip.
 
@bwDraco I'm confused... I thought if you used IMAP, when the email comes in, Yahoo would route it to a separate Spam or Junk folder before Thunderbird sees it.
that's how Gmail's IMAP works anyway
 
ohhhhh :P
 
8:54 PM
because I've been getting messages on boot about CPU fan error, but with CPU fan speed monitoring disabled in my BIOS because it kept giving false positives
 
@Nzall try unplugging your CPU fan and plugging it back in (securely) to the socket where it's installed
in fact, you might want to not only remove and re-insert the plug, but re-seat the fan
 
This threat can download other malware and unwanted software onto your PC. We have seen it download the following threats:

PWS:Win32/Fareit
Ransom:Win32/Crowti.A
That's pretty scary. It's a ransomware downloader.
 
@bwDraco I'd really be careful with doing anything further with that, dude, especially on a PC with your personal logins or data....
 
I'll try that tomorrow, it's getting late right now
 
cryptomalware is no laughing matter
 
8:56 PM
The files have been deleted and were never executed.
 
In fact, I intend to delete the messages containing the attachments in question.
...gone.
VirusTotal shows:
> Microsoft: TrojanDownloader:JS/Nemucod.HC
Because Windows Defender recognizes it, I'm using it to scan the machine. It's not always enabled due to potential conflicts with Norton Security.
("Limited Periodic Scanning" enabled)
5 mins ago, by bwDraco
https://www.microsoft.com/security/portal/threat/encyclopedia/entry.aspx?Name=Tr‌​ojanDownloader:JS/Nemucod
 

« first day (2231 days earlier)      last day (2795 days later) »