« first day (1761 days earlier)      last day (2063 days later) » 
00:00 - 12:0012:00 - 00:00

12:00 PM
@iBug metasmoke is now considered down.
 
@SmokeDetector cc @Glorfindel
 
Warning: 1000 latest connections to metasmoke have failed. Disabling metasmoke
 
Is its threshold actually 1,000 failed connections?
Surely it should conclude MS is down after like, ten, and check periodically if it's up again.
 
!!/ms-down sets it to 999
The actual threshold is, like you said exactly, 10 (consecutive).
 
gotcha
 
12:04 PM
@AJ gone now (submitted the final flag)
 
[ SmokeDetector | MS ] Bad keyword in answer, bad keyword in username, bad keyword with email in answer, email in answer, pattern-matching email in answer, +1 more: Do I intuitively know if someone already used their reaction? by Apollo Hospital on rpg.SE
 
^ the organ donation spammer again
 
A J
$300k for a kidney? O.o
 
wouldn't it be neat if Smokey kept the posts on file when Metasmoke is down so that the flagged posts can be populated when Metasmoke comes back up?
5
... come to think of it, a circular queue of the last 10,000 posts from the API would be useful to have around occasionally even when MS is up
 
A J
sounds like a good idea
 
12:10 PM
@tripleee post_cache/?
Named after time so ls gives sorted results.
 
I was thinking just keep it in memory and maybe save a pickle when you quit
 
@tripleee yes :)
 
I'm a bit more interested in file-based storage of everything
 
@AJ yeah but first you pay a deposit of $500 to go on the waiting list for donation, then $1,000 for a processing fee, then $2,000 to handle the paperwork, don't worry, you'll get it paid back on top of the $300k later...
 
Either 20180230_123456.dat or 1588888888.dat
 
12:13 PM
[ SmokeDetector | MS ] Bad keyword in link text in answer: Cutting my kid loose from my PS account? by PlayStation Support on gaming.SE
 
!!/watch gamesupportonline\.com
 
@iBug MS is not reachable, so I can't see if you have code privileges, but I've created PR#2411 for you.
 
Next FR: cache code priv list in a pickle
 
@iBug nice! #MeLike
 
[ SmokeDetector | MS ] Mostly dots in answer: intertemporal utility maximisation by Adama on economics.SE
 
12:21 PM
@SmokeDetector n
 
[ SmokeDetector | MS ] Blacklisted website in body: Top 5 most used PHP framework in 2018 by Sam Web Studio on superuser.com
 
A J
@SmokeDetector k
 
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, body starts with title and ends in url, link at end of body, +2 more: L'Averla Paris Anti Aging Cream by Kuhnddysfe on askubuntu.com
 
@SmokeDetector k
 
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, potentially bad keyword in body, potentially bad keyword in title: cash app customer service number 1877 57 1 5111 cash app contact number by nancymartin on apple.SE
 
12:36 PM
@SmokeDetector k
 
[ SmokeDetector | MS ] Bad keyword in answer, bad keyword with email in answer, email in answer, pattern-matching email in answer: I have $3500 in Rollover IRA. Should I withdraw it early and pay off my credit card debt? by Robert Klench on money.SE
 
@SmokeDetector k
 
!!/watch [a-z_]*(?:1_)?365[\W_]*300[\W_]*3916[a-z_]
 
@tripleee MS is not reachable, so I can't see if you have code privileges, but I've created PR#2412 for you.
 
@SmokeDetector why
 
@tripleee Body - Offensive keywords: Positions 546-553, 586-593: assholes
 
@SmokeDetector naa
 
@tripleee Recorded answer as an NAA in metasmoke.
 
@SmokeDetector are you sure?
 
1:18 PM
pretty sure you didn't
 
[ SmokeDetector | MS ] Blacklisted website in body, body starts with title and ends in url, link at end of body, pattern-matching product name in body, pattern-matching website in body, +1 more: rapid slim connected prevalent outsider collaborator by Glant on askubuntu.com
 
@SmokeDetector k
 
A J
@SmokeDetector k
 
1:31 PM
[ SmokeDetector | MS ] Mostly punctuation marks in answer: How willl I set MySQL enum datatype default value as 'No'? by Der Safter on stackoverflow.com
 
ooo yay, that "Is Metasmoke Alive" git code stuff works :D
@Glorfindel I'm not sure if the feedback commands actually check if MS is up or not
AFAIK that code is not global
 
@ThomasWard I'd expect it to throw an error.
Or at least say something that actually is the truth.
 
as would I, but we'll have to dig into it deeper
 
Sure :)
 
@Glorfindel it's probably actually erroring, but it's being suppressed or something
it'd need testing :)
and i'm a bit busy sooo
 
1:42 PM
Yeah, it's a low prio thingy.
 
!!/blame
 
@ThomasWard It's Erik von Asmuth's fault.
 
yeah we'll blame them for the feature being missing, @Glorfindel :)
just because.
 
@ThomasWard feedback does stuff that isn't metasmoke, then throws MS sending into a different thread
Or at least that's what it used to do
 
@Undo so there's a component that isn't MS then
which would explain why it wouldn't error.
 
1:47 PM
Whitelisting/blacklisting, yeah
Can look at why it's dead in a few
 
right, I think Glorfindel's complaint is that it came back saying this:
30 mins ago, by SmokeDetector
@tripleee Recorded answer as an NAA in metasmoke.
 
Bet it's that silly CPU thing again
 
@ThomasWard Oh. That's because IIRC NAA feedback only does MS stuff
So that's the message someone wrote
 
indeed.
we can stab it later :P
my guess is we're silently absorbing the "MS Is Down" exceptions
 
1:50 PM
@ThomasWard So hypothetically, if you had a server that mysteriously killed itself every few days, with high CPU usage and you can't SSH in during the event, what'd you do to track it down?
 
CPU problems again?
@Undo i'd do some preventative tracking ahead of time, pull data out of, say, top, to get an idea of what processes (as "timed snapshots" once a minute) are using the most CPU data
 
Yeah, tied to Maria upgrade
 
and then when you break the server again and restart trace the cause
@Undo that sounds like MariaDB isn't properly tuned to the system resources
or you have a running process that doesn't die and just constantly pegs
 
@ThomasWard It's sudden, though. Runs fine then suddenly eats it
 
considered offloading the DB to a separate machine?
 
1:52 PM
Ugh
 
@Undo the question becomes two-fold
(1) what command is running at the time of the CPU spike
(2) are we 100% certain it's MariaDB eating the resources
if #2 is that it's certain it's MariaDB, retuning it to the system resources makes sense
if #1 is just a simple select or something then that's different
determining what causes MariaDB to eat all the memory however is not easy
and I don't use MariaDB as my default, I use Postgres
so I don't know how to figiure out #1 there
 
Gonna just throw ps output into a file every few minutes
Thanks :)
My only evidence for it being Maria is that it started happening immediately after that upgrade
 
yeah I'd try and see what process is actually eating the memory first
if it's MariaDB then the daemon for it will nom all the CPU
if it's not MariaDB but, say, MS's Ruby, it'll be the ruby process eating CPU
and wrt MariaDB I Have had a few Python scripts explode because of MariaDB locking, and it resulted in me getting so angry at it I ported those apps to postgres ;P
 
2:14 PM
!!/status
 
@Undo Running since 11:56:11 UTC (2 hours, 18 minutes, 10 seconds)
 
!!/reboot
 
Goodbye, cruel world
[ SmokeDetector ] SmokeDetector started at rev fde0559 (iBug: Fix MS down generating TypeError) (running on angussidney/droplet, Python 3.6.5)
Restart: API quota is 11211.
 
@Undo It also always happens during one of the scheduled jobs, no?
 
@thesecretmaster Technically yes, but that scheduled job runs once every three minutes
 
2:16 PM
...whut?
What job is that?
 
When it was at 11a UTC there was a clear suspect, now it's just the check_smokey_status thing
 
This one didn't seem to fail until 11:30ish AM UTC
 
Bleh
Can't you put mariadb in docker and docker can do memory/cpu limiting?
 
Got a more robust ps-logger thing, should be able to catch it next time
@thesecretmaster EC2 doesn't like docker
virtualizing inside virtualizing didn't work last time I tried
hackernoon.com/running-docker-on-aws-ec2-83a14b780c56 claims it should just work, though. Maybe something changed
 
2:18 PM
Oh. Well, I'd offer server space for either application code or the DB, but they both have private info, so that can't happen.
 
It's not like you can't get to that private info anyway, the difference is the current setup would leave a trail
 
@Undo do you have Maria's long running queries being logged?
 
Not that I know of
 
mariadb.com/kb/en/library/slow-query-log-overview Enable those, so we can see what is running for an unusual amount of time
 
[ SmokeDetector | MS ] Potentially bad keyword in body: International vehicle registration sticker by Dan Lasak on travel.SE
fp- by CalvT
[ SmokeDetector | MS ] Mostly punctuation marks in answer, repeating characters in answer: Table Salt vs Kosher Salt difference by snoopyhater on chemistry.SE
naa- by CalvT
[ SmokeDetector | MS ] Pattern-matching product name in body: Unexplained Low LTFT on my Mazda cx7 2.3 turbocharged ✏️ by MAK on mechanics.SE
tpu- by Glorfindel on Table Salt vs Kosher Salt difference [MS]
fp- by Glorfindel
 
2:32 PM
!!/watch itineraires-lointains\.fr
 
@Glorfindel Added itineraires-lointains\.fr to watchlist
 
!!/watch- amazontedx\.com
!!/watch- diversesign\.com\.au
 
[ SmokeDetector ] SmokeDetector started at rev 385cd96 (SmokeDetector: Auto watch of diversesign\.com\.au by Glorfindel --autopull) (running on angussidney/droplet, Python 3.6.5)
Restart: API quota is 10975.
 
in SO Close Vote Reviewers on Stack Overflow Chat, 43 secs ago, by gunr2171
smokey, you need to escape that
 
CI on b21b473 succeeded.
 
2:42 PM
@CalvT yet another depiction of a severed arm... :/
 
CI on e312a19 succeeded.
 
!!/pull
 
[ SmokeDetector ] SmokeDetector started at rev e312a19 (iBug: smokey, you need to escape that) (running on angussidney/droplet, Python 3.6.5)
Restart: API quota is 10883.
[ SmokeDetector | MS ] No whitespace in title: InvalidSelectorException by Vipz on ru.stackoverflow.com
fp- by iBug
 
@iBug If I want to make a pull request to fix the bug I just submitted, what branch to I target?
 
@gunr2171 master, of course
 
2:47 PM
thanks, just checking
 
@Undo mmm, Docker runs very well in my Ubuntu 18.04 running in VMware on my Windows laptop.
Provided that I virtualize VT-x in VMware settings
So, it's likely that EC2 didn't virtualize those virtualization technologies themselves, which are critital to containers
 
[ SmokeDetector | MS ] Offensive answer detected: Visual Studio Code SQL results dock to bottom by Mike Twc on stackoverflow.com
fp- by iBug
CI on d1beef1 succeeded.
[ SmokeDetector | MS ] Pattern-matching product name in body: Unexplained Low LTFT on my Mazda cx7 2.3 turbocharged ✏️ by MAK on mechanics.SE
fp- by Erik the Outgolfer
 
3:16 PM
!!/alive
 
@ArtOfCode You doubt me?
 
Given that I got a hair on fire email... yes
 
!!/ms-status
 
@thesecretmaster metasmoke is up. Current failure count: 0
 
4:02 PM
!!/dead
 
@ThomasWard No such command 'dead'.
 
CI on 6c0a7ef succeeded.
CI on bafab7b failed.
CI on ef900d1 succeeded.
 
CI exploded >:D
 
4:11 PM
@SmokeDetector k
!!/watch long-term-pc-support\.us
 
@ThomasWard Added long-term-pc-support\.us to watchlist
CI on 85cf01c succeeded.
CI on bc8459d succeeded. Message contains 'autopull', pulling...
 
Zoe
The question looks like a seed
 
[ SmokeDetector ] SmokeDetector started at rev bc8459d (SmokeDetector: Auto watch of long-term-pc-support\.us by Thomas Ward --autopull) (running on angussidney/droplet, Python 3.6.5)
 
well its gone now so :P
 
Restart: API quota is 9814.
 
4:16 PM
@SmokeDetector the user self deleted their question so if it is a spam bot they made a clever one ...
 
[ SmokeDetector | MS ] Potentially bad keyword in answer: Viewers is more than 500000 per month. Which Hosting type will be good? by Ghulam Qadir on stackoverflow.com
tpu- by Zoe
 
Zoe
@SmokeDetector Appears to be using an ad campaign link.
 
I've mod flagged, as they've done at least 3 (out of 5 posts I opened)
 
!!/watch click\.linksynergy\.com
 
4:27 PM
@Makyen Added click\.linksynergy\.com to watchlist
[ SmokeDetector | MS ] Bad keyword in body: Mankaran singh kr by Mankaran singh on biology.SE
 
@SmokeDetector The bad keyword is the entire post body?
 
5:13 PM
[ charcoal-se.github.io ] continuous-integration/travis-ci/push success on 442574e: The Travis CI build passed
[ metasmoke-deploy ] deploy started by ArtOfCode
[ charcoal-se.github.io ] continuous-integration/travis-ci/push success on 441020d: The Travis CI build passed
[ metasmoke ] ci/circleci success on fca6f78: Your tests passed on CircleCI!
tpu- by Glorfindel
[ SmokeDetector | MS ] Linked punctuation in answer: Lollipop update on MotoG causing no service for sim card by T.J Vassily on android.SE
tpu- by bertieb
 
!!/watch futuretechmagazine\.net
 
@bertieb You don't have code privileges, but I've created PR#2415 for you.
 
@SmokeDetector f
 
Merged SmokeDetector #2415.
fp by FrankerZ
 
5:58 PM
[ SmokeDetector | MS ] Bad keyword in link text in answer: How do I clear the screen command history scroll-back? by rogersteve on askubuntu.com
tpu- by ByteCommander
 
6:15 PM
!!/watch- technologyaol\.doodlekit\.com
 
tpu- by Makyen
 
!!/watch-force onlinephptraining\.com
 
@Makyen That pattern looks like it's already caught by Bad keyword in username; append -force if you really want to do that.
@Makyen Added onlinephptraining\.com to watchlist
[ metasmoke-deploy ] deploy started by Undo
[ metasmoke ] ci/circleci: rubocop failure on 4735711: Your tests failed on CircleCI
 
yeah whatever
 
6:38 PM
[ metasmoke-deploy ] deploy started by Undo
 
:/
 
[ metasmoke ] ci/circleci success on e0e844b: Your tests passed on CircleCI!
 
@Undo pretty sure they just want more of your assets
 
@Undo Travis?
Or circle?
I know travis wants you to email them or fill out some form if you're open source
 
[ SmokeDetector | MS ] Potentially bad keyword in answer, blacklisted user: How to fix "Google couldn't verify this account belongs to you"? by Dan Dodd on android.SE
[ SmokeDetector | MS ] Link at end of answer: Creating in PS a bleed for a die cut sticker by Steve on graphicdesign.SE
Recovered from websocket._exceptions.WebSocketConnectionClosedException: Connection is already closed.
 
6:49 PM
!!/ms-status
 
@thesecretmaster metasmoke is up. Current failure count: 0
naa- by Erik the Outgolfer
[ SmokeDetector | MS ] Potentially bad keyword in answer: python requests .status_code not returning correct value by LeonardoPRG on stackoverflow.com
fp- by Erik the Outgolfer
API quota rolled over with 8245 requests remaining. Current quota: 19999.
math: 321
stackoverflow: 255
physics: 74
askubuntu: 63
superuser: 62
english: 47
unix: 36
serverfault: 33
apple: 30
electronics: 27
money: 24
academia: 23
dba: 22
codegolf: 22
security: 20
workplace: 20
drupal: 18
es.stackoverflow: 16
webapps: 15
mathoverflow.net: 14
mathematica: 14
salesforce: 13
scifi: 13
meta: 13
buddhism: 13
travel: 13
stats: 11
diy: 11
puzzling: 10
pt.stackoverflow: 9
wordpress: 9
rpg: 9
gis: 9
hinduism: 9
worldbuilding: 8
history: 8
skeptics: 8
ell: 8
interpersonal: 8
ethereum: 8
 
7:04 PM
@thesecretmaster That one's Circle
 
@SmokeDetector The supplied phone number is not official. This is the same question on which the user previously posted an answer with the same non-official "support" phone number.
 
7:24 PM
[ SmokeDetector | MS ] Potentially bad ns for domain in body: SD card reader with detachable cord and SD card fitting inside ✏️ by coder on hardwarerecs.SE
 
@Undo Had that a few times, but it's not actually affected us AFAICT
 
[ SmokeDetector | MS ] Bad keyword with email in answer, email in answer: Can you hurt a nonexistent credit score? by Glenn Miller on money.SE
tpu- by Erik the Outgolfer
 
7:54 PM
[ SmokeDetector | MS ] Bad pattern in url answer: Unable to access websocket over Kubernetes ingress by Khaly on stackoverflow.com
fp- by Erik the Outgolfer
 
!!/watch cyberspywear(?:@gmail\.com)?
 
@Makyen Added cyberspywear(?:@gmail\.com)? to watchlist
 
!!/watch- [a-z_]*(?:1_)?951[\W_]*364[\W_]*0961[a-z_]
 
8:13 PM
[ SmokeDetector | MS ] Potentially bad keyword in answer: How to boot from new hard drive by Earl M Buzzell on superuser.com
tp by Nisse Engström on L'Averla Paris Anti Aging Cream [MS]
[ SmokeDetector | MS ] Pattern-matching website in answer: How to give new hard drive "boot, page file, crash dump" status by Earl M Buzzell on superuser.com
naa- by Makyen
 
8:31 PM
!!/test-a thevideos.us
 
> Potentially bad keyword in answer
----------
Body - Position 1-12: thevideos.us
[ SmokeDetector | MS ] Potentially bad keyword in answer: iphone UISearchBar Done button always enabled by JasonD on stackoverflow.com
fp- by bertieb
 
@SmokeDetector Cross-network posting links to this site (Workplace and SU so far)
 
@SmokeDetector Video is on YouTube but that link takes to a mirror site; something afoot (?)
 
[ metasmoke ] ci/circleci success on 5d9e2d5: Your tests passed on CircleCI!
[ SmokeDetector | MS ] Potentially bad ns for domain in answer: Batch Geocoding with Garmin Mapsource by Ramsay Gordan on stackoverflow.com
[ metasmoke-deploy ] deploy started by ArtOfCode
[ metasmoke-deploy ] deploy started by ArtOfCode
 
9:00 PM
[ SmokeDetector | MS ] Potentially bad keyword in answer: How difficult is it to setup a mailserver? by Francis Ndungu on serverfault.com
 
9:11 PM
@SmokeDetector modflagged. All of this user's posts mention Alibaba Cloud Something-Or-Other, which makes my spidey-spammy-sense tingle.
 
9:25 PM
[ SmokeDetector | MS ] Few unique characters in body, repeating characters in title, title has only one unique char: ??????????????????????????????? ✏️ by Daten on math.SE
[ SmokeDetector | MS ] Few unique characters in body, mostly dots in body, repeating characters in title, title has only one unique char: ????????????????????????????????????????????????? ✏️ by Daten on math.SE
[ SmokeDetector | MS ] Few unique characters in answer, no whitespace in answer, repeating characters in answer: How to do Kano model in R by Artem on stackoverflow.com
tpu- by Erik the Outgolfer
naa- by Makyen
 
9:48 PM
[ SmokeDetector | MS ] Bad keyword with email in answer, pattern-matching email in answer: Apologizing for an accident that occurred outside the workplace by Jessica Farronato on workplace.SE
[ SmokeDetector | MS ] Potentially bad keyword in body, potentially bad keyword in title: 1,4 Addition of Secondary amine to conjugated ketone ✏️ by user67895 on chemistry.SE
tp by Tetsuya Yamamoto on How to do Kano model in R [MS]
naa by Tetsuya Yamamoto on How to boot from new hard drive [MS]
tp by Tetsuya Yamamoto on follicle rx dischem south africa [MS]
naa by Tetsuya Yamamoto on Error ITMS-90717: "Invalid App Store Icon" [MS]
tp by Tetsuya Yamamoto on Regal Keto weight burn [MS]
tp by Tetsuya Yamamoto on Batch Geocoding with Garmin Mapsource [MS]
[ SmokeDetector | MS ] Pattern-matching product name in body: Is this a valid argument for the dimensions of this integral? by jacob on physics.SE
fp by Tetsuya Yamamoto
tp by Tetsuya Yamamoto on ??????????????????????????????? [MS]
 
10:05 PM
Rails JS in general, and Turbolinks in particular, are monstrosities and should be killed with extremely prejudicial fire.
This bug has literally had me stumped for weeks
 
[ SmokeDetector | MS ] Blacklisted user: Vectorized approach for the following task by Artem on stackoverflow.com
Restart: API quota is 18282.
 
10:34 PM
@ArtOfCode I just got a mail from zoho that my account was deleted. This zoho thing was something we temporarily used to sign up to the charcoal SO channel, right?
 
aye
it gave me a shiny button that said clean up inactive accounts so I clicked it :P
 
Okay, then I don't have to worry that it might have been something I should be concerned about.
 
[ SmokeDetector | MS ] Pattern-matching product name in body: Is this a valid argument for the dimensions of this integral? ✏️ by jacob on physics.SE
 
10:49 PM
fp- by Makyen
[ metasmoke ] ci/circleci: js failure on ccc83a6: Your tests failed on CircleCI
 
shut up
 
[ metasmoke ] ci/circleci: js failure on b5cb7ea: Your tests failed on CircleCI
 
11:56 PM
tpu- by ByteCommander
 
00:00 - 12:0012:00 - 00:00

« first day (1761 days earlier)      last day (2063 days later) »