« first day (2255 days earlier)      last day (1568 days later) » 

@Undo Osiris is being pointed to it as well
 
@ThomasWard I'm inclined to just go for it, whenever it's convenient for you to flip the switch on LE certs for metasmoke. and ms.
 
teward/Osiris received failover signal.
 
@Undo that part's easy
 
@SmokeDetector hah, so that works
 
3:00 AM
Restart: API quota is 16111.
 
i can have both URLs point at the same but can't do the LE for it until DNS is done
which means MS will have an outage
 
We're gonna have an outage anyway
Weekend is the time to do it
 
true
 
Wait
Can't you point smokey at the test domain?
 
3:01 AM
!!/status
 
@thesecretmaster I did - with Osiris
 
@Undo Running since 22:19:02 UTC (4 hours, 42 minutes, 13 seconds)
 
Cuz I think we want to minimize downtime
 
[ SmokeDetector | MS ] Potentially bad keyword in body (1): Is e-exch.net website is safe for buy or sell e-currency? by Hannah Masroor on security.SE
[ SmokeDetector ] SmokeDetector started in standby mode at rev 7edfd61 (SmokeDetector: Auto watch of (?-i:XwHeR\.jpg) by Makyen) (running on iBug/Coral Cove)
 
I can keep both domains online as well
for a while
that way we get time to migrate
 
3:03 AM
Okay, I'm going for it
We'll accept a bit of downtime
 
@Undo let me know when you're ready for me to run the certs
 
@ThomasWard DNS switched, A's to 173.167.254.37 on both metasmoke. and ms.
 
[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Is SwiftUI framework declarative or imperative or neither? by hlayamin eain on stackoverflow.com
 
LOL I just got locked out of my admin login xD
goes to fix the login on the HAProxy
 
[ SmokeDetector | MS ] Pattern-matching website in body (97): Did the GeoPush active successfully? by user78046 on salesforce.SE
 
3:05 AM
oh
Actually, @ThomasWard it's m.erwaysoftware that we want
sorry
 
@Undo m or metasmoke or both?
 
Both
m instead of ms that I was saying
m is the "we don't have much space in chat messages" thing; it just gets redirected
 
Shouldn't we be getting double reports from SD at this point? One from Isis and the other from Osiris?
 
osiris is off
:)
 
!!/location
 
3:07 AM
@Undo teward/Isis
 
@ThomasWard OK. I thought you have left it pointing to the migration machine.
 
(m|metasmoke|ms)\.erwaysoftware\.com ? Or just (m|metasmoke)?
 
The latter
 
Yeah sorry for confusion
 
3:08 AM
are both pointed to the new IP?
 
A's on both
 
Ah, I just thought to myself "should I have access, aren't there tokens on that box" then I remembered there aren't :)
 
activating multidomain certs
 
Getting Redunda cert on metasmoke., that's progress
 
3:10 AM
Heh
 
that's my fault
cert has to be issued first
and i fubar'd the regex for the LE verification 1 moment
 
kk, ping when it's ready
 
check now
 
LE does multi site certs?
 
theeeere we go
@thesecretmaster yeah, I had one at one point
 
3:12 AM
Neat, good to know
 
nginx stopped on old server
 
@thesecretmaster yep, at the LE level it's just -d domain -d altdomain -d foobarbaz -d hailhydra ... as args
 
Sandy Shores apparently already picked up DNS
 
iBug/Sandy Shores received failover signal.
 
yep DNS propagation is fast
i'm restarting isis shortly
 
3:13 AM
kk
Stopping on new server, to bring dumps across
hard downtime starting now
 
Restart: API quota is 19998.
 
> 503 service unavailable

ummmmmm
that's not good
 
I killed it
intended
 
ah
OK for a moment I thought I fubar'd things xD
 
no point in allowing reqs when I'm just gonna clobber with old db dump
 
3:15 AM
indeed
 
Yeh, this is the Good thing
 
at least the 503 was a fast response heh
 
> Jobs a goodun
I always appreciate that
 
SQL backup script says that
 
3:16 AM
hah
 
Oh, kewl
 
!!/watch- liemcine\.com
 
old Redis instance is unhappy for some reason
 
[ SmokeDetector ] Blacklists reloaded at rev 50b638c (SmokeDetector: Auto watch of liemcine\.com by Makyen) (running on iBug/Sandy Shores)
 
3:19 AM
@Undo What kind of unhappy?
 
doesn't wanna start SSH session
gonna reboot it
 
Warning: iBug/Sandy Shores: 28 latest connections to metasmoke have failed. Disabling metasmoke
 
yes I know
didn't... really expect this one
 
do I need to kick the box at the VMware level?
 
no this is on my side
 
3:22 AM
ah OK
 
metasmoke dump import is acting weird
taking way too long
 
[ SmokeDetector | MS ] Url-only title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +13 more (1190): fitnesreviews.com/pro-keto-rx-uk/ by leshasoules on askubuntu.com
 
undo@metasmoke:~$ sudo mysql -u root metasmoke < ms.sql
^CCtrl-C -- query killed. Continuing normally.
^CCtrl-C -- query killed. Continuing normally.
^CCtrl-C -- query killed. Continuing normally.
ERROR 2013 (HY000) at line 2434: Lost connection to MySQL server during query
Rebooting, then gonna try again
 
fyi i'm keeping an eye on the system usage on my side just in case
 
kk
just dropped the db, restarting import now
This is weird, it didn't barf before
 
3:28 AM
might've OOM'd. System reached 4GB usage
i can bump the RAM if necessary
 
no, it worked once without issue
ohhhhh
am idiot
pretty sure I had nginx running at the same time
 
What'd you do?
 
heh
 
Still running, still got mysql activity in htop
Don't like it taking this long though
 
3:33 AM
that might be because unattended-upgrades is running in the background
 
saturating disk IO that bad though?
 
no that shouldn't happen
disk doesn't look pegged though
 
There it finished
\o/
Now just Redis backup and we're good
(maybe)
 
Redis doesn't need a backup
Stop it and just grab the rdb
 
well yeah
that's what I'm doing
wrong word
 
3:37 AM
Perfect
 
wtf
did I forget how SCP works?
 
Not perfect?
 
$ ssh -i EC2KeypairExample.txt ubuntu@3.81.154.25:/home/ubuntu/dump.rdb .
ssh: Could not resolve hostname 3.81.154.25:/home/ubuntu/dump.rdb: Name or service not known
ten points to whoever spots it first
 
ssh
You mean scp
 
AAAAH
Egads
 
3:38 AM
10 POINTS WOOOOOOOOOO
 
8 mins ago, by Undo
am idiot
 
Also I don't think scp takes -i?
 
yeah it does
 
I think you need it in your .ssh/config
 
just gotta be in the right place
 
3:38 AM
Oh cool
I'm thinking of rsync, nevermind
 
yeah that one's more weird
scp takes basically all the ssh args
One more reboot and we should be up
and there we go?
!!/location
 
@Undo iBug/Sandy Shores
 
Feels kinda slow? Maybe it's just me
 
3:43 AM
No report in ms
 
Or the caches propogating
!!/location
 
@thesecretmaster iBug/Sandy Shores
 
!!/ms-status
 
@thesecretmaster metasmoke is down. Current failure count: 31 (iBug/Sandy Shores)
 
oh right
!!/ms-up
 
3:44 AM
@Undo metasmoke is now considered up.
 
[ SmokeDetector | MS ] Potentially bad keyword in body (1): Drupal 7 and Lets’ encrypt redirection problems by ShimonD on drupal.SE
 
There it is
 
well there's a real one, okay
Yep, that's working
 
3:44 AM
sweet
All my stuff is shut down now
So it's all new now
 
Beautiful
 
and seems to be working stable
 
@SmokeDetector f
Well that wasn't so bad
Sign in works
Gonna guess Blazer will be broke
 
Want to pin a thing where we tell folks to ping Thomas if its down instead of Undo? Cuz now only Thomas can kick the instance.
 
I can do a bit of kicking
Not as much though
 
3:46 AM
[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Cant start docker-compose & failes on No module by jeff on stackoverflow.com
 
yeah Blazer is much unhappy
 
Also, want to take down the MS restart page?
After you deal with blazer
 
> Potentially bad keyword in answer - Position 0-12: <p><a href="
wat
 
That's not MS side, looks the other way and starts whistling
 
lol
too lazy to blame it right now
 
3:48 AM
!!/blame
 
@thesecretmaster It's John Dvorak's fault.
 
Database dumps are gonna be broken too
 
Aren't those pretty easily fixable though?
Redis dumps should be basically the same
 
Yeah, just gotta set up cronjobs and S3 access
 
Just with cp instead of scp
Oh right, s3
 
3:52 AM
oh gotta set up travis deploys too
 
[ metasmoke-deploy ] deploy started by Undo
 
... hellooooooo?
 
[ metasmoke ] ci/circleci success on e6eb1fa: Your tests passed on CircleCI!
 
@Undo Hi
 
manually triggered
lol hi
hah someone forgot how the travis auth system works
Okay so todo
1. Setup Travis deploy jobs (just gotta get the key right)
2. Setup MySQL backups + dump downloads
3. Setup Blazer
Core functionality is there, so I'm done for a bit
 
4:06 AM
[ SmokeDetector | MS ] Link at end of answer (60): Where can I find high quality mock-ups? ✏️ by Tom Dijkslag on graphicdesign.SE
fp- feedback received
[ SmokeDetector ] SmokeDetector started in standby mode at rev 50b638c (SmokeDetector: Auto watch of liemcine\.com by Makyen) (running on Makyen/Windows)
 
core functionality is working though, in a stable way, I take it.
Looks good to me :P
 
anyone want to see a multicolored barf chart of CPU usage on the host? xD
 
[ SmokeDetector | MS ] Potentially bad ns for domain in answer (1): 10 years intraday for a single stock by Peter Smith on quant.SE
 
4:08 AM
fp- feedback received on Did the GeoPush active successfully? [MS]
 
and no that is not all MetaSmoke :)
 
"barf chart"
best thing ever
 
there's 16 CPU cores on the host node soooooo
that turns into multicolored barf on the resource charts xD
 
!!/bisect joyoshare
 
@bmike Matched by joyoshare on line 15135 of watched_keywords.txt
[ SmokeDetector | MS ] Potentially bad ns for domain in answer (1): How can you connect an Amiga 500/600/1000/2000 to a modern monitor? by Murray on retrocomputing.SE
 
4:23 AM
@ThomasWard :-)
 
fp- feedback received
[ SmokeDetector | MS ] Potentially bad keyword in body (1): Sharepoint online - Folder level access - Settings revert back to "view only" by Rachel on sharepoint.SE
fp- feedback received on 10 years intraday for a single stock [MS]
fp- feedback received
 
4:41 AM
[ SmokeDetector | MS ] Offensive body detected, offensive title detected, potentially bad keyword in body, potentially bad keyword in title, toxic body detected (158): Which is worse "Cunt" or "Fuck"? by Julian Tiemann on english.SE
 
Also me ;)
 
[ SmokeDetector | MS ] Potentially bad keyword in body, potentially bad keyword in title (2): Why were so many more Jewish men than Jewish women killed during the Holocaust? ✏️ by Kornja on history.SE
fp- feedback received
 
Even though it is essential, Social Media is crustier than the hairs of my dry butt hole
 
fp- feedback received on Which is worse "Cunt" or "Fuck"? [MS]
fp feedback on autoflagged post: Which is worse "Cunt" or "Fuck"? [MS]
Autoflagged FP: flagged by @SmokeDetector, @Rob, @Makyen
fp- feedback received
fp feedback received on 10 years intraday for a single stock [MS]
 
4:57 AM
!!/watch- e-exch\.net
 
[ SmokeDetector ] Blacklists reloaded at rev 3c7531b (SmokeDetector: Auto watch of e-exch\.net by Eliah Kagan) (running on iBug/Sandy Shores)
fp- feedback received
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, link following arrow in body, +4 more (593): How Brilliance SF Works? by brillancsffr on meta.SE
tpu- feedback received
 
[ SmokeDetector | MS ] Manually reported answer (94): Soundconverter keeps flash crashing on Ubuntu 18.04 LT2 ✏️ by Bertelchen on askubuntu.com
naa- feedback received
 
5:15 AM
@SmokeDetector It seems to me that "please consider deleting yourself," directed at a specifically named user, qualifies as rude/abusive. Whether it was useful for me to report this to SmokeDetector and what feedback is appropriate I am not sure. (I'm not saying this to disagree with the naa feedback given thus far, just to explain what I was thinking in reporting it.)
 
[ SmokeDetector | MS ] Url-only title, bad keyword in body, bad keyword in title, body starts with title and ends in url, link at end of body, +8 more (694): lifecanrun.com/nitroalis-rx-male-enhancement/ by lipohearts4 on askubuntu.com
tpu- feedback received
[ SmokeDetector | MS ] Url-only title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +7 more (791): superfastketopills.com/al-roker-keto-pills/ by SarahMiddle on askubuntu.com
tpu- feedback received
 
[ SmokeDetector | MS ] Manually reported answer (94): How to delete file from folder using javascript? by Infinity Soft Technologies on stackoverflow.com
 
!!/watch- infinitysofttech\.in
 
[ SmokeDetector ] Blacklists reloaded at rev c8ebe88 (SmokeDetector: Auto watch of infinitysofttech\.in by Makyen) (running on iBug/Sandy Shores)
tpu- feedback received
[ SmokeDetector | MS ] Potentially bad keyword in answer (1): The word “expanse” in Genesis 1:6-8 by Kore Burgett on hermeneutics.SE
fp- feedback received
 
5:58 AM
@EliahKagan it seems to be a response to a comment below the non-answer
 
@SmokeDetector While this is definitely NAA, the edit which produced this version was specifically, and only, adding a moderately rude response to a comment on the answer. Given that it was an edit that added this rude portion, it would be considered TP, as we would want to block the edit.
 
fp- feedback received
[ SmokeDetector | MS ] Bad keyword in link text in answer, link at end of answer (141): ios storing custom objects by Dellinspiron52 on stackoverflow.com
tpu- feedback received
 
!!/watch mysmartbazaar\.com
 
[ SmokeDetector ] Blacklists reloaded at rev 84c14e9 (SmokeDetector: Auto watch of mysmartbazaar\.com by tripleee) (running on iBug/Sandy Shores)
[ SmokeDetector | MS ] Bad ip for hostname in body, bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, +3 more (590): Now Must-haves Before Embarking On VIAXXL Male Enhancement by viaxxlmalepills on apple.SE
tpu- feedback received
[ SmokeDetector | MS ] Potentially bad ip for hostname in body (1): PHOTODIODE TO OPAMP ✏️ by jonasolof on electronics.SE
fp- feedback received
[ SmokeDetector | MS ] Potentially bad keyword in body, potentially bad keyword in title (2): Is Perfect money legal website is www.e-exch.net?Can we buy or sell perfect money by Hannah Masroor on security.SE
[ SmokeDetector | MS ] Url-only title, bad ip for hostname in body, bad ip for hostname in title, bad keyword in body, bad keyword in title, +7 more (989): topdietbrand.com/viaxxl-male-enhancement/ by franzrowed on askubuntu.com
tpu- feedback received
[ SmokeDetector | MS ] Bad keyword in answer, pattern-matching website in answer, potentially bad ns for domain in answer, potentially bad keyword in answer (167): Poisson Process Conditional Proability by josephranstetter on math.SE
[ SmokeDetector | MS ] Potentially problematic ns configuration in answer (1): How can I make a USB flash drive appear/not appear as a CD drive? ✏️ by a deleted user on superuser.com
tpu- feedback received on Poisson Process Conditional Proability [MS]
 
6:31 AM
!!/blacklist-keyword viaxxl(?:[\W_]*+(?:enhancement|male|\d++|[\da-f]{4,}+))*
 
!!/watch- (?-i:BCDIEXlby-4)(?#youtu.be)
 
[ SmokeDetector | MS ] Bad keyword in link text in answer, potentially bad ns for domain in answer (82): How can I deal with the Remove Curse spell spoiling my plot hooks? by akankshamehtha on rpg.SE
tpu- feedback received
 
!!/watch oneandonlytownhouses\.com
 
6:37 AM
is it just me or is the tag input box on domain pages in metasmoke broken?
I can enter stuff just fine but the rendering is completely off
 
@SmokeDetector By itself this could perhaps imaginably not have been spam, but the same user followed up with this question.
 
[ SmokeDetector | MS ] Url in title, bad keyword in body, bad keyword in title, body starts with title and ends in url, potentially bad ip for hostname in body, +3 more (392): up2health.com/slimphoria-keto/ by DonaMiller on apple.SE
tpu- feedback received
 
@tripleee It's broken for me as well. I can enter text but none of the existing tags are shown.
It says "No tags" and "Nothing selected".
 
thanks, I'll submit a bug report then
... though as of right now metasmoke is completely down for me
 
I'm having a problem where http:// URLs don't work. https:// URLs do work. The http:// URLs should, and usually have, redirected to https://, but that appears not to be working at all.
 
6:45 AM
[ SmokeDetector | MS ] Pattern-matching website in answer, potentially bad asn for hostname in answer, potentially problematic ns configuration in answer (81): Copper Coil and Electromagnetic Induction ✏️ by Surag Singh on electronics.SE
 
@EliahKagan For example, when I first wrote this message, I copied the URL from a SmokeDetector message. It didn't get me to the page. I manually added the s and it did (and I edited the message and my MS feedback so the working URL would be used). I don't think this has happened before though.
A simpler example is that, for me, metasmoke.erwaysoftware.com does not work; the page takes forever to load. metasmoke.erwaysoftware.com works immediately.
 
@tripleee Norepro, still dead?
 
If you have a problem with metasmoke, please open an issue on GitHub. We switched over to the new server for MS a few hours ago. So far, it's working well. However, there are going to be some things broken. We need to make note of them so they can be fixed.
7
 
@Undo no sorry, it was because of the https -> http issue reported by Eliah apparently
 
ah, yep
that'd do it
 
6:49 AM
it was giving me 503 for a http URL, dunno if it got redirected from https by some glitch or if Eliah linked to a http resource
@EliahKagan he didn't
 
I originally had. Then I edited the message to change it to HTTPS.
 
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching website in body, potentially bad ns for domain in body, +2 more (393): CountDown Keto Reviews: How does CountDown Keto perform in your body! by hnmbuszgqaop on askubuntu.com
tpu- feedback received
 
tag input boxes on domain pages are broken #703
 
[ SmokeDetector | MS ] Potentially bad ip for hostname in answer (1): How do I set my local git identity? by user1131548 on superuser.com
[ SmokeDetector | MS ] Bad ip for hostname in body, bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, +3 more (590): Keto Tone France by cadsyuzt on graphicdesign.SE
tpu- feedback received
fp- feedback received on How do I set my local git identity? [MS]
 
Access via http:// URLs is broken #704
 
7:01 AM
do we want or need that to work?
 
yeah, it'd probably be good
Will probably be an @ThomasWard (low priority) thing to redirect 80:http -> https
 
yeah, agree that a redirect to https is useful and probably necessary (pure http not so much)
 
Pure HTTP has never worked
always just served a 301
@tripleee Has that tag box always looked that weird?
 
no, that's part of the problem
 
Regarding http:// redirection, I think the intended behavior is for them to redirect to the corresponding https:// URL and that this worked before. But the main practical problem, I think, is that some links to metasmoke that SmokeDetector posts in chat are http:// links. Specifically, the "feedback received" links with "[MS]" at the end are http://. Perhaps that should itself be regarded as a bug in SmokeDetector.
But even if that is changed in SmokeDetector, there will still be old http:// link to metasmoke in chat and in feedback that people copied from SmokeDetector's chat messages.
 
7:04 AM
@EliahKagan yeah
^ that's what metasmoke.erwaysoftware.com/domains/50885 looked like before the breakage (still have it open, let's just hope I can avoid reloading it!)
 
That's how it should look, yeah
Maybe we got a new version of selectpicker... somehow
 
these were always slightly brittle
 
@tripleee Do you see the same box on the post page?
I've got a less fancy dropdown text field looking thing
 
it looks sane but now that you ask I'm not absolutely sure it's completely identical to what it used to look like
 
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, bad keyword in username, bad pattern in url body, body starts with title and ends in url, +7 more (872): Keto Prime Diet Review by keto prime diet pills on meta.SE
tpu- feedback received
 
7:12 AM
it doesn't have the "Nothing selected" default text that I have in the screen shot above
 
Different page, though
Eh, will figure it out in the morning
weiiiiird
I expected to see a failing network request or something obvious
 
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, pattern-matching website in body, potentially bad keyword in body (293): Supplement Rise by JosephJMabry on askubuntu.com
tpu- feedback received
 
autoflags a bit slower than usual
 
!!/watch- craiiczhards
 
7:16 AM
!!/watch- health\.blog
 
@EliahKagan That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body; append -force if you really want to do that.
 
!!/bisect health.blog
 
@EliahKagan Matched by health\.blog on line 4649 of watched_keywords.txt
[ SmokeDetector | MS ] Potentially problematic ns configuration in body (1): AG Grid how to set the row drag area icon sticky to the left cell in a row ✏️ by Itay Cohen on stackoverflow.com
fp- feedback received
tpu- feedback received
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, potentially bad ip for hostname in body, potentially bad ns for domain in body, +2 more (297): Where To Buy Viaxxl Male Enhancement? by JasonMury on graphicdesign.SE
tpu- feedback received
[ SmokeDetector | MS ] Url in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +5 more (591): reviews4buyers.org/hydressence-serum/ by hftysdw on puzzling.SE
tpu- feedback received
fp- feedback received
[ SmokeDetector | MS ] Potentially bad keyword in answer (1): integrating Django with Angular js by Shriniket Deshmukh on stackoverflow.com
[ SmokeDetector | MS ] Bad ip for hostname in body, bad keyword in body, bad keyword in title, blacklisted website in body, body starts with title and ends in url, +5 more (886): Empowered Boost Male Enhancement by Adamlson on superuser.com
[ SmokeDetector | MS ] Url in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +10 more (889): fitnessproductcenter.com/electro-keto/ by upinaraln on drupal.SE
[ SmokeDetector | MS ] Potentially bad keyword in answer (1): How do I play in an adventure as a player that I have run as a DM? by Irwiza on rpg.SE
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in title, pattern-matching website in body, +3 more (493): Nature Crave Keto Diet Supplement Ultimate Review [2020] by Splack1972 on meta.SE
[ SmokeDetector | MS ] Pattern-matching website in answer (79): Highlighting text for review by David Diamond on tex.SE
 

« first day (2255 days earlier)      last day (1568 days later) »