« first day (1422 days earlier)      last day (2414 days later) » 

12:01 PM
the email address I see is notifications@github.com but it comes with a real name which is not Jed Fox
but I'm probably mixing two different threads
yeah, sorry, I was trying to connect the email I was seeing in my phone with a PR I had seen before, it wasn't actually yours
 
tpu- by DraconicPenguin
[ SmokeDetector | MS ] Blacklisted user: yii2 REST api for file upload by ovicko on stackoverflow.com (@quartata)
 
@SmokeDetector f
 
[ SmokeDetector | MS ] Manually reported question: Online bus ticket booking by Roophka Pickdrop on travel.SE
 
12:07 PM
@tripleee User is not blacklisted.
 
@suraj User is not blacklisted.
 
@SmokeDetector roophkapickdrop.com already watched
 
[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, link at end of body, potentially bad keyword in body, +1 more: x4peak.com/ultavive-slim/ by fedric pure on drupal.SE
[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, blacklisted website in body, link at end of body, pattern-matching website in body, +2 more: Subsequently, l-arginine changes by Herg1970 on askubuntu.com
[ SmokeDetector | MS ] Username similar to website in answer: MTB with gears or non gears, suspension or non-suspension for stunts? by Team VRz on bicycles.SE
 
@SmokeDetector k
@SmokeDetector k
@SmokeDetector k
 
12:18 PM
@Undo I'm having problems accessing metasmoke, it seems like it doesn't respond after making a proper SSL connection: debug data in pastebin
 
watched teamvrz\.com, CI in progress
 
I know
 
!!/pull
 
[ SmokeDetector ] SmokeDetector started at rev 20f95f1 (tripleee: Remove duplicate 3dprintersbay\.com --autopull) (running on Henders/EC2)
[ SmokeDetector ] SmokeDetector started at rev 20f95f1 (tripleee: Remove duplicate 3dprintersbay\.com --autopull) (running on Henders/EC2)
teward/Solar Flare received failover signal.
 
@Undo if you need assistance to fund any upgraded hosting, please let me know. I'm sure there's others in here willing to also help @tripleee
 
12:25 PM
!!/rev
 
neither of which actually pulled the latest commit ... hmmmm.
!!/stndby teward/solar
typo galore
!!/standby teward/solar
 
another update, CI in progress
!!/pull
 
12:30 PM
mmmm still no? oh I guess the master -> deploy isn't happening, do I do that manually or ... just wait
 
Restart: API quota is 18935.
 
@tripleee Manually, it'll be a few minutes
@YvetteColomb Thanks :)
 
I also have a few servers with some large amount of idle time I can invest in the charcoal project
 
@Undo I think it's time to take some pressure off :)
 
12:33 PM
Pretty sure what I need is some competent sysadmin skills. Something changed recently, it went for months without me touching it
 
[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, +2 more: Real CBD Oil-100% Natural Oil by dhkfgdkjf on meta.SE
 
also, confusing as heck when your wifi drops in the middle of fighting a server
 
T-1 until SpaceX launch
 
I suspect we could throw double the hardware at this and it'd still break. Regex search is my current suspicion.
 
CI on 11c52ba succeeded. Message contains 'autopull', pulling...
[ SmokeDetector ] SmokeDetector started at rev 11c52ba (tripleee: Watch q\W?soft\W?ltd --autopull) (running on Henders/EC2)
Restart: API quota is 18848.
 
12:40 PM
2
Q: Don't age away spam/rude/abusive flags

DesireCurrently, spam flags and rude/abusive flags age away in 2 days if not acted upon. This doesn't really happen on active sites, but it does happen elsewhere. (For example, I have four aged-away spam flags on Chinese Language and three on 3D Printing. And had a bunch on Русский язык before leaving ...

 
@SmokeDetector k
how come those identical spam posts are not being flagged?
 
!!/watch impanix\.com
 
12:50 PM
@tripleee I hit an error while trying to run that command; run !!/errorlogs for details.
 
will add the commit locally and push instead then
!!/errorlogs 50
 
  File "/home/ubuntu/SmokeDetector/metasmoke.py", line 311, in update_code_privileged_users_list
    data=json.dumps(payload), headers=headers).json()['items']

  File "/usr/local/lib/python3.4/dist-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)

  File "/usr/local/lib/python3.4/dist-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)

  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 508, in request
Restart: API quota is 18729.
 
!!/pull
 
[ SmokeDetector ] SmokeDetector started at rev 11c52ba (tripleee: Watch q\W?soft\W?ltd --autopull) (running on Henders/EC2)
Restart: API quota is 18686.
 
!!/location
 
12:58 PM
@Henders Henders/EC2
[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, link at end of body, pattern-matching website in body, potentially bad keyword in body, +1 more: As much as we may cherish the wisdom with by user745500 on askubuntu.com
Restart: API quota is 18557.
[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, bad keyword in title, bad keyword in username, blacklisted website in body, +2 more: Geneticore Boost : All New Formula For Testosterone Boost by Geneticore Boost on askubuntu.com
[ SmokeDetector | MS ] URL in title, bad NS for domain in body, bad NS for domain in title, bad keyword in body, bad keyword in title, +4 more: x4up.org/vtl-male-enhancement/ by user258953 on apple.SE
 
sd k
 
1:17 PM
 
@SmokeDetector k
 
1:30 PM
Restart: API quota is 18304.
 
yeah, I don't understand any of this. It's like it's got a bad state somewhere, but the log messages are crap.
 
What's going on
 
and erwaysoftware itself seems to be down
 
Oh MS
 
@Mithrandir usually is
 
1:33 PM
why does this keep happening
 
@Undo I mean both metasmoke.erwaysoftware and erwaysoftware gave me a 'refused to connect'
 
if I knew I would go fix it
 
@Undo Then you could just undo the error
</terriblepun>
 
@Magisch And then it will work like magic...
 
Restart: API quota is 18185.
 
1:42 PM
I like how undo doesn't even dignify it with a response
 
[ SmokeDetector | MS ] URL-only title, bad NS for domain in body, bad NS for domain in title, bad keyword in body, bad keyword in title, +5 more: x4up.org/vtl-male-enhancement/ by oprs on askubuntu.com
 
Okay, I gotta look at this sometime else. Nothing makes sense.
 
@SmokeDetector k
 
someone needs to come up with a hosting solution for developers that just works.
 
@Undo there is no such environment :P
 
1:46 PM
I know, but this is insanity. At least give me an error message
@ThomasWard ... interested in looking at what's going on? Maybe you see something obvious?
(probably not right now, but when it makes sense for you and me at the same time)
 
[ SmokeDetector | MS ] URL in title, bad NS for domain in body, bad NS for domain in title, bad keyword in body, bad keyword in title, +5 more: x4up.org/vtl-male-enhancement/ by amy whitelege on astronomy.SE
 
@Undo context? I'm too lazy to scroll up and read backwards.
a summary of the existing issue would be nice
 
@SmokeDetector tpu-
 
@SmokeDetector k
 
@ThomasWard Passenger startup processes are hanging, for some reason. I can't figure out how to tell what they're doing.
... Redunda just worked
and now the spawn preparer is running at 100% again. Bizarre.
 
1:50 PM
remind me what passenger does again? Ruby/Perl?
 
privacy error on MS now
 
and this is for MS?
 
@ThomasWard Ruby server, yeah
@ThomasWard and Redunda, same server and same (apparent) issue
 
@Undo New flaw:
 
I'd guess it's some kind of resource deadlock, but it's beyond me what.
 
1:51 PM
LE cert is missing a SAN for metasmoke.erwaysoftware.com
that's as simple as requesting the cert with an extra domain on the cert, but that's a meh issue.
@Undo I actively avoid Passenger
if i can avoid it.
 
I might start doing that
@ThomasWard yeah, it's pulling the wrong one most likely. Probably the one for Redunda, because I shut the metasmoke config file off to see if Redunda could start on its own
 
I don't know enough about Passenger or Ruby to really help that much, unfortunately.
 
Restart: API quota is 18058.
 
I can spin up testing instances to try and find out but meh
 
1:53 PM
STOP DYING SMOKEY I'LL BURN YOU
 
@ThomasWard yeah, I could do that too. Thanks anyway :)
 
!!/location
 
@ThomasWard Henders/EC2
 
Probably going to move over to Puma again. I've done that before.
in like 2015
 
@SmokeDetector Dead post is now dead, the power of the hammer of the Gods.
 
1:53 PM
Also, this is why a serverless architecture is so appealing
 
@ThomasWard you still demon thomas ward?
 
@Magisch moreso today than normal
E: No Coffee
 
__ward__en demon
<grumpy bad puns fueled by lack of peppermint tea to drown my sorrows in>
 
E: No Coffee
E: No energy
E: No Food
E: Depressing day
E: Rain
[KERNEL PANIC] Too many issues attempted to be resolved; system tried to kill init.  PANIC!
 
Possibly simultaneously the most fun and scariest sentence I have ever typed:
"We've just gotta load test prod"
4
 
1:55 PM
@ArtOfCode That... is concerning
 
E: no Tea
E: busy day at work
 
Restart: API quota is 17939.
 
@ArtOfCode I wonder, what tense is that?
"We have just got to"
 
"We just have to", actually. Present.
 
interesting
status:english-too-hard
 
2:05 PM
@Undo do you keep a log of regex searches?
 
E: syntax error E: failed to generate amusing message
 
regex search has definitely tanked MS multiple times, even innocuous ones :(
 
How would you do a serverless MS
 
connect all the Smokeys to each other
 
@quartata Yes
 
2:07 PM
I've discussed that before but no one wants to open ports for it
 
> Url in title
----------
Title - Position 1-26: https://stackoverflow.com
 
@Undo anything good?
 
night all
 
@quartata I don't know when this started happening, can't really pinpoint anything
 
2:09 PM
night o/
 
> Url in title
----------
Title - Position 1-27: https://www.000webhost.com
 
maybe look for things with \b
 
I can't even get Rails to start. At all.
 
reboot the server maybe, sounds like you've got zombieees
brains
 
2:11 PM
yeah, I did
a lot
 
mm well something is definitely running on port 80
 
Nginx is, Passenger won't start
 
is nginx supposed to be up
 
yes
 
at the risk of sounding like Microsoft technical support did you try turning that off and on again
 
2:13 PM
Restart: API quota is 17826.
 
oh yes
 
Annoying that this 'Passenger' won't dump something to the log though... I hate that
 
ok well maybe Passenger leaves behind a PID lock file of some kind
 
hrmph
hold on
 
maybe in /var
 
2:15 PM
[ SmokeDetector ] SmokeDetector started at rev 11c52ba (tripleee: Watch q\W?soft\W?ltd --autopull) (running on Henders/EC2)
Restart: API quota is 17813.
 
or ask google
 
@quartata I just did that :P
 
I'll give ya'll everything I have:
For Redunda, but metasmoke does the same thing:
[ E 2017-10-09 13:58:30.5207 4069/Tb age/Cor/App/Implementation.cpp:304 ]: Could not spawn process for application /var/railsapps/redunda/current: An error occurred while starting the web application: it did not write a startup response in time. If your app needs more time to start you can increase the Passenger start timeout config option.
  Error ID: 49c0f707
  Error details saved to: /tmp/passenger-error-zylYfF.html
  Message from application: An error occurred while starting the web application: it did not write a startup response in time. If your app needs more time to start you can in
This never used to be an issue. Timeout is 90s by default; it'd always start in less than 10. Until now.
That passenger-error-*.html file just contains the message, plus about a megabyte of config info
Nothing backtrace-ish that I can see
 
ohh I see so this is like some kind of fastcgi deal
 
Think so
Now, the fun part. Incoming screenshots of text.
 
2:20 PM
So I wouldn't have this if I have a local version of MS? It's just specific to the server that MS is currently running on?
 
well maybe in that case there's something wrong with your database, I could see all three applications getting hung trying to connect
 
[ SmokeDetector | MS ] Pattern-matching website in answer: How do I prevent memory-modification cheats? by Techinator on gamedev.SE
 
It's mysql right
 
First one is chronologically first; second two are the same rows scrolled horizontally.
@quartata yeah
 
2:23 PM
rack loader?...
I hope that token at the end isn't anything sensitive
 
doubt it
 
is this htop?
 
aye
 
are those threads or processes
 
Those SpawnPreparerShell processes are still running
uh hold on
 
J F
2:28 PM
> This website is powered by Passenger®, a rock-solid, feature-rich web application server
so much for rock-solid
 
@quartata ^ tree view
 
that's pretty
well it probably doesn't mean much other than that passenger no likey
kill them off and maybe try again with less +F /path/to/mysql/log?
 
There's something consistent that it no likey, and I think it changed in that last batch of auto updates on the old server
here we go
... /path/to/mysql isn't a path :P
 
have you tried it with like
a test rails app
 
@Undo Well...technically...probably an invalid one at least
 
2:32 PM
@quartata Redunda is pretty darn simple
2017-10-09T13:54:50.062406Z 8 [Note] Aborted connection 8 to db: 'redunda' user: 'root' host: 'localhost' (Got an error reading communication packets)
2017-10-09T13:56:33.344999Z 5 [Note] Aborted connection 5 to db: 'redunda' user: 'root' host: 'localhost' (Got an error reading communication packets)
2017-10-09T13:56:33.425760Z 7 [Note] Aborted connection 7 to db: 'sentinel' user: 'root' host: 'localhost' (Got an error reading communication packets)
2017-10-09T13:59:29.646900Z 12 [Note] Aborted connection 12 to db: 'redunda' user: 'root' host: 'localhost' (Got an error reading communication
(yeah, I'm using local root for each app 'cause I'm lazy)
But that's interesting
 
These are all the same instance?
 
Instance of?
Timing doesn't line up, though. Those were all a long time ago
 
AWS? Isn't that where this is running?
 
aye
 
hm
 
J F
2:39 PM
@Undo Something like now?
 
Looks kinda neat
Gotta run, plan to move to Puma eventually. ActionCable complicates things slightly, but it shouldn't be too bad.
 
3:05 PM
[ SmokeDetector | MS ] Username similar to website in answer: PHPMailer, parece que envía pero no llega el mensaje by boris on es.stackoverflow.com
[ SmokeDetector | MS ] Bad keyword in answer, pattern-matching website in answer: Testosterone in women vs. estrogen in men? by Mia on cogsci.SE
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, link at end of body, +4 more: Androforce x10 supplementch3mistry.com/androforce-x10/ by Dannishh on superuser.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link following arrow in body, pattern-matching website in body, +1 more: Working process of AndroForce X10: by Niae 19 on graphicdesign.SE
NobodyNada/Raspberry Pi received failover signal.
Restart: API quota is 19330.
Restart: API quota is 19309.
 
@SmokeDetector the more interesting pattern than testosterone seems to be the link to essayshark.com
 
@SmokeDetector tpu
 
@bummi Blacklisted user.
@bummi Blacklisted user.
 
J F
!!/location
 
3:16 PM
@JF Henders/EC2
 
J F
!!/standby Nobody
 
teward/Lunar Eclipse received failover signal.
Restart: API quota is 19285.
Restart: API quota is 12422.
 
J F
!!/location
 
@JF teward/Lunar Eclipse
@JF Henders/EC2
@JF NobodyNada/Raspberry Pi
 
J F
!!/standby p
 
4:02 PM
[ SmokeDetector | MS ] Bad keyword in title: Where should I do online training by Mahima Chaturvedi on stackoverflow.com (@quartata)
 
!!/unnotify~- 11540 stackoverflow.com
!!/location?
 
@quartata Henders/EC2
 
oh
that hasn't booted up in a while, probably has an old notification list
 
[ SmokeDetector | MS ] Mostly dots in answer: Python: force every import to reload by user8747133 on stackoverflow.com
 
4:17 PM
@quartata It's actually been up for longer than the other instances running have been :P It's my webserver so it runs continually anyway. It was the main instance 6 days ago and was pulled then
 
J F
@Henders notifications are local to the instance right now :(
 
:(
It's uptime is 108 days :O
(the server that is)
 
[ SmokeDetector | MS ] Link at end of answer: Computer OS in Arrow by monstercourses support on scifi.SE (@Mithrandir)
 
@SmokeDetector f
downvoted and edited to take out the link
 
teward/Lunar Eclipse received failover signal.
 
4:28 PM
!!/location
 
@Mithrandir teward/Lunar Eclipse
@Mithrandir Henders/EC2
Restart: API quota is 19999.
 
!!/standby teward
 
!!/rev
 
4:46 PM
> second_part_lower
death by variable name
pls
what does it meaaaann
 
J F
@quartata What is this from?
 
best guess: the second part of a command in lower case
 
ok I figured it out it's the feedback type that gets sent to metasmoke
so like fp- or tpu-
 
J F
MS is up now, but it’s slow.
 
@JF send_metasmoke_feedback and all of the feedback subcommands
 
4:53 PM
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, link following arrow in body, +1 more: Elixir Revolution Review by ColeDuclos on askubuntu.com
 
J F
second_part_lower = "" if len(message_parts) < 2 else message_parts[1].lower()
if command_aliases.get(second_part_lower):
    second_part_lower = command_aliases.get(second_part_lower)
@quartata ^
 
!!/alive
 
@Mithrandir Yup
 
@JF yeah I know, it's just what it's being used for here isn't obvious and it's a bad name
 
[ SmokeDetector | MS ] URL-only title, bad NS for domain in body, bad NS for domain in title, bad keyword in body, bad keyword in title, +3 more: wheretoobuy.com/active-x-garcinia/ by qkamnzghadzk on askubuntu.com
 
5:05 PM
then again I'm about to use alias_used which isn't much better
 
J F
subcommand_name?
 
!!/location
 
@Mithrandir Henders/EC2
 
!!/reboot
!!/wut
 
5:10 PM
@SmokeDetector v
 
@Mithrandir User removed from blacklist (766849 on superuser.com).
 
5:23 PM
 
sd f
 
@SmokeDetector fp-
ugh, MS seems to be down again
 
@Undo Just out of curiosity, does MS know what to do if I give it falseu- as a feedback?
like
 
teward/Solar Flare received failover signal.
 
5:36 PM
as far as I can tell there's no preprocessing of the feedback used on the SmokeDetector side
 
!!/loc
 
I just give it whatever command was used, whether that's fp or false or falseu
!!/location/!!
 
Restart: API quota is 19882.
@quartata Henders/EC2
 
oh god I wish I had never ever ever looked at this code
why has this been working
ok I guess I gotta roll with it, no way I'm changing this
 
6:25 PM
[ SmokeDetector | MS ] Username similar to website in answer: Cordova + Angular 4 + Typescript boilerplate template by Bhushan Mithabavkar on stackoverflow.com
[ SmokeDetector | MS ] Username similar to website in answer: Cordova + Angular 4 + Typescript boilerplate template by Bhushan Mithabavkar on stackoverflow.com
 
!!/location
 
@Mithrandir Henders/EC2
@Mithrandir teward/Solar Flare
 
!!/standby a
 
Hmm interesting
MS decided to failover even thought Henders was up
I guess when MS came back up Henders didn't status ping in time
 
6:30 PM
@SmokeDetector n
@SmokeDetector n
 
teward/Lunar Eclipse received failover signal.
 
!!/standby tew
 
Wuh?
!!/location
 
@quartata Henders/EC2
 
6:35 PM
!!/reboot
 
[ SmokeDetector ] SmokeDetector started at rev 11c52ba (tripleee: Watch q\W?soft\W?ltd --autopull) (running on Henders/EC2)
Restart: API quota is 15148.
 
mkay try that again
 
NobodyNada/Raspberry Pi received failover signal.
Restart: API quota is 19017.
 
6:59 PM
Ok what
!!/location
 
@quartata Henders/EC2
@quartata NobodyNada/Raspberry Pi
 
!!/standby No
 
yo @Henders your instance is busted
It's not pinging Metasmoke
 
!!/location
 
6:59 PM
@Mithrandir Henders/EC2
 
@Mithrandir you have admin right?
 

« first day (1422 days earlier)      last day (2414 days later) »