« first day (1186 days earlier)      last day (1002 days later) » 

5:00 PM
@Undo where can I define CHQ-only?
for testing something :p
python nocrash.sh charcoal-hq-only
@ThomasWard hang on a minute
I'm about to post an Introduction to Charcoal/Smokey, and would appreciate if it wasn't interrupted until it's done :)
You guys ever post that doc on meta?
Welcome to Charcoal HQ. If you've just arrived, you're probably wondering just what is going on, and what all these funny messages are. The following few messages are a quick introduction to what happens here, and I'll add some relevant links at the end.
This is Charcoal HQ. This room is run by a group of people called Charcoal, who make a bunch of things that work with SE. Our main focus is on our bot, SmokeDetector. This bot scans all new and updated content across the network, running some pattern recognition to try to find spam.
If it finds a post it thinks is spam, it posts a report of the post in here. It's then down to the people in the room to review the post and work out if it's spam or not, flagging it as necessary. In effect, that makes this room a real-time spam review queue.
Those people can then feed back to SmokeDetector about whether it got the detection right or not. We call a correct detection a true positive, and the feedback for this is represented as k or tpu-. An incorrect detection is a false positive, represented by f or fp-. If you're looking at a message with one of these codes in it and wondering what it is, it's most likely feedback.
That feedback is then sent off to our web dashboard, metasmoke, where we keep a permanent record of it so that we can track how well we're doing. The ins and outs of metasmoke are another story all by themselves, but the TL;DR is that it gives us a platform to track statistics and record things permanently and centrally, which SmokeDetector can't do.
Anything else anyone wants to add to that?
5:03 PM
just next time freeze the room so your mini tirades don't get interrupted :P
I'd remove the reference to $deity. It might offend or annoy people.
I was wondering whom you were greeting, then I read back to your message above that.
Apart from that, good job!
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, pattern-matching website in body, repeated URL at end of long post: Besides, it soaks your muscle by Aimigill on graphicdesign.SE
tpu- by Glorfindel
[ SmokeDetector ] SmokeDetector started at rev fa4828b (ArtOfCode: New check for that postgraduate spammer) (running on teward/Overlord)
Restart: API quota is 9975.
cool that actually worked
5:05 PM
@ThomasWard nah, just throw us a gist link
@CaffeineAddiction Working on the draft currently. See the star board if you want to throw in input
:35457789 yeah, that's kinda not something we need in production Smokey though :P
@ArtOfCode no, but it's not going to be
good good :)
@ArtOfCode i was actually testing a unicode pass to the communication
because i was wondering whether it's properly handled :P
5:06 PM
Actually, all I really want for that post is a graph of the time to deletion then I'm a happy camper :)

Intro to CHQ and Smokey

3 mins ago, 20 seconds total – 5 messages, 1 user, 3 stars

Bookmarked 5 secs ago by ArtOfCode

uhm should I be concerned about bodyfetcherMaxIds.txt not being readable?
is it just a binary file?
sudo chmod -R 755 bodyfetcherMaxIds.txt?
binary files shouldn't be named .txt.
@ArtOfCode E: Not Decodeable in UTF-8
5:07 PM
I sincerely doubt it's a binary file. It might be a pickle file, though.
oh, yeah, okay
it is a binary file
yep it's a pickle file
@ArtOfCode it's a pickle file, but it's being stored as text. Should be .pickle
5:09 PM
pickle isnt binary
datahandling 65, @ThomasWard
AND in gitignore
@ThomasWard that would be .p
same for queue
Yet another proprietary serialization format...
5:10 PM
I'll poke it, @ArtOfCode
these should all be .pickle files
Heh. The scammers decided to actually call me up for once instead of emailing.
"I'm calling from $PhoneProvider to thank you for being a loyal customer and to conduct a review of the account."
@ArtOfCode We're looking at a little bit of a refactor at this rate - if everything is a Pickle file then most of the .txt -> .pickle
anyone still with .txt files will have to rename...
It took one sentence to set the alarm bells ringing
5:13 PM
is that a ... six-character file extension?
@ThomasWard create a migration script to do any necessary moving stuff around while you're at it
@JanDvorak no, because pickle files should be .p not .pickle
@ArtOfCode I intend to.
@ThomasWard you intend to...
but first I have to figure out what needs changed :P
i.e. determine what files are pickle files, change them in code, and use the diff as a migration guide
for creating a migration script
we can even launch it in nocrash.py automatically.
at startup
5:34 PM
that dreaded feeling when you fail to properly ID string indexes in textvar[:-4] and instead write textvar[-4:] which removes the filename. Oops.
[ SmokeDetector | MS ] Manually reported answer: Detecting the presence of wifi devices by Randy Bum Gardener on security.SE
just got to make sure this thing runs...
unless I blew it up.
[ SmokeDetector ] SmokeDetector started at rev 3f9d440 (Thomas Ward: Add pickle migration script for .txt to .p) (running on teward/Overlord)
@SmokeDetector rude
5:37 PM
Is that a response recognized by Smokey?
[ SmokeDetector ] SmokeDetector started at rev fa4828b (ArtOfCode: New check for that postgraduate spammer) (running on teward/Aurora)
whoopsies I accidentally'd Smokey :/
my bad.
Restart: API quota is 1785.
@Riker I was feedbacking since I'm on my phone and can't fdsc
5:38 PM
@JanDvorak think so
MS says Mithrandir TPUd
@Mithrandir Looks like it is :)
rude -> tpu-
@ArtOfCode I think i have a migration script, it goes through all .txt files that're actually pickles, runs a check to make sure it exists, and if it does, renames. Then it'd execute as normal with .p. We can always run it in nocrash.py
it's just an import statement :P
actually it's small enough I could hardcode it into nocrash.py xD
5:43 PM
"that're"? I've never seen that contraction before.
== English == === Etymology === that +‎ 're === Pronunciation === (US) IPA(key): /ðətɚ/ === Contraction === that're (neologism) Contraction of that are. (Can we add an example for this sense?)...
[ SmokeDetector ] SmokeDetector started at rev a94a1b8 (Thomas Ward: indentation) (running on teward/Overlord)
confirmed it works :)
oopsies tests failures o.O
5:58 PM
@ArtOfCode could we run the pickle fix in ws.py, or do we want to avoid that? Nocrash or ws.py?
one of thsoe two places should do it
one would not require a manual pull, the other would
@ThomasWard neither, make it manual
it only needs to be run once
waiting on PR CI
CI on de5df57 succeeded.
6:13 PM
We've been kicking spammer ass all day though
Restart: API quota is 1538.
welcome back Smokey, and welcome to the real world of Pickles
@Undo your RPi will need to have Smokey stopped, a git pull done, execute fix_pickles.py, and then start Smokey again.
(pickle files aren't .txt, so we gave them the right extensions now)
API quota rolled over with 1510 requests remaining. Current quota: 9999.
stackoverflow: 1585
askubuntu: 497
superuser: 437
unix: 239
drupal: 212
electronics: 185
math: 175
english: 163
travel: 163
mathoverflow.net: 143
salesforce: 142
dba: 141
gis: 132
magento: 126
wordpress: 118
meta: 118
webapps: 114
apple: 109
sharepoint: 109
gaming: 99
physics: 98
academia: 92
es.stackoverflow: 89
worldbuilding: 89
security: 85
rpg: 85
stats: 80
ru.stackoverflow: 79
ell: 78
money: 75
scifi: 75
workplace: 74
movies: 74
chemistry: 73
tex: 71
mathematica: 66
outdoors: 59
history: 57
[ SmokeDetector | MS ] Link at end of answer: Drawing a String on C# webpage by Ritoot on stackoverflow.com (@dorukayhan)
tpu- by tripleee
96 autoflagged posts in a day, new record!
274 Autoflags cast in a day, a new record!
6:25 PM
the posts-vs-flags graph is amusing
@Undo I find this whole announcement tag a kind of a joke so if you see me suggest it anywhere you can assume I'm trolling. I would definitely add [tag:announcement] to this room tags ...
So, I made my point
@Magisch well yeah duh
I only see 271 in the flag log though ...?
@rene You're a flower, rene
... and only one of them is mine
two actually, the previous one had apparently scrolled off the first page
well today has used almost 3 pages, so...
6:30 PM
I got 20 flags today
In 274 autoflags today and 24 participants, I should have gotten an average of 11. I guess the lower weight makes it.
@Magisch I have min weight 265 and got only 6 autoflags
so now I know where the other statistical 5 went :P
@ArtOfCode so, apparently I like CI, 'cause I borrowed our Travis and Circle CI configs and adjusted them for my repos' needs. And now I spammed myself with "BROKEN!" and "STILL BROKEN!" messages because flake8 evils
how's 76 emails sound to you :p
that's a few
[ SmokeDetector | MS ] Bad keyword in link text in body: raspberry pi live to youtube issue by jeffh3c on raspberrypi.SE
@ArtOfCode yes, it is
6:38 PM
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title: What do we call to a person who is on a weight loose plan? by Dipika on english.SE
38 build fails, two CI systems
@rene I probably tend to agree, but I'm giving it a chance before passing judgment. Maybe it'll be a useful tool.
@ThomasWard done. Being able to SSH directly to it through IPv6 is awesome.
fp- by Glorfindel
6:40 PM
@Undo indeed.
@AshishAhuja your autoflag has been cast here, but it's an FP. You might want to retract it, but a moderator has already put the question on hold so it might also be declined already.
that's odd
Could probably do another bump to ~99.85%, although I'm not sure it's worth it
why only one flag?
@ArtOfCode EL&U has a spam threshold of 3, so an autoflag setting of max. 1
6:41 PM
@ArtOfCode ell.SE
Methinks we should make autoflag exceptions if the only two reasons triggered on a post are by the same phrase
So that the same bad keyword in body and title doesn't hit autoflag.
@Magisch too complicated for the times that occurs (IMHO)
yeah, parsing why data is REALLY HARD
I tried once.
@Undo thank you though for quickly kicking your RPi :)
6:42 PM
Not sure if irony
aye :)
@Undo why? :P
@Glorfindel Because you think that it's in a nice format, but it actually isn't. Some filters remove certain characters before running, and it throws your indexes off. Some filters remove UTF evilness, some don't
basically we're inconsistent
@Undo Idea: don't parse why data, parse the part of it thats written into metasmoke
not sure what you're saying there
@ArtOfCode Should probably group successful flags on the same post into one row
6:48 PM
that'd be sensible
Not sure how to do it nicely, though. Comma-separated would get long.
pickle it! just kidding
oh cool, apparently Travis and Circle autosubbed me to Smokey build notices o.O yay spam time
@Glorfindel Note to me for later: Include this FP in the meta post
It'd be super nice to have flag retraction in the API
@Andy Around the same place, might be good to mention that we'll retract the 0.01% of bad flags once meta.stackexchange.com/questions/288120/… is implemented.
good idea
7:04 PM
@Undo something to ask pops when he pops around next time
[ SmokeDetector | MS ] Few unique characters in answer: Effect Of Electronic Devices In Close Proximity by DEAN on electronics.SE
naa- by Magisch
7:29 PM
tpu- by Glorfindel
7:49 PM
[ SmokeDetector | MS ] Pattern-matching website in body: Wordpress Widget problems to my site by Seomarketergr on wordpress.SE
Hello! I'm sorry I had to leave suddenly yesterday
You just dont love us
[ SmokeDetector | MS ] Bad keyword in answer, blacklisted website in answer: I can not stop the mackeeper's installing by ninaalisa on apple.SE
tpu- by Glorfindel
@SmokeDetector f
8:01 PM
pretty certain that's website spam plugging - see the user's username
and the domain in the link
but not casting feedback today
woot, new server just came in ... 48 cores, 192 gb of ram ... and a terabyte of SSD
im going to be busy for a while
@NobodyNada I visited that site, everything works and it is riddled with ads for SEO in russian
@ThomasWard if its spam its stunningly incompetent
Hanlons razor says this guy is just not that bright
The guy's asking about what appears to be an actual problem, and he the question got an answer
"how do I build a spam website" isn't necessarily spam
@Magisch well, 'spam' is just advertising one's own site without disclosing affiliation and then plunking your own site in your username and in a link
8:03 PM
@NobodyNada nice avatar, very handsome cat
@Riker Thank you!
i don't see any text widgets that they mention
but this is why i'm not voting
because i'm not 100% sure it's true spam
leave it to the WP people.
@NobodyNada yeah, OK. Problem is he isn't asking to build a spam website, he already did ;)
8:05 PM
that's why I noticed
@ThomasWard OK
@Riker Your cat looks a lot like our other cat
@Riker NobodyNada is your cat, duh
@NobodyNada quit spelling ''kat'' wrongly, cat
@ThomasWard helpful Not a 100% spam post but helpful none-the-less! so @NobodyNada was right.
@M.A.R. lol
@NobodyNada your other cat must also be pretty handsome
I really like cats, my family has 3
8:07 PM
@M.A.R. actually, kat is the Dutch spelling.
@NobodyNada after checking your profile, it makes a bit more sense. :P I am a homeschooled christian teenager from southern california, apparently they tend to have cats
@Riker Wow, neat!
8:11 PM
small kitty is small
just make sure she doesn't carve up my leg
cats like to do that to me
@rene well maybe not until it was edited
all the links, etc. were removed
Note to Self, @Riker and @NobodyNada are both FEDs
Note to self, assimilate @CaffeineAddiction into the hive mind.
@ThomasWard They're outside-only cats, but we bring them into the garage at night. When they want to get our attention (maybe we've forgotten to feed them or it's morning and we haven't let them out yet), they scratch at a board by the door.
8:14 PM
Wait you guys aren't Borg? — AncientSwordRage ♦ Jun 21 '12 at 22:58
well i knew this one cat who was kind enough to gouge my leg pretty badly.
I say 'knew' because that cat died
@Mithrandir lol
@ThomasWard as a result?
@CaffeineAddiction the old axiom "If it bites me and it dies" applies, usually, in these cases, but in this case no
it was outside, wind was blowing fiercely...
... very very large tree branch on a dead tree got snapped of...
... heavy wood projectiles do damage, so...
... yeah...
kitty pancakes
"If it bites me and it dies" ... I would be out a few siblings
8:17 PM
@ThomasWard If you touched my cat I would end you so help me my toxoplasma gondii addled mind
I didn't say I'd touch the cat.
it is in fact random gravitational anomalies and fluctuations that would do it :P
i'm also severely allergic to cats so i stay away from them.
If my cat died I would end anyone in the vicinity
Dogs, on the other hand, most of them like me.
A few don't but eh
My cat is my treasure
Does the big one here like you? :P
8:19 PM
@Glorfindel Dutch no English
@Mithrandir you mean the huge hell hound in the corner waiting to eat spammers, or some other dog?
'cause that hell hound is mine :P
The one that Undoes everything...
ah that one.
he's lurking somewhere why don't you ask him :P
@Andy add a period at the end of the section with the graphs
8:26 PM
@Andy I was reminded of the famous ''shortest telegram evar''
Good grief, this wiki page was vandalized in November and it hasn't been reverted until just now
Undo is one of these small dogs
*waits for two more Ask Ubuntu spam posts*
waits for fifty more
waits for the flood gates to open
8:33 PM
The closest I am to a Marshall is on Science Fiction & Fantasy, with 249...
Do you guys welcome copy/pastes of rude posts that don't seem to have triggered anything here from Smoke Detector?
@Nathaniel Just !!/report it
but if it's deleted he can't do that
@Nathaniel If it was something we should've caught.
maybe a link to a screenshot would be better?
8:39 PM
Yeah it's deleted; seems like it should have been. Let me see...
@Nathaniel yes we do, if it should have been caught
there are some odd cases like Unicode fubaring detections, though...
@ThomasWard And a mod edited out the spam link :/ Maybe someone should have a word in his ear ...
@Nathaniel That should've been triple caught or something
@DavidPostill But we didn't agree on whether the post was spam, so editing the link out preserves the answer without damaging anything IMO
8:43 PM
@Nathaniel can you get a raw of that and run it here with !!/test-a
please :)
since it's deleted I can't :P
@M.A.R. That's what I figured, but I didn't see it in chat.
@ThomasWard Copy/paste of the text:
@Nathaniel any chance it was when Smokey was down when we bounced it briefly to fix the Pickle file problem?
christianity is stupid and retarded, learn science you cuck fucks.
@ThomasWard Nothing to test
8:44 PM
!!/test-a christianity is stupid and retarded, learn science you cuck fucks.
> Offensive answer detected
Post - Offensive keyword: *fucks*
yes it should have.
Why didn't it?
It was down, I guess. *shrug*
not sure
@Nathaniel can you get me a link to what it was? The /a/ link, etc. trying to check MS
looks like
maybe it was in the short time we had it down (3 minutes) to sync up the pickle extension fixes I pushed in
UTC 19:48
that was about an hour ago, which is when I shoved fixes in.
and when i accidentally'd smokey briefly.
Or, it just briefly desynced from the net or was handled before SMokey picked it up
When a post is handled before it's in Smokey's queue it's not going to show up
Which happens once in a while
@CaffeineAddiction lol
@ThomasWard Okay, thanks for checking. Is there an idiot-proof set of keywords somewhere that I could take a look at to know how I can interact with Smoke Detector (given that I don't have any special permissions to my knowledge)?
The gist has been updated with the second false positive (and a screenshot to it). It also links to the retract flags via the api feature request. I think the biggest thing I'm looking for at this point is a graph of deletion times over the last month or so. That might be more useful in a few days though so we can better show the effects of 3 flags vs 1
8:55 PM
@Nathaniel ''k'' is for stuff that's spam or rude, ''n'' is for stuff that's NAA, ''f'' is for legit stuff, and ''v'' is for post vandalism.
@Nathaniel You're blue. AFAIK, since you have a diamond, you have the same privileges that I do.
Feel free to take a look and proof read my grammar :)
@Nathaniel run !!/amiprivileged
@Nathaniel Yes, you are a privileged user.
8:56 PM
All SE moderators are privileged by default
@Andy "which was off topic anyway" should be "which were off topic anyway," now that there are two of them
Grammar Nazis HQ
Do we need the phrase? Or is it better to just let the screenshots speak for themselves?
@Nathaniel So check out the list of commands for users that do and don't have privs
8:58 PM
updated anyway
Where grammatically correct sentences are bred, and meta posts are made
@Mithrandir Ah, there we go – perfect, thanks
@Andy I would've done the latter, with a ''well . . . These are the false positives''
> In that time, we've had a two false positives
9:00 PM
@ThomasWard Thanks!
@Andy LOL. I was involved in both fps (my autoflag the first, and I edited the post in the second) ;p
@ThomasWard You were ninja'd.
> Due to these false positives, we've disabled access to the internet for one user who seems to be involved in all of them
don't care :P
9:02 PM
@Andy rolflmao
@Andy I should be able to get that in metasmoke today or tomorrow, whenever I can find the intersection of downtime and a decent connection
fp- by NobodyNada
@SmokeDetector fp-
Restart: API quota is 8846.
why did you restart
!!/errorlogs 50
9:13 PM
  File "/usr/local/lib/python2.7/dist-packages/websocket/_abnf.py", line 371, in recv_strict
    bytes_ = self.recv(min(16384, shortage))

  File "/usr/local/lib/python2.7/dist-packages/websocket/_core.py", line 427, in _recv
    return recv(self.sock, bufsize)

  File "/usr/local/lib/python2.7/dist-packages/websocket/_socket.py", line 80, in recv
    bytes_ = sock.recv(bufsize)

HTTPError: 500 Server Error: Internal Server Error for url: chat.stackexchange.com/error?aspxerrorpath=/messages/35462963/…
HTTP 500 error o.O
oh i think my date time is also out of skew :/
Must stay out of SOBotics or I'll run out of SO flags
I only have 2 SO flags left today o_O
Morning everyone
9:19 PM
@angussidney Night
o/ morning afternoon
[ SmokeDetector | MS ] Bad keyword in link text in answer: Is it beneficial to publish papers from previous labs during grad school? by OliveRon on academia.SE (@Wrzlprmft)
tpu- by DavidPostill
fp- by Mithrandir
SD - F
@Mithrandir Noobs usually run out of flags. Pros usually don't. VERY pros usually do. Which are you? :)
9:36 PM
@M.A.R. I've raised 20 flags on SO in the past 12 hours, and I don't even use the site. I'll let you decide.
[ SmokeDetector | MS ] Manually reported answer: Eclipse doesn't find javax.swing by Bushdid911 on stackoverflow.com (@dorukayhan)
tpu- by Floern
[ SmokeDetector | MS ] Manually reported answer: Eclipse doesn't find javax.swing by Bushdid911 on stackoverflow.com (@dorukayhan)
tpu- by Floern
[ SmokeDetector | MS ] Manually reported answer: Eclipse doesn't find javax.swing by Bushdid911 on stackoverflow.com (@dorukayhan)
tpu- by NobodyNada
yay free flags!
@NobodyNada User is blacklisted (7571400 on stackoverflow.com).
Anyone have input on this?
10 hours ago, by J F
What do you think about adding InstantClick (or something similar) to the https://charcoal-se.org website?
that's really cool!
9:51 PM
!!/notify 1334 christianity.stackexchange.com-
I’m wondering — is there any way to use actual templates for charcoal-se.org instead of copy-pasting every time something changes?
!!/notify 38408 latin.stackexchange.com

« first day (1186 days earlier)      last day (1002 days later) »