Charcoal HQ

12:55 AM

@ProgramFOX nah, I don't use setup.sh on a regular basis. It's already set up ;)

hichris123

1:25 AM

translate es: test

prueba

4 hours later…

5:35 AM

[ SmokeDetector ] All-caps title: HOW TO USE ISBLANK IN NESTED IF STATEMENT on stackoverflow.com

1 hour later…

6:39 AM

[ SmokeDetector ] All-caps title: HOW TO COMPARE 2 FIELDS IN VIEW USING PHP on drupal.stackexchange.com

[ SmokeDetector ] All-caps title: HOW TO COMPARE 2 FIELDS IN VIEW USING PHP (DRUPAL 7) on stackoverflow.com

2 hours later…

8:48 AM

[ SmokeDetector ] Phone number detected: The file reference 0x3000000003d66 on superuser.com

2 hours later…

10:47 AM

[ SmokeDetector ] All-caps title: UNIFORM CONVERGENCE AND CONTINUITY FOR A SERIES OF FUNCTIONS on math.stackexchange.com

11:18 AM

[ SmokeDetector ] All-caps title: PDO::FETCH VANTAGENS on pt.stackoverflow.com

3 hours later…

1:56 PM

[ SmokeDetector ] All-caps title: HOW TO GET THE AVERAGE OF THE ARRAY? on programmers.stackexchange.com

[ SmokeDetector ] All-caps title: HOW TO GET THE AVERAGE OF AN ARRAY? on programmers.stackexchange.com

[ SmokeDetector ] Bad keyword detected: Air système. bonne de nike tn on drupal.stackexchange.com

2:53 PM

[ SmokeDetector ] Offensive title detected: Add a “don't migrate crap” migration 'path' to all sites on meta.stackexchange.com

3:04 PM

in Tavern on the Meta on Meta Stack Exchange Chat, 4 mins ago, by Ben Collins

@Unihedron why not use the API for this? it's a good example of the kind of thing it would be good for

@hichris123 @Undo @ManishEarth ^

faster than me - I was typing that

lol

@BenCollins Primarily because we don't want to be making thousands of API requests a day.

@Undo vs listening on a websocket?

@BenCollins right

Websockets don't have rate limits.

3:06 PM

the api's rate limit is plenty high for this use case. 30 requests per second for unregistered apps, more for registered

I can't imagine smoke detector would need more than 1 request per second

anyway - just a suggestion - the api gives you a bit more info about the questions than you get from ?tab=realtime

hm?

@BenCollins I think it's 4 new questions every 3 seconds

plus we can batch

sounds doable

@ManishEarth batching is what I had in mind

ah right

forgot about batching

If the API also contains answers it can finally detect spam answers

@ShadowWizard We might be slower on that though

3:09 PM

I sometimes find spam answers weeks old

@ShadowWizard It does contain answers

How about this as a spam catching scheme? (@Undo @hichris123)

- If findspam.py sniffs a bad title, post it immediately

@ShadowWizard That's different, though, I think. We'd have to batch 100-answer-requests every few minutes instead of us getting them pushed to us. But yeah, we could do it.

@ManishEarth take your time, spam won't be over any time soon :)

- Otherwise, batch up the last 10-20 questions, and API request, and then run a better findspam algorithm on the body

We can do the same for answers. Every 2 minutes, fetch new answers from all sites (unfortunately, not batchable)

3:11 PM

@ManishEarth Last 10-20 over all exchanges, right?

@Andy yep

@Undo unfortunately, exam time, and got work on Servo too. Might not be able to work on this myself :/

That'll burn through your API limit quickly though, especially if you have to call answers individually

@ManishEarth I can probably try to squeeze in some work on it

@Andy d'oh, right

@Undo we can't batch across sites

Oh riiiight

3:13 PM

:17562789 Right, but it's still 100+ calls PER answer cycle, because you have to call the API for each site

might be harder

Yeah, this is why we need that answer websocket.

or just pray to the throttle gods :P

api.stackexchange.com/docs/…

@Andy you don't have to get answers individually - they can be either "batched" by just getting all of them or vectorized by IDs

3:17 PM

@BenCollins Right, but if I answer to check answers from SO, SU, SF, Moderators, Chemistry, Meta, that is 6 calls.

@BenCollins I think he means site individually. There isn't an 'all answers across all sites' method, right?

1 per site

yes, I don't think there's a way to batch across sites

Regarding that API link above, you can batch it by being more exact on the to/from date time stamps. We've have to track those per call

not an obvious one, anyway

3:18 PM

And thus we can't pull it off, I don't think. Not without either a network-wide answers method or a socket.

@Undo solution: 1. get hired by SE 2. write SmokeDetector directly on API. 3. Profit!

(or, y'know, fight spam with statistical ML)

We could probably bind to the realtime update websockets for each site, check each one to see if it's a new answer, then batch those per site, but we're still talking way more requests than there should be

@Undo I don't think can't is quite the right word. more like: hard to manage the requests

@ManishEarth we do that now, somewhat

@BenCollins I think we could manage the requests. It's that we could only do it like a hundred times per day (100ish sites and 10,000 requests per day, right?)

that's what I meant: managing the # of requests to stay below the limit

3:21 PM

right

@BenCollins oooo

@ManishEarth podcast.

@Undo too busy to listen to :p

lol

after this week I'll probably be back to usual activity levels on SE, though I'm not sure if I will be able to do SmokeDetector/Charcoal dev much

3:22 PM

our spam stuff necessarily has to be more forgiving than yours though, which is why you still see spam occasionally get through

@BenCollins yeah

@BenCollins There probably wouldn't be any way to get SD implemented server-side, right?

@BenCollins Do you just have spam/not spam options? Could you have a "maybe spam" detector that throws stuff into a review queue?

@Undo it pretty much is already

@Undo we'd lose tweakability though

@BenCollins with reporting. Kill obvious spam, report the rest to a feed or chatroom so bored Internet people can solve it :P

3:23 PM

Or heck, even if you could post things that just barely miss being categorized as spam into chat.

yeah

@Andy LQ flags?

those exist

oy, blue username, you should know :P

@ManishEarth Yes, but there is a difference between low quality and actually triggering a spam detector

@Andy spam is kinda binary - there are some autoflagging triggers, but they can't be too harsh or you wind up spiking a lot of legit questions

Especially some of the more obvious Baba stuff

@BenCollins Fair enough

@Andy we're working on some LQ-detection that will be more nuanced though, including setting up a "triage" queue for stuff that gets detected

3:26 PM

sounds awesome

ooh

ProgramFOX

@BenCollins Nice!

we'll see. I'm shooting for "better than what we have now", and if it's awesome, then bonus

LQ-detection is a good bit harder than I thought it would be when we started

> Baba stuff

?

baba ji. molvii. love magic.

That crap

Q: Add a realtime answers tab to stackexchange.com

3:31 PM

We already have an awesome realtime questions tab for every question that comes in over the whole network. But in the spirit of Optimizing For Pearls, Not Sand, shouldn't we also have a realtime answers tab? It'd be really neat to see the answers coming through for each question in the network,...

There

Threw that out there.

@BenCollins vashikaran/baba/moulvi/etc . One persistent spammer that just posts phone numbers and black magic tips

oh, right. Yeah that's the easy kind. Harder is stuff that to a machine sounds like normal prose and doesn't have any of the obvious trigger words, like "come see our video of a live soccer match"

yep

3:52 PM

"we would then use for good". Heh.

2 hours later…