Smokey Stuff

Jason C

Mar 20, 2017 07:05

Do you guys have any sense of how many posts get deleted as spam or offensive that smokey doesn't catch?

Aside from when it hits quota limits.

angussidney

@JasonC we don't have any official stats, but I guess that we detect ~>95% of spam/offensive posts.

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, repeated URL at end of long post: It prevents the brand new wrinkles from forming by eandra juri on drupal.SE

[ SmokeDetector | MS ] Pattern-matching website in body: Unable to block unnecessary popups in Chrome by learningnewthings on stackoverflow.com

angussidney

We asked Pops the other day if he can send us a dump of all the deleted spam posts that he can get access to, but he hasn't responded yet

SmokeDetector

[ SmokeDetector | MS ] URL in title, bad keyword in body, pattern-matching product name in body, pattern-matching website in body, pattern-matching website in title: megacleanseradvice.com by ralphleger on apple.SE

[ SmokeDetector | MS ] Pattern-matching website in body, blacklisted user: Which one is the best ways to improve a slow Mac performance? by Hyo Thomas on apple.SE

tpu- by angussidney on It prevents the brand new wrinkles from forming [MS]

tpu- by angussidney on http://megacleanseradvice.com [MS]

Glorfindel

@SmokeDetector k

Jason C

Mar 20, 2017 07:09

@angussidney I am pressing for the same, although only for SO. If my request ends up successful and I get it first I'll see if I can piggyback you guys onto it.

angussidney

2 days ago, by ArtOfCode

@Pops are you able to get hold of some data for us that we can't get? Ideally, we'd need site, post ID, and post creation date for every post that was deleted by red flags, for as far back as you can get it.

Jason C

You can get a list of such posts with a query like data.stackexchange.com/stackoverflow/query/643932/…;.

There's just no good way to retrieve the bodies.

angussidney

cc @ArtOfCode ^^

Jason C

Or at least a partial list; it excludes posts deleted by mods but not marked as spam/offensive, which there's really no way to sort out.

Glorfindel

@JasonC 10k users with nothing better to do :)

Jason C

Mar 20, 2017 07:12

Yeah, heh.

I mean if we distributed the scraping task among 10k users we could do it in a reasonable amount of time without getting hit by the throttling or bot protection or whatever it is.

If you try scraping those posts with your auth cookies you'll get logged out after about 20-30. I went all the way up to 10 seconds between requests with no luck.

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, pattern-matching website in body, repeated URL at end of long post: Hit the catch underneath to qualify yourself for a free trial offer. by bettyzozu on graphicdesign.SE

Glorfindel

@SmokeDetector k

@JasonC do you know if they're exposed via the SE API?

angussidney

@Glorfindel nope, they're not

Jason C

@angussidney Fwiw, btw, MS has about 15000 SO posts in its database, but the spam/offensive query returns closer to 27000 (24000 spam only iirc).

@Glorfindel They are not.

Glorfindel

sad :(

Jason C

Mar 20, 2017 07:15

9

Q: Obtaining text of spam/offensive posts

Is there any way for normal people to grab a list of all posts on a site deleted as spam/offensive (presuming I have privileges to view them)? All routes I've explored are dead ends: Archive.org data dump: Does not include deleted posts. SEDE: Drops body text from deleted spam/offensive posts....

angussidney

@JasonC the MS DB only goes back about 2 years, smokey has been around for 3-4 years, and SO has been around since 2009, so that's not surprising

Jason C

Yeah I was just about to add a "PS" to that comment.

Let's see... I think 2016-03-06 is around the time of the first SO entry on MS. The SEDE query only returns 9300 results with creation dates since then. So that's a great sign. Could be any number of reasons that the MS DB is larger, I didn't check. False positives, duplicate entries, dates that mean different things in MS vs the query, etc.

Posts deleted for reasons that don't show up in the votes table.

Hm, I think I'll compare the IDs in that query with the ones from MS and see if I discover anything fun.

angussidney

@JasonC 10900 posts that are TPs on MS

Glorfindel

@JasonC yeah, that'd be interesting.

tripleee

@SmokeDetector @angussidney maybe add endovex to bad_keywords.txt

angussidney

Mar 20, 2017 07:22

@JasonC and then we've got to consider vandalism posts, which we mark as tp-, but that requires some database-level queries that I don't have time to work out at the moment

Jason C

Ah yeah that'll cloud things.

SmokeDetector

CI on 7358a8b succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev 7358a8b (angussidney: +endovex to bad keywords) (running on ArtOfCode/EC2)

Restart: API quota is 17316.

Jason C

@angussidney Do you think "vandalism" is roughly equivalent to "is_tp and deleted_at = null" in the MS database?

angussidney

@JasonC we don't have deletion info for everything, I wouldn't rely on it

Glorfindel

If you get a list of post IDs which are present in metasmoke but missing in SEDE, you can make another SEDE query which counts the number of revisions (=1 is spam/offensive, >1 is probable vandalism)

angussidney

Mar 20, 2017 07:26

@JasonC search for tp- feedback rather than tpu-. And there is revision info for some posts in the DB somewhere too, which may be useful

Jason C

Oh, but, I can actually cross reference with an SEDE query to see if a post was actually deleted.

Oh ok.

tripleee

some vandalism posts do get deleted for various reasons; I wouldn't think this is a very good heuristic

Jason C

Yeah... but it might be the best possible without serious scraping and revision analysis.

tripleee

I would perhaps look at low SmokeDetector score + not all TPU feedback but that requires you to uncover the Smokey scores

Jason C

Oh good call.

tripleee

Mar 20, 2017 07:29

reviewing the unflagged log should give you an idea metasmoke.erwaysoftware.com/flagging/logs/unflagged

but it only goes a couple months back

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link at end of body, link following arrow in body, pattern-matching website in body: DermaBellix If you get cold sores by SteveTorres on drupal.SE

tpu- by Glorfindel

tripleee

the autoflagging score is a relatively new thing

Jason C

Crap. I talked about too many things in the last 10 minutes. Now I forget everything. I should probably just watch Netflix.

3

angussidney

@JasonC the transcript is here if you ever need it :)

Charcoal HQ

Participants