... what if autoflagging were set up to supply the last rather than the first close vote? then you'd have the additional feedback from humans so you could require 3+ TP or whatever
@tripleee basically, here's the workflow: for every item on the blacklist, do a search on MS for all last 100 posts including it, count the number of TPs/FPs/NAA and last TP date, then recommend deletion based on those factors
I'm not running it through any spam checks or anything
@angussidney what if you did a post at a time and ran SmokeDetector against each? That's what I assumed the code would be doing, so I didn't even look at the flow
@tripleee what do you mean by that? Monitor the post score, take a wild guess how many people have spam flagged and try and ninja someone to the last flag? [status-not-possible]
@angussidney am I? The thing I would use it for is to gauge each blacklist entry's usefulness by comparing its false positive and false negative rate as well as absolute numbers to quantify general usefulness
if you get the last n (100, 1000, or whatever) posts from Metasmoke and apply SmokeDetector to them, collect the result for each item on each blacklist, and do the calculation
@tripleee you're still not making sense to me. SmokeDetector itself, alongside all it's rules are irrelevant, we're just trying to find out the frequency/accuracy of each item on the blacklist in the MS DB
@angussidney but applying the items correctly requires a compatible interpreter
the more fixes you put in the code to correctly emulate how Python and our blacklist code works internally, the more vulnerable you are to further changes in that logic
if you make it work now, can you make it work again in six months? at what exprense?
also, the stuff which is hard-coded outside of the blacklists would also benefit from an audit
but if you want to keep your approach, go ahead, maybe I can try mine separately
personally, I would not want to spend time figuring out how to wrap MySQL's regex implementation's head around the Python regex features a small step at a time, but we are not using the complex features in the blacklists much
the built-in hard-coded rules are using a lot of things you basically cannot do in MySQL
negative lookaheads in particular you would basically just have to lose
[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, pattern-matching website in body, pattern-matching website in title, repeated URL at end of long post: testosteroneboosterbits.com/raw-power-xl/ by Tyro Cart on apple.stackexchange.com
[ SmokeDetector | MS ] URL in title, bad keyword in body, link at end of body, pattern-matching website in body, pattern-matching website in title, repeated URL at end of long post: supplementlab.org/stack-xtreme/ by vexadac on workplace.stackexchange.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, repeated URL at end of long post: Zyntix How Its Help? by hodgehekela on graphicdesign.stackexchange.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link following arrow in body, pattern-matching website in body: Alpha Prime Elite by alphaprimeelite on apple.stackexchange.com
@Magisch for me, it depends. This morning I found a potentially valuable answer, so I edited the spam links out. That user might one day become a valuable contributor to Stack Exchange.
The companies I just blacklisted: not so much. Those answers were just garbage.
but I acquired the email of a IT guy there through the chat function and sent them a record of it. The IT guy replied with (paraphrased from german, was a german company) that it was a contractor and that theY'd been "dealt with" and that it wouldn't happen again
someone on SU tested the creds and said they were real
I'm sure the IT department was on fire when they found out
[ SmokeDetector ] URL in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, pattern-matching website in body, pattern-matching website in title: www.supplementoffers.org/alpha-monster-advanced/ by user84702 on graphicdesign.stackexchange.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, pattern-matching website in body, repeated URL at end of long post: Skin Opulent accomplished at by faxizisonia on drupal.stackexchange.com
[ SmokeDetector | MS ] URL-only title, bad keyword in body, bad keyword in title, link at end of body, pattern-matching website in body, pattern-matching website in title: healthytipweb.com/illum-face-cream/ by hickswood on askubuntu.com (@ThomasWard)
[ SmokeDetector ] URL in title, bad keyword in body, bad keyword in title, bad keyword in username, blacklisted website in body, blacklisted website in title, link at end of body, pattern-matching website in body, pattern-matching website in title, repeated URL at end of long post: www.topwellnesspro.com/reaction-male-enhancement-reviews/ by ReactionMaleEnhancement on `security.stackexchan
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link at end of body, pattern-matching website in body, repeated URL at end of long post: Bella Gold Serum by rezul on askubuntu.com (@ThomasWard)
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, link following arrow in body: Nitric Muscle Uptake Don't by SantaRodri on money.stackexchange.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching website in body, repeated URL at end of long post: Skin Opulent dirt and grime by faxizi sonia on graphicdesign.stackexchange.com
tpu- by Glorfindel
[ SmokeDetector | MS ] Bad keyword in body, link at end of body, pattern-matching website in body, repeated URL at end of long post: We should discover more by habu on workplace.stackexchange.com
