[ SmokeDetector | MS ] Few unique characters in body, no whitespace in body, repeating characters in body, repeating characters in title, title has only one unique char (420): aaaaaaaaaaaaaaaa ✏️ by Sesame on quant.SE
> Potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username ---------- Title - Position 0-12: newbecca.com Body - Position 0-12: newbecca.com Username - Position 0-12: newbecca.com
That is, if a post (a) has at least $threshold (start at 1, see how it goes) manual flags on it, and (b) has at least $otherThreshold (start at 1) FP feedbacks on it, and (c) gets deleted, cast a mod flag on it
That flag probably says something like "This post was flagged N times through Charcoal systems and has since been deleted, but was later flagged up as a potential false positive. Please review this post and make sure deletion was appropriate for it; let us know if it wasn't. If you're wondering WTF this flag is, see <some link to website>".
@ArtOfCode Not necessarily. If I moved it to lambdas it'd be inconvenient at best
It'd be kinda fun to make a trapdoor system - tokens go in, tokens don't come out. Then the system just knows a few commands - flag this as this, invalidate token, etc.
@ArtOfCode This will still raise such a flag in situations where we internally reverse direction. For example, where there are some initial tpu- feedbacks and manual flags recorded, but where we decide, after discussion, that it's actually FP or NAA and people retract their flags, but that it still gets deleted as either red-flagged or NAA. Basically, MS has no visibility into users retracting their flags. However, it's not unreasonable to kick it to a moderator, even under those circumstances.
My expectation is that the total number of situations is low.
Another alternative would be to raise a report that an MS admin would need to handle, rather than sending it to site moderators.
If raising an info flag for site moderators, we should consider if we want to including both a links to the MS post and a link to the SD report in CHQ, where there may be more discussion than is recorded in MS comments.
@ArtOfCode I have not actually tried flagging, but the API returns a valid option_id for the custom flag on deleted posts for posts on sites both where I have 10k+ and where I don't. Thus, I expect that the SE API will permit us to send custom flags on deleted posts.
> Potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username ---------- Title - Position 0-11: bekausa.com Body - Position 0-11: bekausa.com Username - Position 0-11: bekausa.com
@Makyen I wonder if there's a weird distribution of stuff around that borderline. Perhaps the 'borderline' stuff we see gets caught for some statistically significant reason.
So our average invalidated feedback rate is around 5%, accounting for who's here when. Using a SW-AG estimate, I'd guess that somewhere around 5 people see each Smokey report. That's already 3.12e-7% chance of something being missed, without accounting for review or admin invalidations.
@ArtOfCode I suspect this is much lower than 5 during some times of day. Most people don't go back through the chat history. For many/most people, SD reports are hidden upon post deletion. For a significant period of time, we required only 1 feedback, which means things didn't, necessarily get reviewed.
@ArtOfCode That might be reasonable, although I'm not sure that I'd limit it to just FP/NAA when deleted (i.e. should probably set the flag even if we get that feedback after deletion).
I assume it disabled everyone's flag conditions that had one with weight set it slightly below the 280 default (i.e. at or below the 257 on that post). I'd be good to find out if it disabled those who only had conditions above 257.
@Undo can you pull prod logs for ActionCable broadcasts of that "flag conditions were manually disabled" message in the last hour? I want a complete list of everyone who had conditions disabled so I can go back through and re-enable them
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, link at end of body, pattern-matching product name in body, +4 more (692): What is Keto Ultra New Zealand? by user963482 on superuser.com
[ SmokeDetector | MS ] Url-only title, blacklisted website in body, blacklisted website in title, body starts with title and ends in url, link at end of body, +5 more (794): evaherbalist.com/beligra/ by Suwbpafy on askubuntu.com
You have repeatedly posted spam for your domain. I removed the spammy link from your post; please refrain from gratuitously linking to it. See also How not to be a spammer. — tripleee23 secs ago
@TetsuyaYamamoto That pattern looks like it's already caught by Potentially problematic ns configuration in answer and Potentially problematic ns configuration in body; append -force if you really want to do that.
@TetsuyaYamamoto That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer; append -force if you really want to do that.
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, pattern-matching product name in title, +2 more (592): KetoFit Canada KetoFit Canada by qlaanskad on apple.SE
@Glorfindel I don't use Reddit either and I guess it would depend on the rules of the specific subreddit. I think that post looks like the sort of thing you'd find on a really lax programming messageboard.
They're not blatantly promoting anything (no link) or harassing anyone -- just being chatty
Yeah, I can certainly see why one thinks that's a false positive. Even more when you consider it has been caught 'by accident' (holiday.it is not a website link here).
If we had a 'gray area' feedback type I would've probably used it.
[ SmokeDetector | MS ] Url in title, blacklisted website in body, blacklisted website in title, body starts with title and ends in url, link at end of body, +5 more (791): evaherbalist.com/beligra/ by Qmhvgmvg on drupal.SE
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, body starts with title and ends in url, pattern-matching product name in body, pattern-matching product name in title, +1 more (589): V10 Plus Male Enhancement by LonnaFinke on wordpress.SE
HeatDetector is an example of a project with a different goal: it wants to catch problems, not just arguments specifically. Therefore, anything problematic is a tp, even if a comment is pointing out spam or plagiarism, or a different comment than the one caught is argumentative.
But since our project is looking for something specific, our feedback should reflect that specific metric. For example, I wouldn't give Natty a tp because it caught a plagiarized post; that's outside of the scope of the project.
Similarly, IMO a SmokeDetector tp is only for when something is definitively spam or R/A, since blurring the lines dilutes the potency of a tp. Finding an egregious question may be helpful, but it's outside the scope of the project because we're spam-hunters, not problem-hunters.
Vandalism is a bit of a special case: we don't try to catch it, but if we find it we give it a tp. This is because Smokey cares about content, not posts (so it doesn't care what a post used to look like). Giving it an fp would act as if we told the system "this content isn't spam" rather than "this post isn't spam."
NAA is another special case: it means "this answer shouldn't have been caught, but it is egregiously bad." If we had NAA for questions, the post under discussion would have met that criteria. (However, an NAA-for-questions feedback would be really subjective and varying site policies would be difficult to keep track of.)