I think logistic regression might be a quicky way to get better autoflagging scores, since the NN was too tricky to train because of our various idiosyncrasies. I'll try it tomorrow since this has come back on my radar
(I have tried several models)
but this dropped off my radar quick because once we got the ridiculous flagging conditions fix the FPs dropped off almost entirely
I do encourage you to keep an eye out and see how many you actually observe now that we've fixed that
I think we might conflating "sharp increase in autoflagging FPs" with "presence of double-dip," even though a lot of the autoflag FPs came from crap with like 50 weight that got hit by site-specific flagging conditions
What we could really do with is some definition of reason difference such that if |A+B| >= 0.7 * Max[|A|, |B|], the reasons involved get a score penalty
I'm looking into this
That's the most obvious and most accurate way to objectively identify double-dippers.
> Blacklisted username, offensive body detected, offensive title detected, toxic body detected ---------- Title - Offensive keyword: *fuk* Post - Offensive keyword: *fuk* Post - Perspective scored 0.9512472 Username - Position 1-4: fuk
@Makyen That pattern looks like it's already caught by Toxic body detected, Offensive answer detected, Toxic answer detected, and Offensive body detected; append -force if you really want to do that.
[ SmokeDetector ] SmokeDetector started at [rev dc2b432 (SmokeDetector: Auto watch of f+u+[ck]+(?<!fuck)(?<!fuk) by Makyen --autopull)](//github.com/Charcoal-SE/SmokeDetector/commit/dc2b432) (running on ArtOfCode/EC2)
In many cases, we may be aware of spammers that mods aren't aware of, so users may have a better time explaining their case here than to a mod (where they can't chat back and forth)
@gparyani Honestly, it feels like you're inserting Charcoal in the middle of something that's between the user and moderators. Perhaps it's just how it's worded, but I felt negative about it when reading it.
Like, for example, I've learned that it's better to pull some folks off to the side and get their opinions before just launching an idea out into the wild
[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, blacklisted website in body, link at end of body, pattern-matching website in body, +1 more: Then do now not ban starchy ingredients by gcdt on askubuntu.com (@Zoe)
@gparyani more specifically, the general policy is don't mention Charcoal unless you really need to, and if you do, ask for feedback on your post in here before you post it
Showerthought: the reason why it's so hard to train some sort of AI/NN/ML on our data is because it doesn't include context. The MS record only contains the post itself.... whereas us humans look at the question, the users profile, individual site norms and policies, search the internet for the user/URL/keyword, try and find affiliation etc
[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching website in body: Lose weight easily and quickly? by gcdt on apple.SE
@DavidPostill FYI: I've submitted a PR to tighten the "Virk" blacklisted username you added. The tightened version still catches all the recent "Virks" we have in MS which are TP, while not catching the two recent FP. Please let me know if there were instances not recorded in MS which don't fit the pattern I used.
@angussidney But we don't need to do that for a vast majority. I can accurately classify the MS corpus manually with no other resources to within a couple nines fairly easily
But I still think there's some missing context of some sort. If we got a person who's never used SE before and only gave them our past posts as reference (didn't even tell them what the SE definition of spam is), they'd probably make quite a few errors
Also the whole social aspect of it - 'this is technically spam, but I don't think it should be deleted because its a good answer'
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, link at end of body, pattern-matching website in body, repeated URL at end of long post: Personal bodyweight in your set by buakclanw on askubuntu.com (@Zoe)
[ SmokeDetector | MS ] Bad NS for domain in body, bad NS for domain in title, blacklisted website in body, blacklisted website in title, link at end of body, +3 more: healthyorder.org/pro-diet-plus/ by Frithat1990 on askubuntu.com (@Zoe)
@SurajRao That's the type of information that's helpful to have with a report. You can have that as the reason for your report by including it in double-quotes after the URL in your !!/report. Alternately, you can make an actual reply to the SD report. When you do so, the text will be added as a comment to the SD report.
@SurajRao To avoid SmokeDetector reporting posts too slowly, you can report at most 5 posts at a time. This is to avoid SmokeDetector's chat messages getting rate-limited too much, which would slow down reports.
@doppelgreener there was a PR about allowing us to specify reasons for flagging but I don't know if it's still open or if someone attempted to implement that
