@AshishAhuja @Mithrandir @angussidney @tripleee You've all commented about fiverr.com being blacklisted. fiverr.com's TP/FP rate is 66.88%, which is significantly below the overall rates we have for the blacklist (body 97%, answer 85%). Clearly, fiverr.com is a decent, but not great, indicator that the post is spam, but does the actual TP/FP rate we see on fiverr.com really justify having it on the blacklist? (try to ignore that it's one that hits a bit closer to home; i.e. advertising coding)
If fiverr.com should be on the blacklist with that TP/FP rate, should other things with similar TP/FP rates be on the blacklist? Or, are we intending to keep the blacklist as really black, meaning that if something is not the blacklist, it's almost certainly spam? Alternately, should we have a "graylist", which includes things in the, say, 50% to 85% TP/FP rate range?
@quartata Any ideas? Patching CE is the most "canonical" solution, yet not all that practical. Since we're only using MD source in very few places, manually fetching when needed is a decent "solution pro tempore", as I've implemented.
A quick grep -nF content_source *.py shows it only appears in the blacklist command processor, so the source fetcher is only used once here.
@TetsuyaYamamoto That pattern looks like it's already caught by Mostly non-latin body and Mostly non-latin answer; append -force if you really want to do that.
If they are in spam, then you should add them as watches, which indicates we've seen that number before and will detect it in something other than a title.
> Blacklisted website in body, blacklisted website in title ---------- Title - Position 7-25: i-digitalpixel.com Body - Position 7-25: i-digitalpixel.com
@quartata I now see it in there, but the watch should be only oneofonehack, because the post uses it as "oneofonehack At G mail dot com" and "oneofonehack dot com".
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, body starts with title and ends in url, link at end of body, pattern-matching product name in body, +3 more: time, so your wellbeing will by bzqwbzqw on superuser.com
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, link at end of body, pattern-matching product name in body, pattern-matching website in body, +2 more: Do not live seated for brought by buos on superuser.com
Keep in mind that some of those provide some desired context. While we don't need to include huge amounts in the "why" it is helpful to see enough to really know why something was detected (e.g. this one specifically has through the "https://", so that people know it's a link). OTOH it's also good to have things not overlap. For instance, on the watchlist, having the above detection actually include everything to the end of the text would have resulted in masking out other watchlist detections.
I'm planning on submitting a PR for the "Link folllowing arrow" detection, because its domain exclusion is currently broken, but if you're making other changes, I'm happy if you fix that too.
Well, the minimum change is that https?(?!://i.stack.imgur.com) needs to be https?://(?!i\.stack\.imgur\.com) for Link following arrow. Having a ? immediately preceding the negative look-ahead and no other requirements means that the negative look-ahead will never disqualify the match.
[ SmokeDetector | MS ] Blacklisted website in body, body starts with title and ends in url, link at end of body, pattern-matching website in body, potentially bad ns for domain in body, +1 more: Maximum Power XL by wnturift on superuser.com
Yes, it makes the s optional, but it's a negative look-ahead. Thus, https://i.stack.imgur.com will still match, because it starts the negative look-ahead at http and the next portion isn't ://i.stack.imgur.com
[ SmokeDetector | MS ] Bad ns for domain in body, bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, +2 more: Rapid Diet Forskolin by osvaldoJTorres on meta.SE
"One of the main challenges online social systems face is the prevalence of antisocial behavior, such as harassment and personal attacks. In this work, we introduce the task of predicting from the very start of a conversation whether it will get out of hand. As opposed to detecting undesirable behavior after the fact, this task aims to enable early, actionable prediction at a time when the conversation might still be salvaged. "
@iBug hmm... if Smokey is also looking for offensives and trolling then I guess this study could be useful.... I sort of meant that the-creators-of-the-spam-detection-algorithm-in-Smokey might want to give this study a read
General
Heat detector is a StackApp whose goal is to catch offensive, rude, snarky comments, and comments that leads to abusive content (offensive, spam posts, or previous offensive comments). It is running a bot that queries comments from the Stack Exchange API and retrieves all comments posted...
Actually, it's in SOBotics (and perhaps a test room), with only a very few reports manually sent to SOCVR, if not enough flags are place from people in SOBotics.
@TetsuyaYamamoto That pattern looks like it's already caught by Potentially bad ns for domain in answer and Potentially bad ns for domain in body; append -force if you really want to do that.
@TetsuyaYamamoto Added sathya\.in to watchlist
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, link at end of body, pattern-matching product name in body, pattern-matching website in body, +2 more: Tuna is a few different low-calorie by jvdy on superuser.com
@TetsuyaYamamoto That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body; append -force if you really want to do that.
> Blacklisted website in body, blacklisted website in title ---------- Title - Position 1-20: svrtechnologies.com Body - Position 1-20: svrtechnologies.com
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, body starts with title and ends in url, link at end of body, pattern-matching product name in body, +3 more: specific shading for consistently by lsqwlsqw on superuser.com
@Melebius You are not a privileged user. Please see the privileges wiki page for information on what privileges are and what is expected of privileged users.
@TetsuyaYamamoto Thank you for dealing with this. However, when I last visited this page, I remembered scan was not a privileged command. Did it change? Why?
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, pattern-matching website in body, potentially bad ns for domain in body, repeated url at end of long post: How does this price compare? by oliviasmith on graphicdesign.SE
@TetsuyaYamamoto yeah, because it's now an alias for !!/report, but really it shouldn't. Non-privileged users should be able to execute this command as before.
@Melebius I've just added you to the privilege list, now you should be able to !!/scan, !!/report etc. Make sure you take a read of the feedback guidance/commands list on the wiki, and don't hesitate to ask anyone in here if you're unsure about something
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, body starts with title and ends in url, link at end of body, +3 more: Maximum Power XL by kusrc vaqw on superuser.com
