Not sure what to do here. The user who made this suggested edit has also, IIRC, recently had a number of answers removed as spam. Is there any mechanism to record/handle Spam edits organized in/by this room?
... the link in that edit was to the same YouTube as in the answers I referred to.
@AdrianMole We don't have a bot to monitor suggested edits, so there's not really a whole lot we could do here (besides watching URLs in SD so that we catch them in subsequent posts).
Fortunately, a lot less harm can be done through edits since they have to be reviewed anyway
(There used to be a bot in SOCVR to monitor the edit queue, but I don't think that's running anymore)
@SmokeDetector Translation: Best wishes to all your friends for Navratri
So I modified that existing PR to incorporate the suggested changes from the original PR, but without a way to test to see if it actually works I'm a bit dead in the water. Also, not entirely sure it would even work in the first place... There are edge cases that aren't accounted for, and... Eh. I'll need to spend more time with this than I thought I guess.
To run a test instance, basic setup instructions are in the readme. In order to make your instance run in a testing room instead of Charcoal HQ, you can create a file called rooms_custom.yml to override the regular rooms.yml
AFAIK you shouldn't need metasmoke integration to test this, so you can leave those config entries blank -- but you will need Git integration, so you'll have to fill those out
@Spevacus not a problem! Feel free to ask in chat/ping me if you run into any trouble. Once you get past the initial setup SD's codebase is mostly pretty straightforward, but it's definitely old enough and complex enough to have a few gotchas.
Might need to register / ask to use it, might be able to reply to it "help" to find out how to operate it. Anyone who's been in Charcoal for a while is likely to be granted permission to use the Bots in that room.
I feel like this GitHub issue is important despite being 30 days old; can someone please remove the stale label and replace it with the confirmed label?
Did MS change how it decides whether to include subdomains in the domain at some point? The spam ones are all older.
I'm actually still sort of inclined to consider unwatching it. Every single post after 2016 is at least plausibly not-spam. One is more dubious, but I could believe it's just low-quality. On the other hand, it's not very many posts and they were very spammy 4-5 years ago. So maybe it's fine.
Zero spam for that domain, one sketchy post that linked it when asking how to to make their webpage (which they also linked) load faster (probably would've FPed it if they'd posted to SO and not Travel). Many, many non-spam posts across the network.
@RyanM Matched by (?:full|movies?|watch(?:es)?|online|free|downloads?|stream(?:s|ing)?)(?!(?:[^<]|<(?!\/?code>))*+<\/code>)(?:[\W_]*+\w+){0,5}?(?:[\W_]*+(?:full|movies?|watch(?:es)?|online|free|downloads?|stream(?:s|ing)?)){2,} on line 15415 of watched_keywords.txt
Only one person has ever made that particular typo when writing something offensive, and...well, let's just say they probably were too busy adding other profanity to the post to proofread too thoroughly.
And zero people have ever spelled it "coc" in offensive posts
On the other hand, "dic" is common shorthand for "dictionary," and a lot of people play Clash of Clans.
@NobodyNada @Spevacus If you clone your local git repository from a GitHub repository (i.e. remote is a GitHub URL), then the GitHub repository to use for git operations is automatically determined without the need to make any code changes. In other words, to set up testing into your own fork: On GitHub, fork the SD repository into your account. On the machine on which you are going to be running SD, from the directory above where you want SD's code to be, do: git clone <URL to your fork>
copy config.sample to config and edit (you'll need an SE account's credentials and a GitHub account's credentials).
Create a rooms_custom.yml which tells SD to post in the chat room you are using for testing (and not everywhere else).
I'm probably going to unwatch that, but I'd also note that the lookbehind is unnecessarily complex: it's equivalent to \d{6}, which works in more regex engines (like MS search)
also it should be after the match, per Makyen's note about efficiency.
@Daniil That's an option, but substantially less desirable once one or more of the specific email addresses are moved to the blacklist. If you felt that was a good solution, why did you make the change so the qq\.com detection didn't detect numeric email addresses?
based on the chat history starting here, I'm wondering if the intention was to do one of those split-detection things like where we watch somekeyword\.com and also somekeyword(?!\.com)
if we do want to watch qq.com emails, @(?:\w+\.)?qq\.com and \d+@qq\.com both get decent results, depending on your preferred TP/FP tradeoff (4 more TP, 23 more FP, 14 more NAA for the first).
It looks like @Makyen's two changes here (one of which was to remove the watch I just removed) never went through. If you did want to rewatch qq.com more broadly, I'd recommend using one of my suggestions above. I'd lean toward the numeric-address one, since it's more accurate for relatively little cost, but both seem fine)
I'm happy to put some sort of watch back on that domain, but at this point I'm very confused about the state of things. It definitely does significantly better if you only look at emails, but I don't know if we handle email providers via special rules rather than watches.
> Potentially bad ns for domain in body, potentially bad ns for domain in title ---------- Potentially bad ns for domain in body - mylifemyroad.com NS suspicious ns55.domaincontrol.com.,ns56.domaincontrol.com. Potentially bad ns for domain in title - mylifemyroad.com NS suspicious ns55.domaincontrol.com.,ns56.domaincontrol.com.
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, body starts with title and ends in url, link at beginning of body, +3 more (631): Maxx Power Libido by Lawrences johnson on stackoverflow.com
Handwavey data analysis: Non-domain watches are about 11.8% accurate at identifying TP posts. Domain watches are about 28.5% accurate.
This uses the highly scientific method for differentiating them of "does the matched string start with words separated by dots?" It also looks only at posts detected solely by watchlist.
@RyanM That is not surprising. Being the place where we put things that are of unknown potential wrt. being spam is, after all, a major part of the purpose of the watchlist.
The general gist of which is that I don't think sorting through 1278 false positives to find 55 true positives is an especially good use of time, and that we should cut anything that can reduce FP with that sort of 95%+ accuracy if we want the queue to stay in check without individual heroics.
with the goal, of course, of finding other creative ways to catch spam with the time saved not reviewing all those FP :-)
The point I'm trying to convey is that I don't think we'd approve a watch that hit 95% FP, so we shouldn't keep one either.
Obviously we should try to cut only FP, which I've done with a number of watches. But sometimes it isn't possible to hit one or two in a mass of hundreds
I also don't want to tell you how to run your project, so please take this as advice, not demands. You've put in a huge amount of effort and I respect that a lot.
I think for that particular one, it probably needs to be either split up more to be more specific ("kill ... moderator" is very likely bad, "kill ... boss" less so because gaming) or redone as a python rule that can be smarter than a regex
We have 2 primary goals: 1. Don't autoflag FP. 2. Detect all spam. We've branched out a bit into detecting R/A posts, particularly to help sites/moderators who have asked for help detecting R/A posts, and have mostly adopted a third goal of also detecting a significant quantity of R/A posts (but how much we target R/A content hasn't really been officially established).
The largest issue with that regex is that iterating on it is very difficult due to how long it takes to test. If I do end up trying a dev task, replacing that regex might be one that I take on.
@RyanM Then try to formulate something that's better than what we currently have. Detections are not intended to be static, particularly not things which are on the watchlist.
@RyanM Yes, the length of time it takes to test these things is quite substantial. It's appreciated that you are going through things.
Basically how I normally refine these is that I do a search on MS for the regex of interest. I then go through the TP and FP to see if there are patterns within or around what is being detected which can be used to refine out the FP, while continuing to include the TP. It's an iterative process and takes time, but the data is, hopefully, available, due to having the existing watch (which may have been written with a known expectation that it might be a bit broad in order to get data to refine).
related: is there a way to run a rule over the MS database? (e.g., in a local SD instance)
@Mast it's worth noting that "watchlist-only match containing the word 'holocaust'" is just shy of 7.5x more likely to be TP than "watchlist-only match containing the word 'kill'"
@RyanM No, but it would be a nice capability to have. I've considered writing code to do it, but I haven't gotten to it. For testing of specific detections, that would be helpful, but it would also be nice to be able to run the current detections against all of the MS posts. Doing so would allow us to have more accurate weights for the current detections.
So my approach would be to look through the "kill" TP and figure out which ones are reasonable to catch with regex and not just a happy coincidence, and write watches for those specifically, then remove "kill" from those two giant regexes.
@RyanM That begins to sound like getting a bit too specialized, but may have just been how you worded that. While we do want to detect the exact thing again, the goal for regexes like that one is to catch future similar posts.
Yeah, I may have phrased it badly. Basically I'd write a "kill"-specific regex, listing things that are specifically problematic when used with "kill." Though...looking at these, I'm having a really hard time classifying them into any sort of "similar" groupings, and honestly most of these seem like 90% luck that they happened to be caught.
@VictorVosMottor Messages/reports from SmokeDetector in Charcoal HQ are generally kept as records. If you really need to delete a message, please use sd delete-force. See this note on message deletion for more details.
@RyanM That pattern looks like it's already caught by Offensive body detected, Toxic answer detected, Toxic body detected, and Offensive answer detected; append -force if you really want to do that.
> Potentially bad keyword in body ---------- Potentially bad keyword in body - Position 30-90: kill Er because he wants to. Ok. So now, according to Jewish
@RyanM That pattern looks like it's already caught by Toxic answer detected, Potentially bad keyword in body, Toxic body detected, and Potentially bad keyword in answer; in addition, the regex contains an unescaped "."; in most cases, it should be "\."; append -force if you really want to do that.
Okay, I went through all the hits from that search. I've added a few watches to catch 6 of them, and gave FP/NAA feedback in cases where I think reviewers (...including me, whoops) picked the wrong option. That leaves 9 posts where I don't have a good answer to how to detect them that I can imagine ever catching something else, and one post that was part of an ongoing troll campaign that could've been legit if asked by someone else.
Plagiarism: 1 Fixed: 6 Actually caught by a different rule and just picked up by the imprecise search: 1
cc @Makyen ^
the only consistent theme is that they're all weird.
"Posts I don't know" should have been "Posts I don't know what to do with:"
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, bad keyword with email in body, messaging number in body, pattern-matching email in body, +1 more (407): Do You Need A Loan? by LAKEVIEW FINANCE on stackoverflow.com
@VictorVosMottor It is. They know how to use Stack Exchange, because they've been doing this for years, but they persist in deliberately misusing it. Also, they're banned and evading said ban.
