@RyanM I wish they gave you an extra like... 50 characters or something. I always find that I want to put in one or two more links but I'm just over the limit.
Got this one in 493 characters (Markdown deliberately unformatted here): The original revision of this appears to be _very_ sneaky spam. The text is almost identical to [this comment](/q/32075177#comment96175188_32075177) by the OP, which I believe is itself spam. It copies the XML from the accepted answer, but reads as though it's written by someone other than the OP with a similar problem. The OP's account appears to be compromised, as they went from [fairly normal activity in 2015 to weird spammy comments in 2019](/users/2435339?tab=activity&sort=comments).
@RyanM Matched by (?:films?|complets?|regarder(?:s|es)?|t[ée]l[ée]chargements|en\s*ligne|gratuits?|diffusion|courant)(?:[\W_]*+\w+){0,5}(?:[\W_]*+(?:films?|complets?|regarder(?:s|es)?|t[ée]l[ée]chargements|en\s*ligne|gratuits?|diffusion|courant)){2,}(?#French verion) on line 15422 of watched_keywords.txt
That would be doable with a backreference, but I'm pretty sure we can't use those in watches given how the watchlist is implemented. Example that catches 15/17 TP while only hitting 10/50 FP:
The link is too long, but the regex would be (films?|complets?|regarder(?:s|es)?|t[ée]l[ée]chargements|en\s*ligne|gratuits?|diffusion|courant)(?:[\W_]*+\w+){0,5}(?:[\W_]*+(?!\1)(films?|complets?|regarder(?:s|es)?|t[ée]l[ée]chargements|en\s*ligne|gratuits?|diffusion|courant)){2,}(?#French verion)
The (?!\1) prevents the later matches from being the same as the first word.
the second group should also be non-capturing; I forgot to put that back post-debugging
@RyanM You would need to use a named capture group, and use a name that's very unlikely to be duplicated. Your watch/blacklist regex must be independent of other possible regex fragments. You can't have any idea which numbered capture group you'd actually end up with.
Currently toying with (?:bulk\W++SMS\W++(?:\w+\W++){0,5}(?:server|services?|marketing|business|supplier|account|Delhi|compan(?:ys?|ies)?)|marketing\W+(?:\w+\W++){0,5}bulk\W++SMS)) but it needs a bit more checking for existing FP...and apparently for TP we didn't catch the first time
@RyanM Matched by 234\W*+[89][01]\d\W*+\d{3}\W*+\d{4,10}(?#comment:234\W*+(?:8032214238|8167873816|8102960248|8149470344|8133873774|8073673757|9055637784|8135790702|9051442970) on line 224 of bad_keywords.txt
@RyanM It's currently 889 Total / 834 TP (93.81% TP) / 51 FP / 5 NAA, which is within the blacklist ballpark. Obviously, fewer FP would be better, but it's not at the level where we should be removing it from the blacklist.
If matches in code are excluded, %TP increases to 846 Total / 827 TP (97.75% TP) / 15 FP / 4 NAA. Two of the 7 TP which were in code would then not have been caught by other detections.
@Makyen For reference, what is the ballpark we're aiming for on the blacklist? The guidance is stricter than that, suggesting that blacklists should not have any FP, but obviously (and sensibly) a few FP are acceptable here and there.
@RyanM The guidance is stricter than that for putting things onto the blacklist, because once they are on the blacklist, people tend to not pay all that much attention to when they should be removed from the blacklist. The criteria exist so that entries that actually make it onto the blacklist stand a chance at remaining at a high level of %TP.
In general, I try to target 95%+ for %TP (preferably even higher). At 95%TP, that's equivalent to 1 FP out of 20. The real goal is 100%TP, but that's not really obtainable.
I've been trying to reduce FP a bit, so I'm particularly interested in at what point detections are considered too FP-prone. Obviously anything getting 90%+ hitrates is always going to have a place somewhere, of course. I'm trying to find cases where I can make substantial FP reductions to watches by trimming them down so that I'm removing detections that would be ~95%+ FP.
My pipe dream is some sort of code-detection likely-FP heuristic, but that would need to be code, not just regex changes :-) A regex-based proof-of-concept was a little under 99% accurate at identifying detected FP.
That accuracy would, it's worth noting, go down a bit if we stopped detecting things like model.fit, so that might be a good next step in my FP-reduction ideas.
It's also worth noting that it could be more accurate if we were consistent about whether "asdfljkhasdfasdfalskjdfasldkfjasldkfj [code dump]" is TP or FP :-)
@Mast No, I think they're actually writing these specifically for this spam campaign. I've checked several at this point, and I can't find original sources on any of them. They're also related to the question, and only mildly worse than a lot of the throw-it-against-the-wall-and-see-if-it-sticks guesses the [android] tag gets constantly.
unrelated to anything: the project links message seems to have escaped its home in the pinned items, could an RO or mod please help it find its way back at their earliest convenience?
It tightens the match on the fourth and fifth digits to the three combinations they've actually used (it previously matched 91, which they've never used), and restricts the length to avoid hitting other non-phone numbers.
[ SmokeDetector | MS ] Bad keyword in username, blacklisted website in answer, potentially bad ns for domain in answer, username similar to website in answer (246): Disqus SSO - Not working by htsindia on stackoverflow.com
Also, props on putting that regex together in the first place. It was easier to tweak using the dataset of the results yours gave than it would have been to create from scratch :-)
[ SmokeDetector | MS ] Potentially bad asn for hostname in answer, potentially bad keyword in answer, username similar to website in answer, blacklisted user (136): The use of prop.test in R by finnstats on stackoverflow.com
@Mast That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body; append -force if you really want to do that.
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching website in body, potentially bad keyword in body, +1 more (489): Male Extra Pills Really Works? by Briangwalker on stackoverflow.com
@SmokeDetector tpu- article posted today, answer is just copy-pasting the start of the article. Copied section is dubiously relevant to answering the question.
[ SmokeDetector | MS ] Url-only title, bad ip for hostname in title, bad keyword in title, blacklisted website in title, pattern-matching product name in title, +2 more (499): ketoavis.com/ultra-keto-x-burn/ by mhukopgqwdgh on askubuntu.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in link text in body, link at beginning of body, pattern-matching product name in body, pattern-matching website in body, +2 more (404): How to use the Viaxal Enhance: by vaxlenhnce on meta.SE
[ SmokeDetector | MS ] Bad ip for hostname in answer, bad ns for domain in answer, blacklisted website in answer, potentially problematic ns configuration in answer (269): Sitecore best practices by brown lee on stackoverflow.com
[ SmokeDetector | MS ] Bad keyword in answer, blacklisted website in answer, pattern-matching website in answer, potentially bad keyword in answer, potentially problematic ns configuration in answer (259): Error installing printer driver by Andrew Jacob on askubuntu.com
