@tripleee That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body Append -force to the command if you really want to add this pattern.
@Ollie From the comment, it appears a site mod is giving them a chance to disclose. Given the many other instances of spam from that user, I don't think that's likely. I've custom flagged linking to the domain.
@Ollie That pattern looks like it's already caught by Blacklisted website in answer and Blacklisted website in body Append -force to the command if you really want to add this pattern.
[ SmokeDetector | MS ] Potentially bad asn for hostname in body, potentially bad asn for hostname in title, potentially bad ns for domain in body, potentially bad ns for domain in title (4): Is this bank a scam? Citione.live by Carola Åkesson on money.SE
Yes, the above two could be easily combined in a regex. However, they appear to be separate campaigns, with different things being advertised, different domains, etc., so combining them to reach the blacklisting threshold isn't really appropriate, IMO. My opinion on this is fairly situational, so I could be convinced otherwise.
In addition, these appear to be primarily intended to catch usersnames, but that wasn't what the original watch was focused on, so I didn't limit it that way. If changed to just be usernames, then I'd be more amenable to combining them, but still not enthusiastic.
> Url in title, potentially bad asn for hostname in body, potentially bad asn for hostname in title, potentially bad ns for domain in body, potentially bad ns for domain in title ---------- Potentially bad asn for hostname in body - suathietbigiadinh.com address 151.106.117.86 in ASN 47583 Potentially bad asn for hostname in title - suathietbigiadinh.com address 151.106.117.86 in ASN 47583 Potentially bad ns for domain in body - suathietbigiadinh.com NS suspicious ns2.tenten.vn.,ns3.tenten.vn.,ns1.tenten.vn.
@tripleee That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body Append -force to the command if you really want to add this pattern.
@SmokeDetector This commit removes + from numbers when they're followed by a space. It also, interestingly, removes trailing punctuation, e.g. ), but not leading punctuation like (. I assume all of this has something to do with word boundaries, but I'm curious what exactly is going on. cc @Makyen since it's your commit.
@cigien That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer Append -force to the command if you really want to add this pattern.
@cigien I'll be submitting a PR, eventually, which has some more explanation along with some changes to the numbers detections. Along with improvements to the detection, part of the changes are to prevent people from adding "numbers" entries which it's impossible for the detection to actually detect and to require the user to use -force if they want to add a "number" that can't have an "exact" match (i.e. that can only be matched "normalized").
The changes also test, as part of the CI testing, the currently existing number entries to make sure that they can be detected and fails CI if they can't.
The commit you're asking about is what's necessary to actually have all the current entries able to be detected and make almost all of entries able to have an "exact" match (potential future changes for the ones which are not after the PR). Currently, no testing is done to see if it's possible to detect the entry which is added, so there were some entries which the detection could never match.
I added the commit now, because it's immediately beneficial, in that it makes some things which are intended to match actually be possible for the detection to match, the change/commit is not actually dependent on the code changes (even though there's more benefit with the code changes), and it's the portion of the changes from the expected PR which are most likely to result in merge conflicts (i.e. it's potentially more work to wait for the full PR).
@Makyen Fewer are able to have an exact match with the current code, but all, after the above commit, can at least match "normalized". With the changes, almost all of the remaining ones will be able to make an exact match.
@cigien The specific issue you noticed, the removal of trailing punctuation, but not some leading punctuation is that for an exact match the current code requires that the pattern begin with either a digit or a "+" (and nothing else).
The code after the PR will allow some additional preceding punctuation which is typically seen in phone numbers. Specifically, it will permit (?:[(+{[]{1,2}\d|\d(?<=[^\d(+{[]\d|^\d)), which is basically that it needs to start with a digit, which may be preceded by up to 2 of "(", "+", "{", and/or "[".
For a "normalized" match, all of the non-digit characters are stripped.
@Makyen Hmm, it looks like spaces still won't be allowed after the initial punctuation. I'll have to keep that in mind, since that turns up in posts every now and then. Is there any reason to not allow a space as the 2nd leading non-digit?
@cigien Other than that leading spaces are not part of a number? [IIRC, there was a more complex reason, but I'm not remembering the context at the moment.]
@cigien I should note that the new code will explicitly tell you that it won't be possible to make an exact match and at least something of why that's the case (can't match start, end, too many/too few digits).
@cigien So, thinking back on it, IIRC, it was because it made things significantly more complex to have to deal with allowing a space, but only as the second character and never as the first of two, or only, non-digit character.
@Makyen Ah, I see. That's fair. I'll give it a shot maybe. Though, with my limited regex skills I doubt I'll get very far, but it should be fun to try.
@cigien There's also a performance issue, which I haven't fully characterized. The regex (actually different permutations of the regex) is being run 28 times over every piece of text which we're testing.
@Makyen Hmmm... now that I really think about what that's doing, using a regex for that really isn't a great way to accomplish that, even if it is the way it tended to evolve when starting from a regex. At a minimum, there should be some pre-optimization applied... Yeah, that's probably reasonable.
