!!/watch-force- (?:\b(?<!-)|_)(?:dio|boss|sod|white)(?:[\W_]*+\d)+[\W_]*+n[\W_]*+e[\W_]*+t(?:\b(?![.-])|_)(?#spam wave 2019-10-03; blacklist if no FP in a few days)
@Makyen While this isn't a website, it's far less compute intensive to search for that text on its own than it is to search for \w*?출장마사지\w*, which is what would be needed on the keyword blacklist due to the added \b bookends. Testing indicates that the Unicode \b also doesn't see the word-break there in the text we actually have seen in spam. It's possible there really isn't a word-break, but someone who actually knows the language would need to say.
From the code comments: # Flee before the ugly URL validator regex! We are using this, instead of a nice library like BeautifulSoup, because spammers are stupid and don't always know how to actually *link* their web site. BeautifulSoup misses those plain text URLs.
@tripleee Those should be blacklisted in the near future. However, because I made them generic wrt. the numbers used, I wanted to leave it as a watch for a few days, just in case. If the spammers come back in the meantime, we can move it to the blacklist immediately.
In general, we haven't used any specific regex/logic for domain name obfuscation. We've added it in to each domain we've encountered based on what we see the spammers using. We could watch a regex which generally detects the form of the obfuscation we've seen here, but we'd need to be relatively careful. The more general regexes which were put in place during the spam wave started producing FP at a moderately high rate shortly after the spam wave was over.
I did play around with making at least one of those regexes more specific in order to reject most of the FP, but didn't finish off the effort, mostly because I saw that the spammers weren't using all that many domains. I can take another look at putting something together, if you'd like, or you're, of course, welcome to do so.
It would be beneficial if we come up with a generic/general set of domain obfuscations. However, if we're wanting to have a general set, then it should be written as more than just a regex (e.g. something similar to what's done for numbers). Accounting for all the possible ways to obfuscate something using a single regex can get either quite complex, or have a lot of unintended detections.
A relatively simple methodology would be to just remove the intervening [^a-z0-9.-] characters, at least when the sequence of them is short, from sequences of [a-z0-9-] and test against that (with the . characters removed from the domain and also test a version with dot instead of the .).
I was thinking these Asian character sets might need some more thinking, there are some obfuscations which make perfect sense when the surrounding text is logographic but not in Latin or (I suppose) Cyrillic text
come to think of it, if the surrounding text is just graphically different enough (Arabic, Hebrew, various Indic scripts) you have a lot more freedom around what you can do with a fragment of Latin and still let people figure out what you mean
on the other hand, all such obfuscations tend in the end to be too much work for most recipients; if they have to do more than just click, chances are they won't
True. Although, we have seen some where the spammer did expect the user to do a fairly significant amount of editing. IIRC, those tend to be gambling sites. I guess people who frequent those are willing to do more work.
This happens when I access a page that requires login, log in, and go back. The page history only knows that it wants to display an error, and has no idea what page I didn't have access to.
The URL is https://metasmoke.erwaysoftware.com/users/denied?required=core
@Dharman The !!/blacklist command has been deprecated. Please use !!/blacklist-website, !!/blacklist-username,!!/blacklist-keyword, or perhaps !!/watch-keyword. Remember to escape dots in URLs using \.
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, pattern-matching website in body, +2 more (491): Reviva Brain Pills by sigmamach89 on askubuntu.com
