Uhhh yeah I mean AAAAA-BBBBB could be an Indian mobile number (and one of the loan providers they spam on other sites is based in India) but also a web search for one of the other numbers suggests they're sort of just inserting dashes randomly
@JeffSchaller The difference in handling between a number being considered North American vs not is that for North American numbers SD looks for the number both with and without a starting "1". That's really the only difference. If SD is told something is North American, but it's not, then there's a small increased chance of FP. If something is a North American and SD is told that it's not, then any future spam which includes a starting "1" without a separator between the "1" and the rest of the number will not be detected. Overall, given the very low rate of FP, even on watched numbers (with a couple of notable exceptions of repeating patterns, which have been removed/changed), it's probably better to error on the side of saying it's a North American number.
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 03:03:26 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.4/questions/1127480?key=IAkbitmze4B8KpacUfLqkw((&filter=!scxINmJWAnjuckNJkP9YfbG196DrCj)DDXQjRKUGfp-QzZ3b1nRh1Qitw7)FdyW_)iLrL8TBxFYZ1&state=PublishedAndStagingGround&site=serverfault.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 03:03:41 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.4/questions/4669177?key=IAkbitmze4B8KpacUfLqkw((&filter=!scxINmJWAnjuckNJkP9YfbG196DrCj)DDXQjRKUGfp-QzZ3b1nRh1Qitw7)FdyW_)iLrL8TBxFYZ1&state=PublishedAndStagingGround&site=math.stackexchange.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 03:03:55 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.4/questions/77288?key=IAkbitmze4B8KpacUfLqkw((&filter=!scxINmJWAnjuckNJkP9YfbG196DrCj)DDXQjRKUGfp-QzZ3b1nRh1Qitw7)FdyW_)iLrL8TBxFYZ1&state=PublishedAndStagingGround&site=cs.stackexchange.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 03:04:09 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.4/questions/282951?key=IAkbitmze4B8KpacUfLqkw((&filter=!scxINmJWAnjuckNJkP9YfbG196DrCj)DDXQjRKUGfp-QzZ3b1nRh1Qitw7)FdyW_)iLrL8TBxFYZ1&state=PublishedAndStagingGround&site=mathematica.stackexchange.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 03:04:24 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.4/questions/3751130?key=IAkbitmze4B8KpacUfLqkw((&filter=!scxINmJWAnjuckNJkP9YfbG196DrCj)DDXQjRKUGfp-QzZ3b1nRh1Qitw7)FdyW_)iLrL8TBxFYZ1&state=PublishedAndStagingGround&site=math.stackexchange.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 03:04:38 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.4/questions/113515?key=IAkbitmze4B8KpacUfLqkw((&filter=!scxINmJWAnjuckNJkP9YfbG196DrCj)DDXQjRKUGfp-QzZ3b1nRh1Qitw7)FdyW_)iLrL8TBxFYZ1&state=PublishedAndStagingGround&site=datascience.stackexchange.com
@Makyen A normalized version, {{'917008172090'}}, of that pattern is already on the number blacklist. You can use !!/bisect-number 91 7008172090 to determine which entries on the number lists match that pattern, which would tell you: Matched by the following: +17008172090 on line 819 of blacklisted_numbers.txt: 7008172090 found normalized 917008172090 on line 824 of blacklisted_numbers.txt: 917008172090 found normalized
@Makyen A normalized version, {{'17008172090', '7008172090'}}, of that pattern is already on the number blacklist. You can use !!/bisect-number 700-817-2090 to determine which entries on the number lists match that pattern, which would tell you: Matched by +17008172090 on line 819 of blacklisted_numbers.txt: 7008172090 found normalized
@Makyen A normalized version, {{'18653694404', '8653694404'}}, of that pattern is already on the number blacklist. You can use !!/bisect-number 865-369-4404 to determine which entries on the number lists match that pattern, which would tell you: Matched by 8653694404(?#IS NorAm) on line 825 of blacklisted_numbers.txt: 8653694404 found normalized
@Makyen A normalized version, {{'9394952861', '19394952861'}}, of that pattern is already on the number blacklist. You can use !!/bisect-number 939-495-2861 to determine which entries on the number lists match that pattern, which would tell you: Matched by 939-495-2861 on line 830 of blacklisted_numbers.txt: 939-495-2861 found verbatim
@tripleee hmmm. the internet tells me that "Ahegao is a term in Japanese pornography for a facial expression of characters during sex" - so arguably, that detection seems, uh, valid in the "inappropriate username" department?
@RyanM That pattern looks like it's already caught by Bad keyword in body and Bad keyword in answer Append -force to the command word(s) if you really want to add the pattern you provided.
@RyanM That pattern looks like it's already caught by Bad keyword in body and Bad keyword in answer Append -force to the command word(s) if you really want to add the pattern you provided.
Hi, I'd like to propose distinguishing between Japanese and Chinese text in the spam detection routines. This can be done very accurately using a simple heuristic: simply check for the presence of (any hiragana or katakana character), before checking for (any unified CJK ideograph).
This is useful information because a) messages written in Chinese are IMX considerably more likely to be spam (since they're sent by the sort of people who bypass the national firewall) and b) on Stack Overflow, people posting in Japanese who have something that doesn't seem to be spam can be directed in the comments to the Japanese-localized site.
it would not be technically impossible but it is problematic because metasmoke keeps track of a rule's weight etc by name ... this should perhaps separately be fixed but until then the unfortunate side effects of renaming a rule the cure might be worse than the ailment
I think we already have a ticket in Github about this but let me check
@KarlKnechtel we do see spam in both Chinese and Japanese; unfortunately, metasmoke's regex engine is not capable of searching for [\u4E00-\u9FFF]+ so it's hard to quantify
@tripleee what does it use to detect those characters then? o_O
(I'm not actually asking it to give a lighter weight when japanese is detected instead of chinese; I just think it's useful information for human reviewers)
@tripleee That pattern may be a North American number. If it is, please use a format which starts with an optional 1 followed by possible separator text and has the main number in the format 628-917-6203 where - could be a single alpha character or any [\W_]*+. Alternately, you can add the comment (?#IS NorAm) to the end of the pattern to force also using the alternate normalized form, or (?#NO NorAm) if it's not a North American phone number and it's incorrectly recognized as one. Perhaps try !!/blacklist-number 628-917-6203. Append -force to the command word(s) if you really want to add the pattern you provided.
@KarlKnechtel there are two separate components here; Smoke Detector is what actually detects spam, but the reported spam is then stored in a database system called metasmoke, which is implemented in Ruby on Rails ... the regex engine actually lets me search for individual hiragana, like metasmoke.erwaysoftware.com/…
(unfortunately the regex search requires you to have a metasmoke account and be logged in)
that particular search currently gets me 738 results, 139 true positives (i.e. spam or rude) and 557 false positives out of the messages in the metasmoke database, i.e. messages detected by SmokeDetector as possible spam or manually reported
