@iBug That is silly, as tripleee said. Is there some reason not to compile them once per read of the file and reuse them? I must admit that I had assumed that the listing of detections in findspam.py was building a list of descriptors, which was then iterated over for each post.
@Federico If what you're wanting to match is an actual space, then just put the actual space in the regex as a space (i.e. a ` `). If you are wanting to match other things, in addition to a space, then you can use one of the character classes, etc.
> Potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username ---------- Title - Position 1-12: living foodz Body - Position 1-12: living foodz Username - Position 1-12: living foodz
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, bad pattern in url body, blacklisted website in body, body starts with title and ends in url, +2 more: Enduraflex reviews by user401005 on meta.SE
but, if running environment permits, eating a bit more RAM is OK
we were finding a balance between memory and CPU
ah, there was another issue that's currently unsure: Python's stock re module had a bug where reusing compiled objects for matching was very slow, so we were compiling it every time
we're on a 3rd-party regex PyPI package now, not sure if there's the same bug
@iBug same way huffingtonpost.com or facebook.com are not spam domains per se
we might still blacklist it if it's predominantly used by a spammer or troll but the domain is not one which was registered by a spammer to further their interests
the "-provisional" tries to say "not sure about this one"
I have no information on the specific implementations for Python, but normally the point of "compiling" the regex is to do some of the heavy processing in advance of when you want to use it. This is normally intended to allow you to reuse a compiled regex at significantly less computing cost vs. compiling the regex each time.
@iBug I believe this was always on regex (for as long back as I've been here at least) ... I vaguely recall somebody attempted to investigate whether it has the same bug but I'm unsure of the outcome
@iBug yeah, the tags are definitely not consistent
@iBug that's been on my todo list for a long time, so yeah, please do if you have the time
One dirty way is to check if rule['regex'] is a string, if yes then compile and overwrite that key with the compiled object. The CPU overhead for an if statement should be too small to notice.
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, pattern-matching website in body, potentially bad ns for domain in body, potentially bad keyword in body: Healthy dietweight-reduction plan by eqdw on rpg.SE
tpu- by paper1111
[ SmokeDetector | MS ] Bad keyword in body, pattern-matching website in body, potentially bad ns for domain in body, potentially bad keyword in body: wounded wound is much more by Cronic84 on rpg.SE
@tripleee For this watch, how about huffingtonpost\.com/[a-z]\w* instead? That will catch all of the TP we've seen and reduce the FP from existing SE posts by ~50%.
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, body starts with title and ends in url, link at end of body, pattern-matching website in body: Individuals as it could not have by plaginjabea on askubuntu.com
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching website in body, potentially bad keyword in body: Praltrix South Africa Review by jonnirayy on askubuntu.com
@iBug That pattern looks like it's already caught by Pattern-matching website in body and Pattern-matching website in answer; append -force if you really want to do that.
[ SmokeDetector ] SmokeDetector started at [rev d4eaa08 (iBug: number[s] autopull)](//github.com/Charcoal-SE/SmokeDetector/commit/d4eaa08) (running on teward/Solar Flare)
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, body starts with title and ends in url, pattern-matching product name in body, pattern-matching website in body, +2 more: Perfect Keto Max Additionally, by Goldar London on graphicdesign.SE
@TetsuyaYamamoto That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body; append -force if you really want to do that.
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, pattern-matching website in body, +2 more: It will provide you with greater muscle by nadw on rpg.SE
[ SmokeDetector | MS ] Url in title, blacklisted website in body, blacklisted website in title, potentially bad ns for domain in body, potentially bad ns for domain in title, +1 more: curlycrayy.com/vital-progenix/ by ghters46 on rpg.SE
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, body starts with title and ends in url, link at end of body, link following arrow in body, +5 more: Perfect Keto Max by jhmyughjfg on askubuntu.com
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, link at end of body, pattern-matching website in body, potentially bad ns for domain in body: Do Not Lose Sleep Over Weight by khonken on superuser.com
