Why is this watch PR showing far more hits ("\>{4,} has been seen in 3145 true positives, 137 false positives, and 8 NAAs.") than the linked MS search returns?
If I could do what I described, and assuming your proposed search is the correct one, the FPs get down to only 1, which is something I'm happy with having caught anyway, FP or not.
[ SmokeDetector | MS ] Few unique characters in body, no whitespace in body, potentially bad keyword in body, repeating characters in body (261): Расширение sssssss ✏️ by peroni1 on ru.stackoverflow.com
I have been spinning my wheels for the past... since that message trying to make this work.
I am trying (?<!\<code\>)\>{4,} on the text from here using regex101.com The trailing > from <code> is still getting matched, when it shouldn't be, because the negative lookbehind doesn't consume characters.
@CodyGray Hmm, I'm not sure. If word boundaries are added then we can skip the closing one, but the one after the lookbehind should still be there I think. I'll play around a bit with it to see.
Wasted? What do you think I do with my time? I'd be getting high and watching youtube otherwise. This way I can get high and learn something about regexes ;)
@CodyGray We could avoid matching the > after code probably, but it wouldn't help with this FP at least, it seems to be caught by arabic>>>>I in the first line.
@cigien When possible, please avoid using look-arounds as the first thing in a regex. Using a look-around as the first thing tends to be computationally expensive. In most cases, the regex can be easily adjusted to have the look-around at least a few characters after the start of the regular expression, which can reduce the computational load substantially (up to more than an order of magnitude, depending on the text being matched and the regular expression).
Matched by the following regexes: norman[\W_]*+reedus[\W_]*+cbd[\W_]*+oil(?:[\W_]*+(?:\d++|[\da-f]{5,}+)s?)* on line 26393 of watched_keywords.txt webself\.net on line 2835 of blacklisted_websites.txt
Matched by the following regexes: herbal[\W_]*+(?:grown|native)(?:[\W_]*+(?:cbd|oil|order|price|\d++|[\da-f]{5,}+)s?)* on line 2424 of bad_keywords.txt herbal[\W_]*+(?:grown|native)(?:[\W_]*+(?:cbd|oil|\d++|[\da-f]{5,}+)s?)* on line 26367 of watched_keywords.txt
@CodyGray I guess it depends on your point of view. The first one seems more obvious to me that it's looking for a static string of characters which must match the case supplied, then looking backwards to something, which will, of necessity, mean that it's looking backwards for something that is prior to the already found string. Given that it was looking for a static string, it wouldn't make sense for it to be only re-verifying that the static string was what it found.
It does have to include the static string in the look-behind, but it's a given that that portion of the look-behind matches.
@cigien Merging two entries like this is going to depend on if you feel that it's the same product by the same people. Effectively merging this is grafting the prior 6 TP onto the new item, which may, technically, allow blacklisting the combined regex prior to the new product getting all that many TP or testing/exposure on the watchlist. It's probably OK in this instance, but it's not really something we should be doing on a regular basis.
It would be OK to try to find a pattern that more generally matched multiple occurrences, but this instance feels like it's trying to merge them in order to blacklist the newer product name sooner.
It also concerns me a bit because the pattern that's being used is not all that restrictive. If the "CDB oil" portion was required instead of being optional, I'd feel more comfortable about it. Basically, "herbal native" doesn't fit into as many "normal" sentence structures as "herbal grown" may, which could mean that "herbal grown" might end up seeing some FP.
I see that I should have gone through the transcript earlier, or at least gotten back to to this ping sooner, as you've already made the adjustment. It's reasonable for you to have assumed I would have replied prior to this (i.e. my fault). We're probably OK with it as it is now, particularly given the additional TP posts with "herbal grown" which have shown up since the message I'm replying to.
@RyanM The deletion tracking isn't all that robust. It's not uncommon for it to miss that a post has been deleted. A very quick check using the test in this question didn't show any problems with the SE WebSocket. However, that test is really intended to detect only a specific small subset of the possible problems on SE's side. All things considered, I wouldn't read much wrt. issues on SE's side into SD/MS missing detections of posts being deleted.
@Makyen I hadn't looked at whether the posts were the same product by the same people. I hadn't even realized that that was particularly relevant, especially for the cbd-oil and keto spam. I'll definitely keep that in mind when deciding whether to merge entries in the future.
Looking at the posts more closely, I don't see any evidence that they are the same spammers. In fact, the native series being grouped tightly a year ago, and the grown series in the last 2 days suggests that they are not the same.
It seems that you're comfortable with how my adjustments ended up, but if it's ok with you I think I'd like to split those entries up. The native series can be blacklisted given the TP on them. Additionally, the "cbd oil" portion can be made mandatory, as it catches all TP, even though you made the original watch with that being optional, and as you point out, "herbal native" is unlikely to generate FP on its own.
For the grown series, I agree that making "cbd oil" mandatory would be much better, so I'd like to do that. Also, it seems it might not technically qualify for blacklisting, but making the adjustment for "cbd oil" to be mandatory makes it quite unlikely to generate FP, so I think I'll blacklist it anyway instead of waiting for additional TP. Is this reasonable? I won't make any of these adjustments till you respond. Take as long as you need, of course.
And please don't apologize for not responding within some time frame; I went ahead with the adjustments because I felt they were reasonable, not because I assumed you'd have responded by then. If I do need you specifically to respond, I'd ping you at at least a second time as a reminder.
@cigien Merging two entries like this is going to depend on if you feel that it's the same product by the same people. Effectively merging this is grafting the prior 6 TP onto the new item, which may, technically, allow blacklisting the combined regex prior to the new product getting all that many TP or testing/exposure on the watchlist. It's probably OK in this instance, but it's not really something we should be doing on a regular basis.
For example, imagine two entirely made-up hypothetical unrelated Java help websites. java-help.com and java-examples.com. We could catch both with a watch on java-(?:help|examples)\.com, but we wouldn't want to do that, because one getting itself added to the blacklist shouldn't result in the other getting added to the blacklist as well.
Yes, but it's often done at the granularity of an entire watch (and is far, far easier to do if done that way, because the blacklist commands also remove the duplicate entry from the watchlist)
Otherwise you'd have to manually notice that they're unaffiliated (which you might not, since you'd probably assume they were watched together for a reason) and then unwatch the combined one, manually rewrite the regex into two regexes, watch one and blacklist the other. This would be extremely annoying.
@Mast That spam flag would be very likely to be declined on many sites; I would always flag something subtle like that with a custom flag, because context (from a link) is needed..
@Yatin No, it's a how-to question where people have provided wrong answers recommending apps for a question that asks how to do something programmatically...
...plus an NAA ("Does this work ...") with an upvote. Amazing.
In an attempt to start collecting bookmarklets, I've created the following gist. Anyone, feel free to run with it, none of them are mine anyway, and let me know if someone has a bigger collection stored anywhere.
I feel like blacklisted_websites not requiring word boundaries is gonna bite us one of these days
also I love the edit summary:
> You are recharging from there is fair, and you get that thing cheap, if you use promo code, you will take the same thing, if you do not get the expensive something, then the company does it because you are with the company. Stay connected.
@RyanM Not having bookending on the blacklisted websites does bite us from time to time. When it does, we've added a couple/few different types of bookending to the specific entry. It would be reasonable for us to add it to the baseline, but being thorough in changing it would require searching MS for each entry with and without the new bookending and seeing where there's a difference to determine which are relying on there not being bookending in order to be effective.
