@Makyen is it behaving?

[ SmokeDetector | MS ] Bad keyword with a link in answer (82): What is the difference between include_directories and interface_include_directories with headers only library‭ by BING BAI‭ on stackoverflow.com

naa feedback received on [MS] What is the difference between include_directories and interface_include_directories with headers only library‭

@ThomasWard It seems to be. There was a minor issue earlier (my response to the ping, effectively saying "needs more investigation").

!!/stats

@Makyen "uptime" stats: Posts scanned: 23501, Q(8017), A(15484); Scan time: 32670.1, Q(24216.31), A(8437.76); Posts per second: 0.72; Grace period edits: 7532; Unchanged posts: 7940, Q(6523), A(1417); No post lock: 485; Errors: 0; Max scan time: 84.02; Max scan time post: pt.stackoverflow.com/318164; Post processing lock: 0.78; Check unchanged: 6.89; Threads: 2853, API(1229), 155QA(1594), EW(1259), BFrr(0); High CPU: 140; Thread limit: 2659; Site limited: 383, SO(383), nonSO(0); All errors: 0; Started: 2023-01-30T18:59:06.545194

!!/stats list

@Makyen The currently stored stats sets are named: "all", "uptime", "ms", "Makyen-2022-12-16_at_11:26", "Makyen-2023-01-27T19:09".

!!/stats Makyen-2023-01-27T19:09

@Makyen "Makyen-2023-01-27T19:09" stats: Posts scanned: 832194, Q(262463), A(569731); Scan time: 940412.55, Q(685834.71), A(254024.47); Posts per second: 0.88; Grace period edits: 272444; Unchanged posts: 344514, Q(265460), A(79054); No post lock: 25700; Errors: 0; Max scan time: 199.34; Max scan time post: math.stackexchange.com/4626082; Post processing lock: 30.95; Check unchanged: 235.91; Threads: 64979, API(37446), 155QA(35431), EW(29535), BFrr(0); High CPU: 3335; Thread limit: 45921; Site limited: 12555, SO(12555), nonSO(0); All errors: 0; Started: 2023-01-27T19:10:39.186305

!!/stats all

@Makyen "all" stats: Posts scanned: 13999438, Q(4923882), A(9075556); Scan time: 16616728.2, Q(12860672.9), A(3732287.5); Posts per second: 0.84; Grace period edits: 3973788; Unchanged posts: 111788599, Q(54853082), A(56935517); No post lock: 648999; Errors: 0; Max scan time: 2424.64; Max scan time post: stackoverflow.com/75037757; Post processing lock: 1619.04; Check unchanged: 6472.72; Threads: 2488750, API(1623848), 155QA(1311465), EW(1176958), BFrr(0); High CPU: 183915; Thread limit: 1167967; Site limited: 198492, SO(198492), nonSO(0); All errors: 0; Post errors: 105; Started: 2022-10-16T02:28:53.707663

!!/stats Makyen-2022-12-16_at_11:26

@Makyen "Makyen-2022-12-16_at_11:26" stats: Posts scanned: 9349613, Q(3036253), A(6313360); Scan time: 9792328.57, Q(7302434.18), A(2483098.9); Posts per second: 0.95; Grace period edits: 2962721; Unchanged posts: 3056223, Q(2508644), A(547579); No post lock: 175817; Errors: 0; Max scan time: 2424.64; Max scan time post: stackoverflow.com/75037757; Post processing lock: 283.75; Check unchanged: 2898.22; Threads: 989337, API(452261), 155QA(539706), EW(449582), BFrr(0); High CPU: 170692; Thread limit: 699317; Site limited: 148796, SO(148796), nonSO(0); All errors: 0; Started: 2022-12-16T11:26:42.854970

@Makyen nice

yeah i'm more concerned if SD is up and running and not eating my infra XD

@SmokeDetector Hmmm... "No post lock" is substantially higher than what it's been in the past. It indicates where a post ID is actively being scanned in another thread at the time a thread thinks it should scan it. [Hmmm.. there's a post versioning race condition there which is not handled explicitly, but was expected to work itself out.]. So, while that should go up with more CPUs I wouldn't expect it to go up that much.

@Makyen wonder if this is what is breaking CPU :P

> was expected to work itself out

under what cirumstances has anything just "worked itself out" lol

@ThomasWard It shouldn't be. If a thread isn't able to get a lock on processing a specific post (i.e. another thread is actively scanning that post), then it just skips that post and doesn't scan it in the thread that's not currently scanning it.

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, potentially bad keyword in body (193): FANSCLUB99 | SITUS SLOT SERVER KAMBOJA MUDAH MAXWIN‭ by tigabola‭ on stackoverflow.com

my question about when has it ever worked itself out properly still stands :P

tpu- feedback received on [MS] FANSCLUB99 | SITUS SLOT SERVER KAMBOJA MUDAH MAXWIN‭

!!/watch tigabola\.com

@Jesse You don't have blacklist manager privileges, but I've created PR#7525 for you.

PR#7525 ("Jesse: Watch tigabola\.com") opened by SmokeDetector

Do we want to watch the IP address from that question too?

!!/approve 7525

[ SmokeDetector ] Blacklists reloaded at rev 47b38366bf (SmokeDetector: Merge pull request #7525 from Charcoal-SE/auto-blacklist-1675113970) (running on teward/Thoth)

!!/watch 15\.197\.153\.122

[ SmokeDetector ] Blacklists reloaded at rev 1303985d20 (SmokeDetector: Auto watch of 15\.197\.153\.122 by Spevacus) (running on teward/Thoth)

I guess that's a yes.

@Jesse We can, yeah. ;)

Sounds good.

Yeah, when considering whether we should watch something, generally speaking, if you think there's a decent likelihood we'll see it in future spam, then it's not a bad idea to add it to the watchlist.

We do also put IP addresses being used as websites on the watchlist as opposed to the numbers watchlist, which is geared towards phone numbers rather than any other type of number. If it's blacklistable, we would also promote it to the blacklisted websites list rather than the blacklisted keywords or numbers list.

@ThomasWard Yeah, it's some thing that should work itself out for SO, which, currently, is the primary one which will be trying to scan the same post in more than one thread, because it gets all active, rather than specific posts. It should work it self out, because SD fetches active posts from SO a lot and rescans them if they've actually changed from the last version which was scanned. Thus, it's unlikely that we wouldn't try scanning the post again later. OTOH, it is something that needs to be better accounted for when we switch over to getting all active posts from each site, as those sites are not fetched/scanned nearly as often.

@Spevacus Alright. The reason I was wondering is because I would imagine the IP address is fairly trivial to change, so I'm not exactly sure how often that exact one would crop up.

But also I guess it's fairly trivial to add to the watchlist so it probably doesn't matter too much.

@Makyen "should work itself out" is not a "will work itself out"

:P

@Jesse We've (well, others, I wasn't involved in the convo) chatted a bit in the past about whether we should be considerate of the size our watched_keywords list is becoming, and whether or not we should start pruning items from the watchlist and/or being more mindful of what we add. The last time this came up is, IIRC, when we noticed that we couldn't preview the watched_keywords.txt file on Github anymore as it was too large.

We decided that we shouldn't worry about the size of the list (well, within reason) so long as we're adding things that aren't likely to generate FP (in other words, the same thing we've been doing for awhile)

True.

So... Toss it on the watchlist. Maybe we'll see it pop up again 4 years from now. Maybe we'll never see it again. Safe than sorry, right?

Alright sounds good. Thanks

Thanks for participating as much as you have, even though you got privileges not too long ago. We greatly appreciate new folks coming into the fold. Don't get discouraged if you find that the way this place works has a decent learning curve. Firstly, you seem to be doing great, and secondly, we're all still learning new things here.

[ SmokeDetector | MS ] Potentially bad keyword in body (1): how with python can navigate after login?‭ by ernesto gevara‭ on stackoverflow.com

@Spevacus Yeah no problem. Gives me something to do when I get bored ;)

@ThomasWard Yes. However, we're fetching and scanning SO active posts on average every 25 seconds, or so. We'd have to go minutes between scans and the question would need to not have been active more recently. This, however, is possible in the case of very long scan times (which we have more often then would be desirable) and some edge-case grace period edits. We could, and probably should, re-increase the max time-span for which we obtain active posts, once SD is started.

fp feedback received on [MS] how with python can navigate after login?‭

@Spevacus @Jesse Basically, the numbers watchlist is used for numbers that might be obfuscated by the spammer, but are still intended to be read and understood by a human as a number. In other words, that the text that's in the spam will be translated through a human brain in order to determine the actual number. Phone numbers are the primary example of this, because spammers assume that there will be a human reading the spam and then typing the number into a phone.

That is, however, not the case for almost all URLs, because there usually isn't a human involved with translation (i.e. URLs are almost always just copied from the spam and pasted into the URLbar of a browser, or just clicked on). So, it's a machine (i.e. a browser) which needs to understand the URL text and browsers specifically don't do de-obfuscation of the URL for security reasons.

[ SmokeDetector | MS ] Potentially bad ip for hostname in answer, potentially bad ns for domain in answer, username similar to website in answer (72): Will my shelf be strong enough despite one of the brackets being in drywall anchors?‭ by Mora Contractor‭ on diy.SE

[ SmokeDetector | MS ] Potentially bad asn for hostname in answer (1): Regex Validation in Model Driven PowerApps‭ by Owen Wetherow‭ on stackoverflow.com

Would that second to last one be classified as spam? It answers the question, but there is an obvious self-promotion link that is unrelated to the question. I edited the question to remove the link, just waiting for it to be approved. Should I also flag it as spam and tpu-?

Makyen/MS-Relay: Executing automatic scheduled reboot.

[ SmokeDetector ] SmokeDetector started at rev 6b79b1731 (Makyen: Ping Makyen for some new metasmoke users) (running on Makyen/MS-Relay, Python 3.11.0)

On the "How to not be a spammer" page it does explicitly state "Don't include links except to support what you've written", so I'm starting to lean more towards "yes".

naa feedback received on [MS] Regex Validation in Model Driven PowerApps‭

tpu- feedback received on [MS] Will my shelf be strong enough despite one of the brackets being in drywall anchors?‭

[ SmokeDetector | MS ] Potentially bad keyword in body (1): public adress blocked by firewall/NAS of internet provider‭ by Rafał Pasternak‭ on stackoverflow.com

fp feedback received on [MS] public adress blocked by firewall/NAS of internet provider‭

[ SmokeDetector | MS ] Link at beginning of body (41): How to upgrade the libxml2 version‭ by EXEC1‭ on stackoverflow.com

fp feedback received on [MS] How to upgrade the libxml2 version‭

[ SmokeDetector | MS ] Bad pattern in url body (84): TLS v1.2 connection problem in Teams app on Ubuntu‭ by tux-friend‭ on stackoverflow.com

fp feedback received on [MS] TLS v1.2 connection problem in Teams app on Ubuntu‭

API quota rolled over with 67507 requests remaining. Current quota: 79999.

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): How to remove words from a file‭ by Oberluz‭ on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): ServiceNotFoundException for "spec_gatherer" on CiviCRM 5.55+ when hosting with Kinsta‭ by Jeremy B‭ on civicrm.SE

[ SmokeDetector | MS ] Potentially bad asn for hostname in body, potentially bad keyword in body (2): Changing a store bought LED power supply's output‭ by Djelle‭ on electronics.SE

fp feedback received on [MS] ServiceNotFoundException for "spec_gatherer" on CiviCRM 5.55+ when hosting with Kinsta‭

fp feedback received on [MS] Changing a store bought LED power supply's output‭

[ SmokeDetector | MS ] Link at beginning of body (41): Is this code running in parallel or concurrently?‭ by jsp2000‭ on stackoverflow.com