@ThomasWard So, I was working on getting my SD instance running on Python 3.11, due to the notable performance improvement, which my instance could certainly use. An incompatibility with Python 3.11 got me looking at tasks.py. [It's currently not compatible with Python 3.11, due to the removal of the @asyncio.coroutinedecorator, which has been deprecated since 3.8; I'll be fixing it.]
Looking at tasks.py, I was reminded that there's a specific capability to define a periodic task, rather than have a Tasks.later() have to set up a delayed call to itself. You can use Tasks.periodic(). The ws.py file has a few examples of using it, such as Tasks.periodic(Metasmoke.send_statistics, interval=600).
I should probably note that if it's something you think might block for a while, then it might be a good idea to use its own thread.
Alternately, I've been intending to adjust the Tasks class in tasks.py to change it such that it instantiates a new tasks thread for each instantiation of the class. Doing so would allow each module to have a separate tasks thread for itself, which would remove the possibility that some module adds a task that blocks execution of tasks created by other modules. While doing that should be relatively easy, I haven't worked on it, yet.
@Makyen thanks for the info. Yeah i'm not hunting toooo hard on this, but if we do need to do periodic tasks like writing to file, etc. then we can look at it there. i'm still going through some mongo db courses I got cheap on udemy (black friday whoooo!)
@Makyen there's obviously a hundred ways to do the same thing, but yeah this is something we'll have to coordinate on. When/how do we write the pickles now, and do we have any cases where the pickle is not written but data might need to be synced (a-la not on shutdown or reboot of SD)?
and if not, would we need to create a dump function that locks/unlocks briefly just to make sure we have an updated pickle on a regular basis
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 06:48:45 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 06:48:59 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 06:49:14 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 06:49:28 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 06:49:57 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
@SmokeDetector The MS report has the body intact, although I can’t see if that title was always a mess or if they for some reason vandalized just the title before deleting it (I’m under 10k and can’t see deleted posts)
[ SmokeDetector | MS ] Bad keyword in body, bad keyword in link text in body, bad keyword in title, blacklisted website in body, potentially bad ns for domain in body, +3 more (364): Plumbing Service in Kolkata by Prism Lead India on superuser.com
[ SmokeDetector | MS ] Few unique characters in body, no whitespace in body, no whitespace in title, potentially bad keyword in body, repeating characters in body, +1 more (391): aaaaaaaaaaaaaaaaaaaaaaaaa ✏️ by Mike Joseph on stackoverflow.com
[ SmokeDetector | MS ] Few unique characters in body, no whitespace in body, no whitespace in title, potentially bad keyword in body, repeating characters in body, +1 more (391): aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ✏️ by Mohamed on stackoverflow.com
[ SmokeDetector | MS ] Few unique characters in body, no whitespace in body, no whitespace in title, potentially bad keyword in body, repeating characters in body, +1 more (391): aaaaaaaaaaaaaaaaaaaa ✏️ by Sachdev on stackoverflow.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 14:53:47 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 14:54:01 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions?site=stackoverflow.com
@Ollie That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body Append -force to the command word(s) if you really want to add the pattern you provided.
@Ethan That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer Append -force to the command word(s) if you really want to add the pattern you provided.
[ SmokeDetector | MS ] Link at end of answer, potentially bad asn for hostname in answer, potentially bad ns for domain in answer, potentially bad keyword in answer, username similar to website in answer (134): How to store a custom string inside the BIOS by APKSPIRAL on superuser.com
@Ethan That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body Append -force to the command word(s) if you really want to add the pattern you provided.
The frustrating thing is, you can't tell if the actual thing you're trying to watch is already watched, or if it's just a false positive because we are basically watching everything containing "apk". :-)
sdc report stackoverflow.com/a/74582526 "Undisclosed affiliation with the linked channel: Video was posted just four hours ago, and the channel's profile picture has "Pradeep" on it, which matches the username here. In addition, post is VLQ (link-only). Custom flagged."
@SmokeDetector Currently, this is just off-topic and not really a question. It's probably failed spam where the user forgot to include some type of contact information, even in the Markdown.
@Ethan That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer Append -force to the command word(s) if you really want to add the pattern you provided.
Up to you, but consider changing that one to something like `ultimate[\W_]*+hacker[\W_]*+jerry` as they left a comment containing "Contact ULTIMATE HACKER JERRY"
sdc reject 7364 "Domains that have been seen in spam, meet the MS criteria, and have a relatively low FP rate can be watched. They'll also catch more spam than a more specific watch such as this one."
@Ethan That watch is quite specific. True, that is the only hit in MS (for now), but there's no reason not to watch the domain. It's not like a hotmail watch, where it's obviously legit.
@Ethan That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer Append -force to the command word(s) if you really want to add the pattern you provided.
@Vickel I’d go with off site resource. Their profile doesn’t indicate affiliation and they include more than just the link. While it is a poor question, I’d wouldn’t call it spam, despite the link
@cocomac I helped closing it as off-Site resource, BUT: it is very easy to hide a spam link in a disguised how do I scrape whatever question. Hence my comment to SD and this room
