Ahh... Yeah. A while back, I change the CI testing to use threading to run multiple tests in parallel. CircleCI has more CPU cores, so uses more threads. This makes the issues more likely to appear on CircleCI. The failures and subsequent investigation indicated there are substantial threading issues in SD and our dependencies. I have merged a couple of PRs with some fixes and have another PR pending, which should help with some of the issues, but definitely not all.
However, in the last little bit, I've taken a deeper dive into the regex package code and found that what I'd done to alleviate problems will have helped, some, but definitely won't have solved the fundamental issues in that package. I'm currently thinking about/kicking around different possible methods of avoiding the fundamental issues which it has in its core C code.
teward/Osiris: In getting MS post information, recovered from requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='metasmoke.erwaysoftware.com', port=443): Read timed out. (read timeout=10.0)
@Makyen Oooohh, more messages to grammatically nitpick! ;-)
@Makyen I'd say let me know if you need any help understanding or fixing issues in C code, but... at the moment, I probably have even less spare time/bandwidth than you do. :-( Happy to brainstorm or discuss possible solutions, though, as time permits, without any specific committments/obligations. I can't recall what your familiarity and/or comfort level is in C.
[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, potentially bad ns for domain in body, potentially bad keyword in body, potentially bad keyword in username (195): What do you know about antennas by fernanda on electronics.SE
[ SmokeDetector | MS ] Bad ip for hostname in answer, blacklisted username, blacklisted website in answer, potentially bad ns for domain in answer, blacklisted user (353): How to join vCards lines by Roger Watson on unix.SE
@JeffSchaller That pattern looks like it's already caught by Blacklisted website in answer and Blacklisted website in body Append -force to the command word(s) if you really want to add the pattern you provided.
@Makyen That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body Append -force to the command word(s) if you really want to add the pattern you provided.
@CodyGray Thanks. I may take you up on the offer. In the past, I've done a reasonable amount of work in C, but it's been a quite a while since I actively worked in it. I'm frustrated, rather than stymied. I thought I'd worked around any possible lower-level threading issues by creating a copy of every compiled regex object to separately use in each thread. That seemed potentially overkill (and the testing results indicated minimal effect), so I was investigating if copying was necessary.
What I found was that, yes, some current state information is kept in each individual regex data structure, so using the same one across threads is ... not a good idea. In other words, one way or another we need to not be using the same compiled regex data structure at the same time in more than one thread.
I also found that the regex package specifically implements both __copy__ and __deepcopy__ methods, which Python uses to create copies of the data structure, by just increasing the Python reference count on the original data structure and returning a reference to the original (i.e. my making and using copies was useless, which does explain the testing results). I'd really rather that they'd just not implemented any copy function, rather than faking it.
There are a variety of ways to resolve it. The most obvious of which is to just implement a __deepcopy__ method in the C code which actually makes a copy, rather than merely returning a reference to the same data structure without making a copy.
Doing that requires learning more about CPython and how it handles creating, destroying (garbage collecting), tracking, and handing-off data structures between C and Python. Also needed is to try to figure out why the fake-copy was implemented rather than a real copy (i.e. is there something relying on fake copy returning a reference to the original rather than an actual copy; this is mostly because I can't easily see a legit reason to make a fake copy routine fully public as "copy").
While doing that might be fun/interesting, it takes time, which is not something in abundance.
An alternate solution would be to fully investigate the internals of the regex code and design-out the storage of state information in the base regex data structure (i.e. no writing to the compiled regex once it's compiled), which would make it safe to use the same compiled regex across threads. This would probably be ideal, but likely the most work.
A probably reasonable hack, which potentially resolves this for us, is to use the pickle package to serialize/deserialize the regex, which is supposed to be supported by the regex package. Doing so does create a copy of the underlying regex object. This should be usable as a quick, easy hack while more permanent options are explored. I've implemented this, because it's quick and easy, but it still needs to be fully tested in combination with other changes.
I'll know in a while if the pickle hack resolves the issue, at least for us (currently tracking down an issue in the larger set of changes I was testing; it's not clear if that's a result of the pickle hack, or not). If it does resolve issues for us, then it should substantially lower the priority of having a "real" or "right" solution, depending on the impact of any secondary effects.
I agree that orcid.org isn't something we want to watch unless it becomes much more common in spam. It's a perfectly legitimate site which has just been used by a few spammers.
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 16:22:08 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions/31220;31221;31222;31223;31224
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 16:22:30 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions/78223
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 16:22:44 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions/313154
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 16:22:59 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions/1725971
The quota_remaining property was not in the API response. Error: Violation of backoff parameter at 16:23:13 UTC. Backing off on requests for the next 12 seconds. Previous URL: https://api.stackexchange.com/2.2/questions/4470422;4470423;4470424;4470425;4470426;4470427;4470428;4470429
@Ethan If it's just low quality, then it's FP. Beyond things which are "nuke this on sight", there's only a relatively small amount of stuff which gets TP. A large portion of that is described in Feedback Guidance.
