In 90% of cases, we do respect the backoff. In the other ~10% though there is a race condition (which we confirmed); adding a lock might actually be the correct solution there, I'm not sure.
It's not the resources, it's just that locking anymore than we have to defeats the purpose of multiple threads. Although, given we're using threading, not sure that's too important...
Ideally, all network stuff would get fired from a single thread which can maintain state and respect backoffs. Since we can't do that, a lock is the next best option to ensure that backoffs stay respected by making sure only one thread at a time gets at network stuff.
@Andy Where we respect a backoff, then promptly slam it with n requests as soon as the backoff expires. Then the API freaks out, sends a backoff on the first one, and the other ones promptly violate it.
Yeah, that is the other, actual, problem. I was thinking of a way to fix that one without a lock, just by increasing the time slept to the time of the backoff.