54585

[ SmokeDetector | MS ] Repeating characters in answer: NEED HELP NOW!!! CAN'T restore ROM! by Cool Nandroid 4 Life on android.stackexchange.com (@AndrewT.)

naa- by Gothdo

Backoff received of 10 seconds on request to questions/78790?site=travel at 22:35:24 UTC

Wonder if we could just space those out in the first place.

Just thinking out loud, don't shoot me

I'm not, strictly speaking, against, but let me note that it might adversely affect the quality of moderation provided by this room.

@JanDvorak We're probably talking single-digit seconds, no more.

oh

still... won't they back up?

yeah, that'd be the issue

I still think just respecting the backoffs would work best :P

But I got shot down last time and no one explained why it was a bad idea.

In 90% of cases, we do respect the backoff. In the other ~10% though there is a race condition (which we confirmed); adding a lock might actually be the correct solution there, I'm not sure.

It might not be perfect from a resource perspective, but... we're not going to max out that Pi.

How come there's no lock yet?

It's not the resources, it's just that locking anymore than we have to defeats the purpose of multiple threads. Although, given we're using threading, not sure that's too important...

@JanDvorak github.com/Charcoal-SE/SmokeDetector/commit/…

All we need to do is un-revert that and we're good to go, if this is actually the problem

Threading can help with the CPU-intensive parts (regex testing). Trying to use threading to bypass network limitations is a bad idea.

@JanDvorak Exactly.

Maybe if we move the backoff code up, not sure. But having a 30 LOC lock is not a good idea.

^ That. I say do it.

@hichris123 yeah, that's an easy fix.

Not like any of those 30 are particularly gnarly, though

@JanDvorak We're using threading, so we aren't really doing it for the CPU reasons. ;) Mostly just so the main thread doesn't stall.

Although I would like to switch to multiprocessing sometime. It's just having the time to switch and iron out all the issues...

You can have one thread to chat with Charcoal and one to chat with the API

Ideally, all network stuff would get fired from a single thread which can maintain state and respect backoffs. Since we can't do that, a lock is the next best option to ensure that backoffs stay respected by making sure only one thread at a time gets at network stuff.

@JanDvorak Our thread layout is much more complicated than that... :P

CE is already async

That doesn't sound good

@hichris123 Not really. We just shunt everything off from the main thread as soon as possible

@JanDvorak Eh. It's just basically a new thread is spawned for a chat message and every new post.

Usually on a per-post basis.

@undo Right, pretty much.

@hichris123 that doesn't sound efficient OR manageable.

We have to keep the main thread open for incoming websocket messages. That's pretty much the only reason we even do it.

Thread pool?

@JanDvorak It... is? We're just spawning threads, not processes.

Even spawning a thread takes resources

@JanDvorak I wouldn't have thought so, but we've got about two years of evidence that says otherwise.

I was running this on a toaster for about eighteen months.

@JanDvorak Not much, just fork() I believe.

Virtual, hyperthreading threads, not the physical threads that your CPU datasheet says it only has 8 of.

@undo That's because you don't spawn hundreds of threads per second

Exactly, and we never will.

We could spawn thousands of threads and not run out of resources

Hence performance doesn't matter?

You seem to be running out of manageability, though ;-)

We use very little CPU. At that point, it doesn't matter.

@JanDvorak It does matter, but only if it's likely to be or contribute to a Real Problem.

@JanDvorak Not really. ;) Just a simple race condition.

Real Problems include, but are not limited to: (1) Smokey stops working, (2) my Pi combusts.

Otherwise, even a few seconds of latency in processing are completely okay.

Combustion does not have to be spontaneous.

any type of combustion is undesirable.

@hichris123 where are the race conditions?

@Andy Still those API things

Internal? I hear they make some pretty good engines based on that kinda combustion.

@Andy Thread 1 has backoff, thread 2 sends request 2 before thread 1 reports backoff to thread 2.

@Andy Where we respect a backoff, then promptly slam it with n requests as soon as the backoff expires. Then the API freaks out, sends a backoff on the first one, and the other ones promptly violate it.

Yeah, that is the other, actual, problem. I was thinking of a way to fix that one without a lock, just by increasing the time slept to the time of the backoff.

Restart: API quota is 2975.

sd f

[ SmokeDetector | MS ] Link at end of answer: C int to char converstion adds garabge characters by azavalla on stackoverflow.com

Backoff received of 10 seconds on request to questions/48767;4212?site=music at 23:46:49 UTC

[ SmokeDetector | MS ] Pattern-matching website in answer: Showing a number is not an algebraic integer by user369038 on math.stackexchange.com

tpu- by Jan Dvorak

@SmokeDetector f