« first day (1200 days earlier)      last day (2603 days later) » 

10:00 PM
Like if that was the issue we'd notice with automatic reporting surely
 
Indeed
But there has to be something different
 
J F
@DavidPostill It might still be useful since nobody’s actively watching the post for comments.
 
@JF True.
 
@Undo Well, the one difference I can think of is that it needs to retrieve the person's name to say who reported it. But that's not much
 
That's a start
I can't think of anything else
 
10:02 PM
what I'm tempted to do is trace through the thing, and determine what the state of the websocket is after a !!/report
but I want to bring up Overlord for that first :P
!!/standby Aurora
 
There's quite a bit the report command does. Checks if it can post, performs at least one api call, checks if it's already been posted, then posts to appropriate rooms
 
Those wouldn't cause the chat socket to close
 
J F
Could a mod undelete this room please?
 
Undo's Raspberry Pi received failover signal.
Restart: API quota is 9991.
 
10:05 PM
done
 
!!/standby Undo
 
Oh no it's happening again
 
[ SmokeDetector ] SmokeDetector started in standby mode at rev 6528713 (Undo1: Exclude codegolf from shortened URL check) (running on Undo's Raspberry Pi)
 
@quartata no it's not stop assuming that
or i'll smack you
 
...jeez, sorry. Just joking.
 
10:06 PM
that joke gets annoying after the third time.
!!/location
 
Restart: API quota is 9991.
@ThomasWard teward/Overlord
 
alright now someone just has to !!/report something
 
Snag one of my posts anywhere except SO and AU
 
i'm lazy, someone else call it :P
 
10:07 PM
 
@SmokeDetector MS API returned empty response.
 
oops I thought I turned you off
 
heh
 
Happening when there's no feedback, seems like
 
10:08 PM
@Undo The reporting thing?
 
@quartata Empty MS API response
 
@Undo Oh, OK.
 
10:09 PM
hmm
 
fp- by DavidPostill
 
I guess I'll have to wait for the case where it doesn't send anything :P
 
@ThomasWard We should probably delete that post from MS too
 
@ThomasWard Thought: We're doing a ton of stuff on the chat thread. Wouldn't surprise me if blocking that thread for that long is messing something up
 
If that's possible
 
10:09 PM
@quartata I'll do it in a few
 
@quartata ask @Undo
 
Anyways looks like this has officially entered the Heisenbug realm
 
fp- by Floern
 
@quartata We've had quite a few of those.
Some of which I've found squishworkarounds :p
 
10:10 PM
Is report failing when certain sites are reported?
 
i'm not sure what the "it died" criterion were
 
@Undo I thought .watch was its own thread in CE?
 
if it were one that triggered any type of exception it'd blow up and restart.
 
Like the callback is called in a new thread
 
@quartata Not sure
 
10:12 PM
[ SmokeDetector | MS ] Manually reported question: What happens to frame when reached switch and router? by dargemir on superuser.com
 
foo
bar
bah
sd f
 
Does a superuser report post to multiple rooms?
 
Don't think so
foo
 
Oh?
Looks like something happened
 
@Undo Post 1: Could not find data for this post in the API. It may already have been deleted.
 
10:13 PM
Oh.
 
well, kinda
 
@quartata breakpoints
 
Let's test one that does report to two rooms, but let the other room know
Or, report from that room and see if it posts there and here
 
I assume an SO post would do the trick? (SOCVR)
 
@quartata Overlord is a purely debugger system - it doesn't have the niceness that Aurora does, so there's breakpoints throughout that trigger during a !!/report call so I can see what happens :P
@quartata we should give them a heads up if we're doing heavy tests
DAMN YOU LAN NETWORK!
brb
 
10:14 PM
@Undo Bah, of course. LEFT JOIN vs INNER JOIN, again.
 
SOCVR will work.
 
@ThomasWard All right.
@ThomasWard Gotcha.
Did it hit anything of note?
 
left join feedback and it'll be fine
 
o.O no breakpoints called
 
10:14 PM
!!/alive?
 
So it just disappeared?
 
oh look chat died off
@Undo looks it.
that's what happened
 
Before or after the message came in?
 
Huh.
 
More breakpoints!
 
10:15 PM
@Undo died right after
 
Line?
 
but no errors or exceptions
@Andy never triggered a breakpoint
 
Mixing breakpoints and threads is a nuisance.
 
@quartata not when you have it suspend everything :P
 
@Andy Off the top of your head, anything in chat command processing code that would ever fail silently?
We could be locking something somewhere
 
10:16 PM
Not that I can think of. I'm on mobile so can't do much more than glance at github
 
k, thanks
 
well Overlord just blew up but I accidentally kill -9'd the debugger so
relaunching
 
@ThomasWard You might throw a couple breakpoints in check_permissions (:31)
Could be that we've got something odd in privilege caching
 
Restart: API quota is 9957.
 
10:17 PM
@Undo possibly, I've got a breakpoint at the first line of it, so it'll be step-through by hand now
just ping me before each run
because I really don't want to have to keep hitting "resume" :p
 
@Undo I think if that was the case we'd notice it with reports that only go to one room too
 
you'd think, yeah
 
Then again I'm not really qualified to talk about this so
 
okay, here we go @ThomasWard...
 
[ SmokeDetector | MS ] Manually reported question: How can I collect block values in Ruby without an iterator? by Sasgorilla on stackoverflow.com
 
10:19 PM
got past priv checks without issue
 
...I was about to say "at least it's consistent"
except it's not
 
1 moment I have to VPN my other work laptop in for something
 
Out of curiosity, did the first report make it into MS?
 
wouldn't have, no
 
Hmm, you reported two different posts.
 
10:20 PM
Needs to get pretty deep before that gets triggered
@quartata Aye
 
Could something be happening when retrieving the post?
 
Super skeptical that there's anything about post content there, but I guess it's possible
 
I mean it doesn't make that much logical sense but when do bugs ever
 
fp- by Mithrandir
 
oops thread death 1 moment
 
10:22 PM
 
Another test: report a post and then post a second message before smokey would respond. Perhaps the second message is messing with the socket before we are done with the first message
 
that was my fault i accidentally nuked a thread.
@Andy much harder to test/track
 
Restart: API quota is 9930.
 
@Andy Tried to do that earlier, chat ratelimiting was getting me
 
Why? You have it suspending :)
 
10:23 PM
Okay, reporting the first SO post in ten
 
@Andy true, but it's the same breakpoint and I don't get a debugger per thread
 
not really
stepping
 
[ SmokeDetector | MS ] Manually reported question: Using compiler directive to warn if a file is missing by SpaceDog on stackoverflow.com
 
evil one that wasn't erroring past the permissions
so i just resumed it
 
10:25 PM
Hm. Well, that was my good idea for the hour. Please wait 36 minutes for the next good idea to arrive
4
 
@Undo fun fact the thread still cached all the chat msgs lol
just got a small flood of them in stdout
 
fp- by DavidPostill
 
can we smack people
 
That's good, then
 
@ThomasWard why do you want to?
 
10:26 PM
not the smacking people, the flood of chat messages
 
@ThomasWard now
 
yay lag
well this has been fun, it hasnt really died yet lol
at least not with any errors
I guess we'll just wait :P
 
10:45 PM
Is the FRIGGING REPORT COMMAND fixed yet?
Oh yay
Seems fixed
 
sorta
we've only had it die off once, and it didn't even die with an error
 
So, what else is broken?
 
oh, you know, the usual
politics, the economy, our sanity...
 
teward/Aurora received failover signal.
 
and Overlord just died lol
 
10:48 PM
Restart: API quota is 8526.
 
!!/location
 
@ThomasWard teward/Aurora
 
good
i have a headache so i'mma take a short nap.
back later
 
Have a good nap
 
J F
11:19 PM
!!/alive
 
@JF Of course
 
J F
!!/queuestatus
 
buddhism.stackexchange.com: 1
monero.stackexchange.com: 1
hermeneutics.stackexchange.com: 1
ru.stackoverflow.com: 6
arduino.stackexchange.com: 1
ja.stackoverflow.com: 1
opensource.stackexchange.com: 1
opendata.stackexchange.com: 1
matheducators.stackexchange.com: 1
mythology.stackexchange.com: 1
blender.stackexchange.com: 2
judaism.stackexchange.com: 1
graphicdesign.stackexchange.com: 1
sitecore.stackexchange.com: 1
codegolf.stackexchange.com: 4
hinduism.stackexchange.com: 1
gamedev.stackexchange.com: 1
 
11:46 PM
[ SmokeDetector | MS ] Repeating characters in body, repeating characters in title: oracledddddddddddddddddddddddddddd by Lee Johnny on stackoverflow.com (@dorukayhan)
 
@SmokeDetector v
 
@SmokeDetector v
 
@ArtOfCode Did you push the fix for the API query thing?
 
I didn't, no
 
11:56 PM
@SmokeDetector Hrmh. This reason isn't doing so good
 
fp- by Floern
 
The problem is that it is catching a lot of "spam" in the sense of personal blogs that are borderline
 

« first day (1200 days earlier)      last day (2603 days later) »