Charcoal HQ

Thomas Ward

3:00 AM

uhm

@quartata i have concern.

you wrote queue_timings right?

quartata

no

art did

Thomas Ward

ah

@ArtOfCode i shall slap thee

quartata

What happened?

Thomas Ward

he's referencing a variable before it has any initialization

also python 3 and not python 2

though for queue_timings there's no syntax differences

well, there is, but...

quartata

!!/errorlogs 50

sd del

Thomas Ward

3:06 AM

!!/errorlogs 50

SmokeDetector

  File "ws.py", line 277, in <module>
    a = ws.recv()

  File "/usr/local/lib/python2.7/dist-packages/websocket/_core.py", line 293, in recv
    opcode, data = self.recv_data()

  File "/usr/local/lib/python2.7/dist-packages/websocket/_core.py", line 310, in recv_data
    opcode, frame = self.recv_data_frame(control_frame)

  File "/usr/local/lib/python2.7/dist-packages/websocket/_core.py", line 323, in recv_data_frame
    frame = self.recv_frame()

  File "/usr/local/lib/python2.7/dist-packages/websocket/_core.py", line 357, in recv_frame

Thomas Ward

no that's OK to keep in place

and I was reading that so blah

quartata

You can see deleted messages anyways..

Thomas Ward

not from my tablet I can't.

hey at least the warning capture is not exploding heh

quartata

Wait really?

se chat pls

Thomas Ward

3:07 AM

@quartata not easy to find post logs from the mobile interface

:P

but now i have my laptop so im switching to that

@ArtOfCode cPickle replaced with pickle module implementation changes instead in Python 3.x

also you had a variable you'd be referencing that had no value so you'd explode

SmokeDetector

CI on f604590 succeeded.

Thomas Ward

so i had to update queue_timings.

and why Python3?

angussidney

@ThomasWard probably because Python 3 is newer, and since that code isn't actually run as part of the bot (and hence doesn't have any compatiabily issues), we may as well use the most up to date version

Thomas Ward

@angussidney then the fix I just uploaded has to be included for a run - cPickle will give an import error in most Python3

and there's still the "NoneType" object error I just poked to fix.

i'm not a fan of that but...

anyways time to keep poking the postdata part of the refactor heh

i'd love to test it, but... it looks like nothing's going on today

o.O

quartata

3:24 AM

Yeah it concerns me

Thomas Ward

what does

PostData() and Post() being pretty much the same object, except the second is far better handled / laid out?

or the fact we're not getting any Answers showing up?

Undo

What'd I break?

quartata

@ThomasWard The latter.

angussidney

@Undo Thomas's !!/blame system :P

Thomas Ward

@Undo my brain

quartata

3:26 AM

@Undo Nothing. Thomas apparently had a better getter for parent in another branch

Thomas Ward

because you stole my coffee.

@quartata actually i thought you made other changes, all's good :)

mmmmmmm brownie pizza

mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm

Undo

@ThomasWard This sounds hard to execute properly

Thomas Ward

@Undo basically dessert pizza with a metric ton of chocolate.

@Undo can you make sure Smokey is actually checking answers, via MS logs?

angussidney

And I was just about to say welcome back to @hichris123.....

Thomas Ward

i want to make sure that it's working as intended.

angussidney

3:28 AM

when he left

Undo

@ThomasWard No.

I have no data on that

quartata

I do have some cookie pie in the fridge but I can't say I have ever heard of a brownie pizza

Thomas Ward

hrm OK

oh i know what I can do

Undo

here, there's an easy solution

Thomas Ward

!!/standby ArtOfCode

SmokeDetector

3:29 AM

ArtOfCode/EC2 is switching to standby

[ SmokeDetector ] SmokeDetector started in standby mode at rev 13f9301 (ArtOfCode-: Ain't flake great) (running on ArtOfCode/EC2)

Thomas Ward

fuuuu hang on

SmokeDetector

Undo's Raspberry Pi received failover signal.

Restart: API quota is 19963.

Undo

!!/test-a quickbooks support number

SmokeDetector

> Bad keyword in answer
----------
Body - Position 1-26: quickbooks support number

[ SmokeDetector ] SmokeDetector started at rev f604590 (Thomas Ward: Python2/3 Reverse Compat functionality, also 'return' if EOFError) (running on teward/Overlord)

quartata

Overlord time

Thomas Ward

3:30 AM

!!/standby ArtOfCode

quartata

!!/rev

SmokeDetector

@quartata 066251b (angussidney: +products to pattern matching product name --autopull)

@quartata f604590 (Thomas Ward: Python2/3 Reverse Compat functionality, also 'return' if EOFError)

quartata

crap

Thomas Ward

eww

TypeError: list indices must be integers, not str

quartata

I thought I fixed that?

Yeah I did.

Thomas Ward

3:32 AM

doesn't look it from the master code base

quartata

Also it was string indices or something actually

Thomas Ward

!!/pull

@quartata well...

TypeError: list indices must be integers, not str
2017-03-18 03:31:00.626484 UTC
  File "/home/teward/VCS Repos/GitHub/Charcoal-SE/SmokeDetector/excepthook.py", line 48, in run_with_except_hook
    run_old(*args, **kw)
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/teward/VCS Repos/GitHub/Charcoal-SE/SmokeDetector/bodyfetcher.py", line 90, in add_to_queue
    self.queue[site_base][str(post_id)] = datetime.utcnow()

that's... in Master

so str(post_id) is exploding.

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev f604590 (Thomas Ward: Python2/3 Reverse Compat functionality, also 'return' if EOFError) (running on Undo's Raspberry Pi)

Thomas Ward

watch it explode here too

which would explain why things are blowing up

quartata

Oh not me. I assumed this was in FindInSpam since I got a similar error

earlier

!!/errorlogs

SmokeDetector

3:33 AM

  File "/media/sda1/SmokeDetector/ChatExchange/chatexchange/browser.py", line 311, in send_message
    {'text': text})

  File "/media/sda1/SmokeDetector/ChatExchange/chatexchange/browser.py", line 133, in post_fkeyed
    return self.post(url, data, headers)

  File "/media/sda1/SmokeDetector/ChatExchange/chatexchange/browser.py", line 113, in post
    return self._request('post', url, data, headers, with_chat_root)

  File "/media/sda1/SmokeDetector/ChatExchange/chatexchange/browser.py", line 102, in _request

Thomas Ward

!!/location

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev f604590 (Thomas Ward: Python2/3 Reverse Compat functionality, also 'return' if EOFError) (running on teward/Overlord)

Thomas Ward

well, there we go.

kablooey

Anonymous

So... Unit testing? :D

quartata

just disable queue timing

@Mego We do

SmokeDetector

3:35 AM

[ SmokeDetector ] SmokeDetector started at rev 13f9301 (ArtOfCode-: Ain't flake great) (running on Undo's Raspberry Pi)

Thomas Ward

!!/standby Undo

@quartata i'd rather fix the issue first

Anonymous

Unit testing should be able to catch trying to use a str as a list index before deployment and subsequent kablooey

quartata

Do we have tests for body fetcher actually

Thomas Ward

wow 404 error xD

Undo

@quartata that's a difficult thing to unit test

Thomas Ward

3:36 AM

@Mego not everything has unit tests.

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev a46f747 (ArtOfCode-: Shorten site names) (running on Undo's Raspberry Pi)

quartata

@Undo Particularly this part.

SmokeDetector

[ SmokeDetector ] SmokeDetector started in reverted mode at rev a46f747 (ArtOfCode-: Shorten site names) (running on teward/Overlord)

Thomas Ward

ahhhh i see what's going on...

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 33ab6fe (ArtOfCode-: Make analysis script python 2) (running on Undo's Raspberry Pi)

Thomas Ward

3:38 AM

i see what we need to do

We're expecting that the site ID list is in a dict

but it's really not actually doing that

it's creating lists of ints.

that is why queue timing fails

quartata

That commit is after the merge from time in queue by the way.

revert the pi to 92adc01

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev ec0b1fa (ArtOfCode: Merge pull request #598 from Charcoal-SE/time-in-queue) (running on Undo's Raspberry Pi)

Undo

it'll get there eventually

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 92adc01 (ArtOfCode: Merge pull request #599 from quartata/master) (running on Undo's Raspberry Pi)

quartata

there we go

SmokeDetector

3:42 AM

Restart: API quota is 19959.

Thomas Ward

hrm

@Undo I think that ArtOfCode failed to refactor this proper

i'll have to have an if statement to check the type of the item, and if it's not a dict convert to a dict.

this is going to be painful.

Undo

Honestly, if it's becoming work, then don't worry about it. This is a hobby project, no reason to be working on anything you don't want to

SmokeDetector

Restart: API quota is 19957.

Undo

!!/status

SmokeDetector

@Undo Running since 03:43:31 UTC (2 minutes)

[ SmokeDetector | MS ] Pattern-matching website in body: Map of neighbourhoods for Canada by Tom on gis.SE

Undo

3:48 AM

!!/errorlogs 100

!!/alive

Thomas Ward

i think something blew up

SmokeDetector

Restart: API quota is 19939.

Undo

I see it

AttributeError: 'dict' object has no attribute 'append'
2017-03-18 03:48:25.573244 UTC
  File "/media/sda1/SmokeDetector/excepthook.py", line 48, in run_with_except_hook
    run_old(*args, **kw)

  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)

  File "/media/sda1/SmokeDetector/bodyfetcher.py", line 85, in add_to_queue
    self.queue[site_base].append(post_id)

Okay, don't touch the code. I'm reverting some stuff

Thomas Ward

erm, you're four minutes too slow

@Undo revert my change too.

@Undo though I was going to try and fix all these issues

oh wait that's a one-line fix in addition to my code

want me to fix?

Undo

I'm just going to batch revert everything between 95ee2 and HEAD

After that, you can revert the revert locally and do whatever

Thomas Ward

3:51 AM

OK

@Undo this 'work' is something i want to do :P

so it's not really 'work' per se

also keeps my brain sharp :)

Undo

k. Just trying to head off burnout

SmokeDetector

teward/Aurora received failover signal.

Restart: API quota is 19999.

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link at end of body, pattern-matching product name in body, pattern-matching website in body: Did it is well known that there are amazing dietary supplements by kacasuj on meta.SE

Thomas Ward

LOL

or until i'm better

Undo

bah, git is hard

Thomas Ward

@Undo note that that 'append' isn't there no more. at least ,not in my code.

probably has something to do with the refactor/conversion

Andy

3:57 AM

@ArtOfCode After you're done figuring out what's going on up there ^ , here's that pickle run through panda's simple describe() and adding median

                     count        mean      median         std       min         25%         50%         75%         max
islam                  4.0  127.269991   74.985501  169.960406  0.000285    0.000376   74.985501  202.255116  359.108676
vi                     2.0    6.722596    6.722596    9.506857  0.000233    3.361414    6.722596   10.083778   13.444959
superuser              2.0    0.159724    0.159724    0.225348  0.000379    0.080051    0.159724    0.239396    0.319069
english                2.0  421.326939  421.326939  595.845718  0.000391  210.663665  421.326939  631.990212  842…

Thomas Ward

code format is useful @Andy

just saying ;)

Andy

Yeah yeah, it didn't format the first time

Thomas Ward

blame SE

Andy

Obviously, that's a small dataset you sent me since only a couple sites are in it and even fewer have more than two items in the list.

SmokeDetector

tpu- by teward

Thomas Ward

4:00 AM

!!/location

SmokeDetector

@ThomasWard teward/Aurora

Thomas Ward

!!/rev

SmokeDetector

@ThomasWard 158e861 (Thomas Ward: Slight refactor of chat logon handling.)

Thomas Ward

oh right that's not a recent pull

!!/standby Aurora

SmokeDetector

[ SmokeDetector ] SmokeDetector started in standby mode at rev 158e861 (Thomas Ward: Slight refactor of chat logon handling.) (running on teward/Aurora)

teward/ESXi received failover signal.

Thomas Ward

4:01 AM

seriously wtf Overlord

ValueError: invalid literal for int() with base 10: 'users'

!!/standby teward

SmokeDetector

teward/ESXi is switching to standby

Thomas Ward

osnap

SmokeDetector

[ SmokeDetector ] SmokeDetector started in standby mode at rev 066251b (angussidney: +products to pattern matching product name --autopull) (running on teward/ESXi)

[ SmokeDetector ] SmokeDetector started at rev 5c23a3f (Thomas Ward: Queue: List/Tuple replace with a Dict for proper functions.) (running on teward/Overlord)

Undo

Reverted a bunch of stuff. @ThomasWard you'll want to pull and revert 0177abf locally if you're trying to figure this out

!!/pull

aaaactually, maybe shouldn't have pulled that. Sorry if I messed up a debugging session

Thomas Ward

it's dead

it blew up with attribute errors later on

Undo

4:04 AM

Ah, okay.

Thomas Ward

@Undo did @ArtOfCode write this recent queue stuff

Undo

Which queue stuff?

Thomas Ward

the stuff you just reverted

Undo

Tentative yes

It's in the commit history

Thomas Ward

well permit me to yell at @ArtOfCode because lists and dicts aren't interchangeable objects.

and he's using lists everywhere for some unknown reason

SmokeDetector

4:05 AM

ArtOfCode/EC2 received failover signal.

Undo

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 0177abf (Undo1: Revert a lot of things) (running on ArtOfCode/EC2)

[ SmokeDetector ] SmokeDetector started in reverted mode at rev 5c23a3f (Thomas Ward: Queue: List/Tuple replace with a Dict for proper functions.) (running on ArtOfCode/EC2)

Restart: API quota is 18163.

Undo

oh come on

!!/errorlogs 100

SmokeDetector

    return self._request('get', url, data, headers, with_chat_root)

  File "/home/ubuntu/Smokey/ChatExchange/chatexchange/browser.py", line 105, in _request
    time.sleep(0.1)


WARNING: Cannot update TLD names, due to `tld` being system-wide installed and not user-level installed.  Skipping TLD names update.
KeyboardInterrupt
2017-03-18 00:33:08.514255 UTC
  File "ws.py", line 277, in <module>
    a = ws.recv()

  File "/usr/local/lib/python2.7/dist-packages/websocket/_core.py", line 293, in recv

Undo

!!/status

SmokeDetector

4:08 AM

@Undo Running since 04:06:36 UTC (1 minute)

Undo

Stable now?

Maybe reverting wasn't necessary after your last fix

Thomas Ward

@Undo it'd fail later on with other attribute errors

the problem is dict vs. list

and they're really not interchangeable.

Undo

yeah

So when was that introduced?

Andy

@ArtOfCode, I'm gone all Saturday and most of Sunday. I'll poke again Sunday night. If you have ideas on what I should look at/for, let me know.

Thomas Ward

sometime since i went offline.

Andy

4:09 AM

@Undo A few hours ago

Undo

Ohhhhh

Thomas Ward

@Undo it was art of code who introduced all those changes

so beat him over the head

Undo

I know

Small thing.

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 0177abf (Undo1: Revert a lot of things) (running on Undo's Raspberry Pi)

Andy

here: chat.stackexchange.com/transcript/message/36114766#36114766

SmokeDetector

4:12 AM

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link at end of body, pattern-matching website in body: Rocky Balboa Diet And Weight Loss Plan by Nitin Prakash on askubuntu.com

Restart: API quota is 19999.

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link at end of body, pattern-matching website in body: Rocky Balboa Diet And Weight Loss Plan by Nitin Prakash on askubuntu.com

Undo

yeah, so @ThomasWard we'll just need to nuke bodyfetcherQueue.p, then current rev should work fine

!!/location

SmokeDetector

@Undo ArtOfCode/EC2

@Undo Undo's Raspberry Pi

tpu- by angussidney

Undo

!!/standby art

SmokeDetector

tpu- by angussidney

Thomas Ward

@Undo by 'current rev' we mean which?

SmokeDetector

4:13 AM

[ SmokeDetector ] SmokeDetector started in standby mode at rev 5c23a3f (Thomas Ward: Queue: List/Tuple replace with a Dict for proper functions.) (running on ArtOfCode/EC2)

Thomas Ward

my last push minus your revert?

Undo

@ThomasWard HEAD on master

SmokeDetector

[ SmokeDetector | MS ] URL-only title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +1 more: menintalk.com/zilotrope/ by gedwsctysedx on askubuntu.com

Thomas Ward

because that'll need changes too

Undo

Including the revert. It may also work without the revert, haven't tested

SmokeDetector

4:13 AM

tpu- by Undo

Thomas Ward

@Undo I'll test local

Undo

Sweet

Andy

And pulled in around here-ish

Thomas Ward

since I didn't merge your revert yet on Overlord

@Undo for queue timings, if we have a failure case, what do we want a 'default' queue timing to be, -1? 0? 999999999999? Infinity?

(there will be failure cases, which is unfortuante but eh)

Undo

@ThomasWard Might as well not record it

Andy

4:15 AM

@ThomasWard -1

Thomas Ward

@Undo I can set it to None, or just skip over.

Undo

@ThomasWard ~~Just skip over~~

Noooooooooooo

I defer to Andy

i'll put -1 in place

4:16 AM

Actually, I take that back. First: What is this failure from?

Failure to input into the queue? Or failure from the API?

Undo

Pretty sure it'd be failure from the API, or from parsing something from the API.

Andy

Oh. Then yes, skip it.

SmokeDetector

[ SmokeDetector | MS ] URL in title, blacklisted website in body, blacklisted website in title: purelifegreencoffeebeanadvice.com/vitaClara/ by Zilpyf on graphicdesign.SE

tpu- by Undo

Undo

Well, I guess we're processing stuff

Andy

Over reacted with that wilhelm scream.

Thomas Ward

4:19 AM

@Andy the 'failure' would be where there's no 'add time' to compare with

which can happen in empty cases.

Andy

empty?

Thomas Ward

post_add_times = [v for k, v in new_posts.iteritems()]

if there's nothing in iteritems() here it's an empty list

Undo

But that should be rare, right?

Thomas Ward

@Undo yes, but i want to be totally sure :P

Undo

Yeah. If we have unusable data, I'd vote to skip it.

Andy

4:21 AM

Agreed

Thomas Ward

let me debug-run this as it was, so I can examine what data's actually in here

though I did just nuke all my pickles so probably nothing :P

Undo

@ThomasWard This'd be a good time to test the no-chat argument, if you're interested

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 4d7620c (Thomas Ward: Better handling of queye timing items, try/except captures) (running on teward/Overlord)

Thomas Ward

!!/location

SmokeDetector

@ThomasWard Undo's Raspberry Pi

@ThomasWard teward/Overlord

Thomas Ward

4:23 AM

@Undo shortly

!!/standby Undo

Undo

k

Thomas Ward

i want it to emulate a normal run first

SmokeDetector

Restart: API quota is 19991.

[ SmokeDetector ] SmokeDetector started in standby mode at rev 0177abf (Undo1: Revert a lot of things) (running on Undo's Raspberry Pi)

Thomas Ward

i think there was an explode

Undo

Everything looks normal from chat, but you know more than I do from your debugger

(afk)

Thomas Ward

4:25 AM

yep

it breakpoints on certain errors being raised

so i see all the error failures

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 4d7620c (Thomas Ward: Better handling of queye timing items, try/except captures) (running on teward/Overlord)

Thomas Ward

hence why it's called a debugger

SmokeDetector

Restart: API quota is 19989.

Thomas Ward

@Undo OK so if we nuke the existing pickle for the queue first, it properly has list objects without me messing with the code.

and that solves both the append error we had beofre which I refactored out

and other attribute errors later on.

so we need the instances to erase the pickles. Which means we need to kill Art's EC2 until he can do that manually so it doesn't come back online.

Or implement a one-time kill code for the queue pickle file.

@Undo so, how do we want to handle that?

SmokeDetector

4:41 AM

teward/Aurora received failover signal.

Restart: API quota is 19981.

Undo

@ThomasWard This'd probably be the best way to do it, yeah

Thomas Ward

OK I'll push that to ws.py after I test my revert of your revert plus my changes, with a clean pickle

... that sounded weird

Undo

Thanks for digging into that, good that it's a simple fix

Thomas Ward

eh not so simple, but we convert the general 'list' to a dict so we can use it earlier on which is what we were blowing up with

but we still need to nuke existing pickles and start again.

@Undo so what i'm going to do is this:

once i test this code and it works

i'm going to push to ws.py to delete that pickle file if it exists.

then cycle all the instances to pull in the new ws.py and also execute it.

then pull that 'kill the pickle' out.

and also cycle to everything and pull ws.py

that's a painful process but not as painful as fixing things

Undo

Yep, sounds sane

Thomas Ward

4:45 AM

any objections except for the massive standby/active/standby/active chain that'll happen?

actually

just turn them all to active

then pull

then revise and pull again

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, pattern-matching product name in body, pattern-matching product name in title, pattern-matching website in body, +1 more: Alpha monster advanced by gysw apos on apple.SE

Thomas Ward

MASS OVERLOAD

Undo

@ThomasWard Nope

Thomas Ward

nah i'll do it piecemeal

!!/standby teward/Aurora

SmokeDetector

[ SmokeDetector ] SmokeDetector started in standby mode at rev 158e861 (Thomas Ward: Slight refactor of chat logon handling.) (running on teward/Aurora)

Undo's Raspberry Pi received failover signal.

Undo

4:46 AM

Easy enough to failover all of them, then !!/pull

Thomas Ward

yup

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev bad35f0 (Thomas Ward: Revert "Revert a lot of things") (running on teward/Overlord)

Thomas Ward

!!/standby Undo

i haven't committed my latest changes yet

SmokeDetector

[ SmokeDetector ] SmokeDetector started in standby mode at rev 0177abf (Undo1: Revert a lot of things) (running on Undo's Raspberry Pi)

Thomas Ward

so i just need it to wait for a few

there we go it's got queue items now and NO ERRORS

waiting for the timing stuff to hit

SmokeDetector

4:48 AM

Restart: API quota is 19952.

Thomas Ward

... explosion

SmokeDetector

[ SmokeDetector | MS ] URL in title, link at end of body, pattern-matching website in body, pattern-matching website in title: platinumcleanserinfo.com/laveaderm/ by ewancollins on meta.SE

tpu- by angussidney

Thomas Ward

looks like I fubar'd a type check oops

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev bad35f0 (Thomas Ward: Revert "Revert a lot of things") (running on teward/Overlord)

Restart: API quota is 19948.

Thomas Ward

restarting it because I blew it up

oops

SmokeDetector

4:51 AM

[ SmokeDetector ] SmokeDetector started at rev bad35f0 (Thomas Ward: Revert "Revert a lot of things") (running on teward/Overlord)

Restart: API quota is 19946.

Thomas Ward

@Undo yeah i think i fixed it

everything looks to be behaving. I'm going to push this code up and then add the "Kill Pickle" in ws.py, and pull on every instance sequentially

SmokeDetector

teward/Aurora received failover signal.

CI on 10425ad succeeded.

Restart: API quota is 19963.

Thomas Ward

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 10425ad (Thomas Ward: TEMPORARY: Add "Destroy Queue Pickles" to ws.py init.) (running on teward/Aurora)

Thomas Ward

!!/standby teward/Aurora

SmokeDetector

4:57 AM

teward/Aurora is switching to standby

[ SmokeDetector ] SmokeDetector started in standby mode at rev 10425ad (Thomas Ward: TEMPORARY: Add "Destroy Queue Pickles" to ws.py init.) (running on teward/Aurora)

ArtOfCode/EC2 received failover signal.

Restart: API quota is 18145.

Thomas Ward

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 10425ad (Thomas Ward: TEMPORARY: Add "Destroy Queue Pickles" to ws.py init.) (running on ArtOfCode/EC2)

Thomas Ward

!!/standby ArtOfCode

SmokeDetector

ArtOfCode/EC2 is switching to standby

[ SmokeDetector ] SmokeDetector started in standby mode at rev 10425ad (Thomas Ward: TEMPORARY: Add "Destroy Queue Pickles" to ws.py init.) (running on ArtOfCode/EC2)

teward/ESXi received failover signal.

Thomas Ward

waits

!!/pull

SmokeDetector

4:59 AM

[ SmokeDetector ] SmokeDetector started at rev 10425ad (Thomas Ward: TEMPORARY: Add "Destroy Queue Pickles" to ws.py init.) (running on teward/ESXi)

Transcript for