« first day (1842 days earlier)      last day (1981 days later) » 

12:03 AM
[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Giulietta High LTFT & Low manifold vacuum by james walker on mechanics.SE
 
@thesecretmaster How's this step work?
 
yesterday, by thesecretmaster
I think we could really speed up MS by splitting off some of its parts into smaller services. Like autoflagging. If we could move that fully over to the lambda, that'd be great.
@thesecretmaster did you have what i had in mind or did you just mean sending the request
i think i see why Art thought i was referring to time performance now
because you were
oops
 
12:27 AM
@Undo API Gateway key
Then add it to MS config
 
yep, figured it out
I think it's almost working, just doing some DNS magic so it can use MS' API keys
 
redirect_uri has to be under erwaysoftware.com, not some Cloudfront URL
 
Oh
Why?
It wasn't before
 
Mostly a style choice; I'm opting to use metasmoke's API key and want to maintain an easy path to switching back if we need to
Trivial, just need ACM to get itself figured out
 
12:35 AM
Ok
 
After that, how does migration work?
Planning to migrate over the opt-in people first, then the others en masse.
 
Restart: API quota is 14619.
 
@Undo Flip all the token_migrated flags to false
Then you can bulk migrate and flip em back
The opt in people would have to opt in again if you want to do it that way
 
angussidney/droplet received failover signal.
 
uh
MS restart?
 
12:42 AM
!!/location
 
@thesecretmaster teward/Solar Flare
@thesecretmaster angussidney/droplet
Restart: API quota is 19998.
 
!!/standby angus
!!/location
Whoops?
!!/alive
Ehm
Halp?
 
The status userscript reports all but Henders/EC2 as unresponsive
that seems to be the only instance which is still pinging back
 
@Undo Did you do something?
 
seems to be MS again
 
12:45 AM
Henders/EC2 received failover signal.
 
!!/location
 
@thesecretmaster Henders/EC2
 
Restart: API quota is 19991.
 
Wasn't me
 
12:46 AM
Warning: 11 latest connections to metasmoke have failed. Disabling metasmoke
 
Looks up to me
 
that's at least better than crashing.... if it doesn't
 
looks down to me
"DNS_PROBE_FINISHED_NXDOMAIN"
 
Did I screw that up?
dig metasmoke.erwaysoftware.com?
 
12:49 AM
Nah, I'm getting it
Although maybe I'm cached
 
Warning: 11 latest connections to metasmoke have failed. Disabling metasmoke
 
CNAME overriding A somehow?
 
2 mins ago, by Undo
dig metasmoke.erwaysoftware.com?
 
ya screwed up, @Undo
 
12:51 AM
Not seeing it here
 
no answer to a dig a
 
I'm getting no response to my dig
 
... wut
 
owen@JESS:~$ dig a metasmoke.erwaysoftware.com

; <<>> DiG 9.9.5-3ubuntu0.18-Ubuntu <<>> a metasmoke.erwaysoftware.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 22310
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;metasmoke.erwaysoftware.com.   IN      A

;; AUTHORITY SECTION:
erwaysoftware.com.      168     IN      SOA     ns-cloud-e1.googledomains.com. dns-admin.google.com. 115 21600 3600 1209600 300
 
Although I still have cache which is nice :)
 
12:51 AM
> ANSWER: 0
 
I see about the same thing as Art, without the a
 
@Mithrandir a is default, I just made it explicit
 
noted
 
also no answer to a CNAME
...or NS
 
Warning: 11 latest connections to metasmoke have failed. Disabling metasmoke
 
12:53 AM
No idea what I'm doing, but let's try this
 
...or even TXT
you literally have no DNS
 
@ArtOfCode dig blaze.erwaysoftware.com?
 
no answer
oh wait
 
Heh, we can figure out how many layers of dns cacheing each instance is behind
 
;; ANSWER SECTION:
blaze.erwaysoftware.com. 3599   IN      CNAME   s3.amazonaws.com.
s3.amazonaws.com.       2774    IN      CNAME   s3-1.amazonaws.com.
s3-1.amazonaws.com.     4       IN      A       52.216.98.61
 
12:54 AM
there we go
dig foobarbaz.erwaysoftware.com
should get an A record
 
Got an answer for ms
 
;; ANSWER SECTION:
foobarbaz.erwaysoftware.com. 1799 IN    A       54.236.78.156
 
yeah
 
MS seems to be back up now?
 
What'd you do?
 
12:55 AM
;; ANSWER SECTION:
metasmoke.erwaysoftware.com. 3599 IN    A       54.236.78.156
 
DNS has always been running with an @ A 54.(...) record
Which apparently works fine, until you CNAME a (...).metasmoke subdomain, then something breaks
So I made a metasmoke A 54.(...) record, which I guess fixed it
 
Restart: API quota is 19996.
 
Never even saw the issue here
 
!!/location
 
@Mithrandir angussidney/droplet
@Mithrandir Henders/EC2
 
12:56 AM
!!/standby angus
 
!!/rev
 
@thesecretmaster Where's the bulk migrate code MS-side?
 
!!/pull
 
@Undo Are all the flags flipped to false?
 
Restart: API quota is 19929.
 
No, just want to know where the code is
 
well socket science works
in Socket Science, 1 min ago, by SmokeDetector
.
2986gqhsb2NhdGlvbrR0ZXdhcmQvTHVuYXIgRWNsaXBzZal0aW1lc3RhbXDLQdcBH+SymM0=
 
ah, u.migrate_token
 
12:59 AM
@Undo It isn't, you'd do it with httparty in the rails console
 
k
 
I can walk you through
Before you change MS's AppConfig, you need to flip the flags to false
 
Restart: API quota is 14596.
 
So here's what I'd like to do - leave the flags on, and just copy the existing MS token from MS to Lambda
Which I could do manually through the console, but it'd be easier if I knew the bulk import syntax
 
I'm not sure I follow what you want to do
 
1:02 AM
have a sample request for /load_tokens? That's all I want
 
My plan was 1) go back to traditional autoflagging 2) copy tokens to ddb 3) flip autoflagging over to lambda 4) clean up MS code
@Undo Oh, that's what you want. 1 sec
Pretty sure it's {tokens: [{account_id:,access_token:}]}
 
k
 
I'm on mobile, or I'd check
 
ah, okay. I'll get it
 
[ SmokeDetector | MS ] Bad keyword with email in answer, email in answer, link at end of answer (221): Australian PR/Indonesian Passport - Do I need a visa to go to Korea? by Ahmed khan on travel.SE
[ SmokeDetector | MS ] Bad keyword with email in answer, email in answer, link at end of answer (221): Australian PR/Indonesian Passport - Do I need a visa to go to Korea? by Ahmed khan on travel.SE (@JoErNanO)
naa- feedback received
naa- feedback received
fp feedback on autoflagged post: Australian PR/Indonesian Passport - Do I need a visa to go to Korea? [MS] (@quartata)
Autoflagged FP: flagged by @SmokeDetector, @DJMcMayhem, @quartata, @FrenzyLi
Autoflagged FP: flagged by @SmokeDetector, @DJMcMayhem, @quartata, @FrenzyLi (@quartata)
 
1:07 AM
!!/location
 
@thesecretmaster teward/Solar Flare
@thesecretmaster Henders/EC2
 
!!/standby henders
 
Henders/EC2 is switching to standby
 
Thanks buddy
!!/location
 
@thesecretmaster teward/Solar Flare
fp feedback received on Bootstrap side by side grid view [MS]
[ SmokeDetector | MS ] Toxic answer detected (82): How do you cook pork chops without drying them out? by Chrisdy on cooking.SE
 
1:20 AM
@ArtOfCode Want to volunteer for a flag test?
@thesecretmaster I think it's all set up, but getting this:
2.5.0 :049 > ap User.find(1).spam_flag(Post.last)
  User Load (0.2ms)  SELECT  `users`.* FROM `users` WHERE `users`.`id` = 1 LIMIT 1
  Post Load (0.3ms)  SELECT  `posts`.* FROM `posts` ORDER BY `posts`.`id` DESC LIMIT 1
   (0.1ms)  SELECT `moderator_sites`.`site_id` FROM `moderator_sites` WHERE `moderator_sites`.`user_id` = 1
  Site Load (0.1ms)  SELECT  `sites`.* FROM `sites` WHERE `sites`.`id` = 11 LIMIT 1
[
    [0] false,
    [1] "[beta] /autoflag/options 403\n{\"content-type\"=>[\"application/json\"], \"content-length\"=>[\"23\"], \"connection\"=>[\"close\"], \"date\"=>[\"Mon, 03 Dec 20
403 on /autoflag/options
Should be a valid route
AppConfig['token_store'] has all the right stuff in it
 
1:35 AM
@Undo can't do anything right now, but if you just need to use my account go ahead
 
k
 
[ SmokeDetector | MS ] Few unique characters in answer, offensive answer detected (165): Javascript Security: is storing sensitive data in a self invoking function more secure than cookies? by user10736752 on stackoverflow.com
[ SmokeDetector | MS ] Potentially bad keyword in body (1): CSV File and TypeError: a bytes-like object by DeltaTroz on stackoverflow.com
fp- feedback received
 
2:01 AM
@Undo Huh
Weeeird
 
Trying to convince CloudWatch to show me something useful; no luck
 
Check API Gateway logs?
There's a switch in API gateway to turn them on
 
I thought I found it, but it's not doing anything
 
Screencap?
 
2:03 AM
Oh, you sure that's not showing up in CW?
 
It's like it's writing hundreds of empty logs
 
Weeeird
 
CloudWatch and me have never gotten along well
 
afk for 5 mins
Actually wan't 5 mins, I'm back
If you click, still empty?
 
very empty
 
2:07 AM
If you hit any other routes, does it log?
 
no
 
Also, are you sure you don't have cloudfront mixed in anywhere?
 
freaking CloudWatch. This shouldn't be hard
 
Because that error looks like it's coming from cloudfront
 
@thesecretmaster Custom domain name
 
2:09 AM
I have a feeling that error lives in cloudfront
 
well, let's try it the other way
 
Because it's not hitting AG
 
Nope, CloudFront is just relaying it
[0] false,
[1] "[beta] /autoflag/options 403\n{\"date\"=>[\"Mon, 03 Dec 2018 02:11:43 GMT\"], \"content-type\"=>[\"application/json\"], \"content-length\"=>[\"23\"], \"connection\"=>[\"close\"], \"x-amzn-requestid\"=>[\"c71f082b-f6a0-11e8-816f-df9b58dd1fa2\"], \"x-amzn-errortype\"=>[\"ForbiddenException\"], \"x-amz-apigw-id\"=>[\"RTuS2EHQoAMFUPA=\"]}\n{\"message\":\"Forbidden\"}"
 
Here's the thing: If it's hitting AG at all, it would log something. At least it'd open an empty stream
 
aha
CloudWatch finally decided to stop lying
(c71f082b-f6a0-11e8-816f-df9b58dd1fa2) API Key **********************************hMiENF not authorized because method 'GET /autoflag/options' requires API Key and API Key is not associated with a Usage Plan for API Stage 7kjkmf3x9d/Production: No Usage Plan found for key and API Stage
 
2:13 AM
Oh, you didn't add a usage plan to the key?
Yeah, that's horrible design on Amazon's side
 
Nope, but why the heck did it work for importing?
 
But you need a usage plan for a key for it to work
Did it?
Did you require a key for importing?
 
oh, wasn't required
there
 
Should be :)
Everything should be keyed except the /auth routes
 
 
2:14 AM
Actually /auth/confirm should be keyed also
@Undo Yeh, there are grumbles on AWS fora
Oh, and you need to change the oauth uri on stackapps
 
nope, MS subdomain
 
Oh, nice
 
There we go
Feels like it's working
 
[ SmokeDetector | MS ] Manually reported answer (94): rails --api doesn't work by Undo on stackoverflow.com
 
redirect_uri is not under the registered domain for this application
I can't sign up
 
Metasmoke post 146617 (rails --api doesn't work) destroyed by Undo
Metasmoke post 146618 (Given a Model.title, how to best query for all records with the first letter = X) destroyed by Undo
 
Restart: API quota is 14000.
 
@thesecretmaster fallout from flailing while trying to fix quota plan issue
Fixed the config, looks like it works now
 
\o/
Looks good to me too
 
2:21 AM
I'd like to run it like this overnight, then do full migration
 
@Undo hey you're here :) Do you think it'd be useful to allow Smokey to pull from force-pushes?
 
ehhhhh
useful, maybe, but also a bit scary
 
@Undo Wanna try flagging something?
 
Implementation: git reset --hard origin/deploy, instead of regular git pull
 
@thesecretmaster Already abused Art's account on SO, but can do it again if you've got a target in mind
 
2:22 AM
Nah, if it works, it works.
Have you loaded in all the tokenz?
 
Assuming all edited files are in .gitignore, which has been true for years.
 
Just the 15 or so that signed up for the experiment, want to load the rest tomorrow
 
@Undo You can load all of em, and MS still won't use em.
 
oh, point
 
We'll also need to do some MS-side code changes... I'd freeze autoflagging registration until one of us gets around to fixing the flow around that
 
2:26 AM
Can it be part of the migration? Let people sign up in the current form, then when we remove the old code hit /load_tokens one last time?
 
Sure, no reason why not
 
Super excited about finally not having those tokens out there
 
It'd be annoying to test the migration, but we could try it.
I'd prefer to just swap it now... can I just make + push those code changes?
 
Sure, why not
I can always pull a backup if things go south fast
 
Where does autoflagging regis live?
 
2:29 AM
Looks for OCS
one click setup or something like that
 
Kinda terrified because I don't know this code, but I'll try
You sure it's not flag_settings controller/view?
 
link?
 
app/controllers/flag_settings_controller.rb or app/view/flag_settings/*
I know regis is at /flagging/preferences
 
2:34 AM
I don't think so, but don't have time to dig right now
 
k
Interesting, those pages use and instead of &&
 
[ SmokeDetector | MS ] Repeating characters in answer (82): What language is this? by sean on chinese.SE (@TetsuyaYamamoto)
 
@SmokeDetector has
 
naa- feedback received
[ SmokeDetector | MS ] Bad keyword with email in answer (86): Play FLV from ByteArray in Flash Player ✏️ by upalse on stackoverflow.com
[ SmokeDetector | MS ] Bad keyword in answer, blacklisted username, potentially bad keyword in username (179): Play FLV from ByteArray in Flash Player by daveclark966 on stackoverflow.com
 
2:49 AM
@SmokeDetector K
@SmokeDetector n
 
[ SmokeDetector | MS ] Manually reported question (93): What language is this? by Steve Baughman on chinese.SE (@TetsuyaYamamoto)
tp- feedback received
 
0
Q: Moderator flag on non-spam declined?

iBugThis answer was deleted a long time ago and it had spam (R/A) flags on it. The content is: test if this website has nofollow for external links http://www.baidu.com Seeing it is considered spam or R/A, I raised a mod flag with this text: This is not spam but just an innocent test. baid...

 
@Undo Changes pushed
 
Will require further cleanup later, but it's there
And should be deployable, but I'd like eyes first
 
3:02 AM
[ SmokeDetector | MS ] Few unique characters in answer (86): Markov chain inference confusion by user158565 on stats.SE
[ metasmoke ] ci/circleci: rails-tests failure on a8c16ce: Your tests failed on CircleCI
 
Oh wait, I screwed up
 
@SmokeDetector F
 
[ metasmoke ] ci/circleci success on 952fd76: Your tests passed on CircleCI!
[ SmokeDetector | MS ] Potentially problematic ns configuration in answer (1): Why are all of the values in my array null? by Chienih Louise Bunao on stackoverflow.com
[ charcoal-se.github.io ] continuous-integration/travis-ci/push success on 5a25b5c: The Travis CI build passed
 
!!/amiprivileged
 
@thesecretmaster ✓ You are a privileged user.
 
3:12 AM
Are my privs supposed to go with me from here to the tavern, or is this status-bydesign?
in Tavern on the Meta on Meta Stack Exchange Chat, 54 secs ago, by SmokeDetector
@thesecretmaster ╳ You are not a privileged user. Please see the privileges wiki page for information on what privileges are and what is expected of privileged users.
 
@thesecretmaster there's a pinned message by Art
 
[ SmokeDetector | MS ] Potentially problematic ns configuration in answer (1): A method that reads the data from the file into the HashMap by Chienih Louise Bunao on stackoverflow.com
 
in Tavern on the Meta on Meta Stack Exchange Chat, 2 days ago, by ArtOfCode
If you used to have Smokey privileges here but have lost them, ping me. I've just made some major changes to how they're handled, and some accounts may have got lost in the mayhem.
 
!!/watch-force- mymoneyhour\.com
 
@Makyen That pattern looks like it's already caught by Potentially problematic ns configuration in body and Potentially problematic ns configuration in answer; append -force if you really want to do that.
tpu- feedback received
 
3:18 AM
@iBug I thought the changes was that tavern inherited privs from here?
 
@thesecretmaster not sure, haven't checked what Art did.
 
Rob
Reading the Wiki page you linked it says: "Diamond privileges do not inherit like normal privileges do - if your privileges in CHQ are based on your diamond, they will not inherit to Tavern on the Meta.".
 
Ah, I bet mine are diamond based
Perchance I should get normal privs, but eh.
 
[ SmokeDetector | MS ] Potentially bad keyword in answer, username similar to website in answer (52): How to read these verses in the Quran? ✏️ by quranteaching on islam.SE
 
Rob
3:24 AM
There was a reply, to SmokeDetector, from Quilll whom typed the same command to which it replied that he is privileged. --- Perhaps he could bump your status. (Was offering to?).
 
Eh, I can just make a PR. I don't care much, was just curious since I was stopping by the tavern anyways.
 
@Rob mamaging privs for non-diamonds require push access, or you can start a PR
 
fp- feedback received
 
@SmokeDetector As of a few days after this was first reported (in May 2018), disclosure of affiliation had been edited into the post. This has previously been flagged and reviewed by site moderators.
 
3:43 AM
CI on bfbbdb5 succeeded.
 
!!/rev
 
Rob
Lurk and learn ;)
 
!!/pull
Maybe I'm not allowed to do that
!!/rev
Uh oh. Maybe I broke something important.
 
Rob
You lost your wallet in The Tavern 😀
 
3:49 AM
!!/alive
 
Henders/EC2 received failover signal.
 
!!/alive
 
@thesecretmaster ¯\_(ツ)_/¯
Restart: API quota is 19822.
 
@ThomasWard I appear to have broken solar by pulling, not quite sure what I did though. It's stopped pinging MS.
!!/rev
 
3:51 AM
@ThomasWard FYI: Both of your SD instances appear to be non-responsive. I'm not sure why. @iBug @NobodyNada Your instances also appear to be not pinging MS.
 
@Makyen Since you're here, did I break protocol somehow by pulling? That's when solar failed.
 
@Makyen ran out of disk space; fixed
 
@thesecretmaster No, it's not a break in protocol. A pull should have been fine.
 
Restart: API quota is 13426.
[ SmokeDetector | MS ] Potentially problematic ns configuration in body (1): PHP mysql data not retrieving from last id by SA CH on stackoverflow.com
[ SmokeDetector ] SmokeDetector started in standby mode at rev 164e570 (𝐢𝐁𝐮𝐠 ♦: ! slightly adjust PR approval message) (running on NobodyNada/Raspberry Pi)
 
3:54 AM
!!/location
 
@thesecretmaster teward/Solar Flare
@thesecretmaster Henders/EC2
fp- feedback received
fp- feedback received
 
I think the problems with the other instances might have been associated with MS issues earlier.
 
!!/standby henders
 
!!/rev
 
That's really old. Should we pull again?
Or go to master, since it's in reverted mode?
 
It appears to have started in reverted mode. That implies there's a problem with the current version. Give me a sec to look through the transcript to see if I can determine where things went wrong.
 
@Makyen I PR'd to add myself to the privs list
 
Yeah, that shouldn't have caused an issue, although I didn't check the actual PR.
 
3:59 AM
@Makyen I'm going to step away, since this isn't really my department.
Could we go back to master then check the errorlogs?
 

« first day (1842 days earlier)      last day (1981 days later) »