Charcoal HQ

@Ferrybig My suggestion: Store the data in MS. When Smokey starts, it tries to pull the data from MS and if successful creates a local file cache of the config. If MS is down, it then uses that latest cache. Additionally, give smokey a new command (!!/refreshcache?) which does the pull of data from MS and creates the local file if successful.

This protects against MS down time by keeping a local copy of the cache

Quill

@Undo I'm planning to implement a model like system in a later PR to clean that up a bit... for now is mainly syntax changes, adherance to PEP8, etc

ArtOfCode

4:16 PM

@Andy and it means we get to push to GH

Undo

@Andy And commit said cache to GitHub for when both MS and my Smokey copy are down.

Andy

It also allows easier updates (ie. No git push/pull/CI/build/deploy/etc)

ArtOfCode

The refresh cache command is definitely something we should do

Undo

Getting the CI step out of blacklist changes would be awesome

ArtOfCode

otherwise we're looking at websockets or waiting for polling

Undo

4:17 PM

We could probably start with the big website blacklist, that's just a bag of strings.

And easy to throw in a Pickle file or something

ArtOfCode

And the patterns, for that matter

just strings that you have to parse into regexes

Andy

@Undo Pickle...bad.

Ferrybig

Those types are just regexes, we could write a test funtion on metasmoke to test the regexes before publishing

Undo

@Andy csv, whatever :p

ArtOfCode

Would require some creative thinking on the metasmoke end... either we make more people admins, or we need a new level of user

between reviewer and admin

Undo

4:19 PM

@ArtOfCode latter would work

It'd force @sandwich to make another metasmoke account, but that's okay

;)

ArtOfCode

@Undo Yeah, just yet another attribute on the user model :)

what is it now, like 25?

Undo

noooo

Andy

That push to Github step...I have a suggestion that makes the smokey project slightly more complicated, but just slightly

SmokeDetector

[ SmokeDetector ] Few unique characters in answer: How to properly style an infowindow? by Angelo on stackoverflow.com …

Ferrybig

We probably also need more account levels on smokedetector, 1 people saying "...stopit" or "...clearbl" and we have trouble

ArtOfCode

4:20 PM

sd k

@SmokeDetector k

@Andy Aye?

@SmokeDetector k

@Ferrybig Like level 2: wizard?

ArtOfCode

@Ferrybig Yeah, I've been wondering about the clearbl command... do we really need it?

PhMgBr

4:21 PM

Level 3: Grandmaster

Ferrybig

@ArtOfCode I have the same wonderings

Undo

@Ferrybig If someone can't keep themselves from messing with that stuff, they don't need access.

PhMgBr

Level 4: God AKA admin

ArtOfCode

level 4: Overlord

Undo

@ArtOfCode probably not

ArtOfCode

4:21 PM

level 5: Overflow

level 6: ~~Mod~~ God

PhMgBr

Level 5: Undo

ArtOfCode

level 7: Undo

Andy

In the refreshcache function (runs on start up and on command), if something is new, commit to a new repository. This repository is a subproject of smokey. Only Undo (or whoever hosts it now) can commit there and there is a policy to only let the bot do it. This keeps the project clean and you don't have to worry about pulling updates each time.

Ferrybig

level 8: redo

PhMgBr

@ArtOfCode Too many levels

@Ferrybig That would be Madara, right?

Andy

4:22 PM

So, the Smokey project has a subproject called config

ArtOfCode

level 9: overdo

Ashish Ahuja ツ

level 10: underdo

Andy

The bot itself pushes the cached config files there

PhMgBr

@ArtOfCode This Brad

Undo

@Andy Possible, yeah

ArtOfCode

4:22 PM

level 10: Shog

Undo

Not thrilled about giving it my GH credentials, but that can be worked around

Andy

Other users then just need to git update submodules (or whatever the real command is)

Noooooo

And...

Ashish Ahuja ツ

level 11: smokey

ArtOfCode

@Undo Smokey's got a GH account

Undo

yeah

Andy

4:23 PM

@Undo In environment variables, or a file that doesn't get committed, etc.

ArtOfCode

Env vars is more secure from accidental commits

Ferrybig

This will be another config file for the git credentials, and optionally environment variables

Undo

@Andy Smokey would still need to have access to it somehow, and Smokey is a giant remote-execution-vulnerability-as-a-feature

ArtOfCode

REVaaF

Ashish Ahuja ツ

I feel so dumb right now :P

ArtOfCode

4:24 PM

Almost sounds like I'm Dutch

Andy

@Undo If it's not your smokey, it doesn't need to push to github. The other ones are temporary.

Ferrybig

Why does the subproject even need to be code? Can't it be just plain text files?

ArtOfCode

It'll still be on MS for when yours comes back

@Ferrybig basically, yeah

Undo

@Andy What I'm getting at is this: GlobalVars.charcoal_hq.send_message(ENV['gh_password'])

Quill

I'm gonna head to bed... night everyone :)

Undo

4:25 PM

followed by !!/pull

Ashish Ahuja ツ

@Quill Night

Undo

Not saying that anyone here would do that, but it's still there.

ArtOfCode

@Undo I vote that the first person to do that gets the longest chat suspension we can possibly impose. Also, set up 2FA on the GH account.

Andy

@Undo Instead of doing it there, set the gh_password to a variable earlier in the code. If it's None, don't run that command.

Ferrybig

@ArtOfCode 2fa won't work as we need to let it login multiple times

Can't smokey push the settings to metasmoke and then metasmoke makes a git commit? We already have the metasmoke key inside smokey that can be exposed

ArtOfCode

4:26 PM

@Ferrybig I'm thinking more protection against humans logging into the account

Undo

@ArtOfCode Not necessarily traceable, though. git push --force is a thing, as is sending stuff to a server that isn't chat.

Andy

gh = ENV['gh_pass']
# STUFF HAPPENS!
if gh:
	send_message()

Undo

I have an audit trail that persists through force pushes, but still

Ferrybig

@Undo Github help center - Blocking force pushes to a repository

Ashish Ahuja ツ

Night

Undo

4:28 PM

@Ferrybig people can turn that off

ArtOfCode

@Ferrybig That's GH Enterprise

Undo

@ArtOfCode IIRC it's a thing in classic GitHub too, but yeah

Seth

git push --force https://t.co/YrHoeXlKlr

Tweeted by thatcoderiguana on March 21, 2016 at 3:27 PM

ArtOfCode

@Undo Done slightly differently, but yeah. Branch protection.

Seth

just seemed relevant.

Andy

4:29 PM

I'm stepping out for my lunch walk - YAY! SUNSHINE! - I'll be back in about 30 min

Ferrybig

@Undo Only if they have admin level access to the repo, right? And by default everyone in the organization is contributor?

@AshishAhujaツ Night!

Undo

although that could be fixed

SmokeDetector

[ SmokeDetector ] Few unique characters in answer, repeating characters in answer: Hyper-V, Is it possible to have a host in cluster whilst still have locally stored VMs by Jim Ohyea on stackoverflow.com …

Undo

At some level, we just have to trust eachother not to the evil things. But it'd be nice if there was some technical limitation there too.

ArtOfCode

Who actually has access?

7 accounts.

1 of those is a bot.

4 of the rest are network mods.

PhMgBr

4:31 PM

@Undo Don't ever make me an admin then

Undo

@ArtOfCode well, anyone on github.com/orgs/Charcoal-SE/people can theoretically push arbitrary Python to something on my local network

PhMgBr

The thought is tempting

Undo

Yeah, I trust everyone on there. Do I trust some guy at an airport who happens to pick up their lost laptop? Maybe not.

ArtOfCode

The other 2 have been around this project for ages, and I'd vouch for them.

Ferrybig

Can't you DMZ the PI to a separate network segment?

Undo

4:32 PM

could, but now it's harder for me to SSH to it

ArtOfCode

@Undo True. On the other hand, what are the chances of (a) losing a logged-in device AND (b) getting it nicked rather than handed in AND (c) getting it nicked by someone who's a decent Python programmer AND (d) one of us not picking up on the fact that some dangerous code went in AND (e) one of us pulling it?

PhMgBr

97.8953%

Undo

@ArtOfCode insanely low

Just trying to figure out a way to have the least risk on this thing.

PhMgBr

Also consider that the random decent python programmer might be a good and civil guy.

Undo

(and remember, if it's not possible for someone to do it it's not possible for someone to be accused of doing it, which is another factor here)

ArtOfCode

4:35 PM

I get that the arbitrary code execution vector is there and that it's a real security risk if it does happen, but that does have to be balanced against the changes of it actually happening to get a realistic perspective on security.

Ferrybig

People with push right can already get access to the smokedetector account at the moment, if they print the correct variables

Undo

@ArtOfCode Still doesn't hurt to minimize possible damages.

ArtOfCode

@Undo Aye, I just don't want to see a potentially really useful feature hampered by security concerns that are small. It needs to be sufficiently secure, and no more.

Undo

Also, this is one of the reasons I want tighter SE integration to be through metasmoke; I hold the sole deployment keys to that. Far less risk.

@ArtOfCode Oh, don't worry about that. Unless it's something totally insane, I'm not gonna block features over these crazy-minor security things.

ArtOfCode

sweet :)

Ferrybig

4:38 PM

Do you actually trust your PI? When you installed it, it came with some magic blob of binary code for the bootloader.

Undo

Yeah, I'm not too concerned about that :P

ArtOfCode

Immutable Law of Security #1: If a bad guy can persuade you to run his program on your computer, it's not your computer any more.

The thing about security is how to define bad guy, and how to work out when to trust people.

Because at some point in the chain, you have to trust someone.

So is this something we actually want, and I should start modifying metasmoke?

Ferrybig

I am neutral on the idea, so I skip this vote

Undo

@ArtOfCode Not entirely sure yet, would want to get @sandwich and @hichris123's opinions first.

ArtOfCode

aye

Wai Ha Lee

4:59 PM

sd f

PhMgBr

:29764727 Wrong tags. Should be tagged with [RegEx-request]

SmokeDetector

[ SmokeDetector ] Messaging number in body: need help in code js by H.S on stackoverflow.com …

ArtOfCode

Why is there not a "pile of steaming horse dung" close reason?

2

SmokeDetector

[ SmokeDetector ] All-caps answer, few unique characters in answer: java.lang.ClassCastException:fragment cannot be cast to android.support.v4.app.Fragment by user6357422 on stackoverflow.com …

PhMgBr

@ArtOfCode Because the question version of horse dungs don't steam.

SmokeDetector

5:23 PM

[ SmokeDetector ] Few unique characters in answer: Word for one who does not eat onions by bob on english.stackexchange.com …

Undo

sd k

sandwich

Sorry, lost track of conversation there and can't tell what the subject is. If it's about GH access, I'm very happy to have the lowest level that allows push.

ArtOfCode

@sandwich It was about moving the website blacklist and possibly patterns to metasmoke

sandwich

Some patterns are in methods like email with keyword.

A big list of "these are strings to match" makes sense as an imported resource.

Regexes and associated details (sites to exclude, other rule parameters) are a but more complex and belong naturally in the code.

Undo

I agree that the regexes can stay in the code, at least for now

They're really code

sandwich

5:30 PM

An alternative approach would be to move findspam to a submodule, with different set of users pushing there.

Undo

And probably should be run through CI

@sandwich This is more about eliminating the CI step when adding blacklisted websites, etc.

Because we really don't need it, it's mostly annoying

But yeah, I suppose a submodule would work there

sandwich

Oh. I thought it was about security.

Undo

Security was tangent, talking about how we could have Smokey push to GH without revealing the creds.

Seth

@ArtOfCode there is. It is called "downvote".

sandwich

Sadly, it does not enable deletion of answered questions.

wythagoras

5:45 PM

@Undo Can I have my stats, please?

Undo

@wythagoras metasmoke.erwaysoftware.com/admin/…

wythagoras

@Undo Ah, I had missed it was made public. Thank you!

Undo

:)

SmokeDetector

5:58 PM

[ SmokeDetector ] Shortened URL in body: Que tan personalizable y agil es tu CMS by Tufik Chediak on es.stackoverflow.com …

SmokeDetector

6:14 PM

[ SmokeDetector ] Offensive answer detected: "pros and cons" vs "advantages and disadvantages" by Nigerian on ell.stackexchange.com (@PhMgBr) …

PhMgBr

@SmokeDetector K

SmokeDetector

6:55 PM

[ SmokeDetector ] Phone number detected in title: 1-877-778-8969 Lenovo customer care by Jordan Mkt on askubuntu.com …

PhMgBr

SD K

sandwich

@SmokeDetector k

sandwich

7:32 PM

!!/test www.healthportalsite.com

SmokeDetector

> Pattern-matching website in body, pattern-matching website in title
----------
Title - Position 5-24: healthportalsite.co
Body - Position 5-24: healthportalsite.co

sandwich

Thought so. I now can't help noticing such patterns, even in a site that's used by my doctor...

Andy

@sandwich Your doctor is a spammer

PhMgBr

7:45 PM

What does he treat?

Spamophobia?

SmokeDetector

Backoff received of 10 seconds on request to

http://api.stackexchange.com/2.2/questions/32886;32869?site=puzzling.stackexcha‌nge.com&filter=!)E0g*ODaEZ(SgULQhYvCYbu09*ss(bKFdnTrGmGUxnqPptuHP&key=IAkbitmze4B‌8KpacUfLqkw((

Undo

newline in the site again

where the heck would that come from?

sandwich

Whitespace enforcing code in chat?

Ferrybig

@Undo And it also seems wrong site, the api expects puzzling instead of puzzling.stackexchange.com

Andy

@sandwich Its in the key too

PhMgBr

7:48 PM

@SmokeDetector This is undoubtedly the largest and most noticeable improvement to Smokey.

Undo

@Ferrybig no, you can pass the domain name

Ferrybig

@Undo Doesn't seem to work when you copy the url smokey posted

sandwich

because you copy it with zwnj I guess

Undo

what the heck

@sandwich no, this is weird. Copy

http://api.stackexchange.com/2.2/questions/32886;32869?site=puzzling.stackexcha‌nge.com&filter=!)E0g*ODaEZ(SgULQhYvCYbu09*ss(bKFdnTrGmGUxnqPptuHP&key=IAkbitmze4B‌%E2%80%8C%E2%80%8B8KpacUfLqkw((

into your browser

hichris123

I suspect that's just some quirk -- not an actual thing Smokey is experiencing.

SmokeDetector

7:49 PM

[ SmokeDetector ] Offensive answer detected: QtCreator hotkey to move selected block by aferriss on stackoverflow.com …

hichris123

Otherwise the call would fail and we should get an error message...

Andy

You can see the extra characters there @Undo. Copy it directly from here and you get it too:

3 mins ago, by SmokeDetector

Backoff received of 10 seconds on request to

http://api.stackexchange.com/2.2/questions/32886;32869?site=puzzling.stackexcha‌nge.com&filter=!)E0g*ODaEZ(SgULQhYvCYbu09*ss(bKFdnTrGmGUxnqPptuHP&key=IAkbitmze4B‌8KpacUfLqkw((

Undo

True

I have no clue where those are coming from

sandwich

Maybe send API URL as a formatted link?

Ferrybig

@Undo Ok, its weird, if you go to the history of the message smokey posted, and copy the url from there, it works?

Undo

7:51 PM

API URLs break link formatting

@Ferrybig huh, that's interesting. So I guess we broke the chat formatter

sandwich

Hm. But we don't need filter and apikey here every time.

Just cut off by the first &

Undo

Possible that the filter is invoking a bad query plan

sandwich

Yeah, kind of defeats the purpose. Then again, what was the purpose?

SmokeDetector

[ SmokeDetector ] Repeating words in body: Why the transision doesnt work? by Angelo Nuho on stackoverflow.com …

Ferrybig

@Undo Ok, it is even weirder, according to notepad, both urls LOOK the same, but 1 works, the other not

Undo

7:53 PM

Purpose with printing the URLs that backoff was "maybe we're doing something crazy", but I don't think we are

sandwich

Yes, and we also found that it's arbitrary sites that throw it, not just SO.

Ferrybig

I usually see Shog coming to the SOcvr when a member made a app that didn't respect the backoff, maybe he can take a look at smokey api calls to see if he knows why it backoffs so frequently

Undo

It's been passed on to devs already, haven't heard anything back yet

(read: don't go ping Shog about this, he's busy enough already)

sandwich

@ArtOfCode Looks good; I made a minor edit.

SmokeDetector

8:12 PM

Backoff received of 10 seconds on request to

http://api.stackexchange.com/2.2/questions/29846;29822?site=mechanics.stackexch‌ange.com&filter=!)E0g*ODaEZ(SgULQhYvCYbu09*ss(bKFdnTrGmGUxnqPptuHP&key=IAkbitmze4‌B8KpacUfLqkw((

SmokeDetector

8:39 PM

[ SmokeDetector ] Few unique characters in body: why is this function not running by user5631466 on stackoverflow.com …

[ SmokeDetector ] Few unique characters in body: Installing python using pip-3.2 by user5631466 on stackoverflow.com …

Ferrybig

sd 2tp-

SmokeDetector

[ SmokeDetector ] Few unique characters in body: Why is this not working in PHP? by user5631466 on stackoverflow.com …

Ferrybig

@SmokeDetector tp-

SmokeDetector

Restart: API quota is 2189.

SmokeDetector

9:34 PM

API quota rolled over with 1939 requests remaining. Current quota: 9999.

stackoverflow: 1563
askubuntu: 515
superuser: 404
drupal: 281
unix: 226
salesforce: 182
english: 157
math: 147
electronics: 140
mathoverflow.net: 133
wordpress: 128
sharepoint: 123
security: 117
dba: 112
gaming: 107
gis: 105
academia: 103
apple: 103
travel: 102
mathematica: 100
scifi: 93
magento: 93
ell: 90
programmers: 86
android: 80
physics: 77
worldbuilding: 76
meta: 74
puzzling: 72
rpg: 72
stats: 71
tex: 70
money: 65
ru.stackoverflow: 64
raspberrypi: 60
diy: 58
webapps: 54
pt.stackoverflow: 50

Yvette

@ArtOfCode the wiki looks good, there's just one grammar error > Additionally, the chatrooms are a professional environment - we're friendly and laughing and joking are fine, but keep it professional.

we're friendly, laughing and joking are fine, but keep it professional.?

SmokeDetector

[ SmokeDetector ] Link at end of answer: noIndex, noFollow and Canonical Tags on Facet Page SEO by Bagga Technologies on stackoverflow.com …

sandwich

@Yvette I think you should be able to edit it directly.

@SmokeDetector k

Yvette

@sandwich I'm not part of the repo team?

ArtOfCode

@Yvette fixed

@Yvette it's a wiki, anyone can edit

i think

Undo

9:46 PM

pretty sure you have to be affiliated with the project somehow

sandwich

It'd be nice to find out...

Yvette

ok I will try and edit it

ooh I can edit it.. That's not good, anyone can edit it

sandwich

GH wikis are weird. There are no pull requests. Either it's closed or open to everyone.

Yvette

ah

keep a copy of it somewhere

sandwich

There's a revision history, of course.

Smokey's wiki has been editable for over a year.

in Tavern on the Meta on Meta Stack Exchange Chat, Jun 28 '15 at 19:12, by hichris123

@1999 I may come to regret this, but I opened editing of the wiki up to all.

Okay, slightly less than a year.

Yvette

9:51 PM

Do you think 'Getting Privileges' should be 'Priviliges'?

SmokeDetector

Backoff received of 10 seconds on request to

http://api.stackexchange.com/2.2/questions/29888;29900?site=fitness.stackexchan‌ge.com&filter=!)E0g*ODaEZ(SgULQhYvCYbu09*ss(bKFdnTrGmGUxnqPptuHP&key=IAkbitmze4B8‌KpacUfLqkw((

Yvette

without the typo

SmokeDetector