that's neither plastic nor a knife

Y'all know better then I do, but it seems like this opens up quite a can of worms.

Aaaand big sparkes. That's my queue to get back to work.

@thesecretmaster Oh yes it does. Scary as heck.

@thesecretmaster yah, but I think that is also why it was brought up for debate rather than Andy just doing it

To an extent it does

To an extent it doesn't

If we went with the model Undo's talking about, the beauty of it is that it's actually always been possible.

(thing is, anyone could do this now - nothing in licensing that keeps me from starting a business)

... that. Darnit.

ninja'd

When it was GPL, that woulda been a pain, but we got that figured out.

I wonder how SE would react w/ 2x SDs running around

flag wars between bots

might get ugly

Only one would have the doubled API quota

@Undo ugh, we were under GPL?

yah but say 4x people fork it ... then the non official would have 4x API quota

@NobodyNada way back, aye

Been dual MIT/Apache for a good year or three, though

Yah, but even if it was GPLed ... pretty sure what Andy is talking about is a massive rewrite using the orig as Context

On that note... I'm kinda coming to the conclusion that a split like this does need to be a total rewrite.

Take the blacklists and checks with us, obviously, but Smokey as it is now is just... not what we'd need

You could reuse much of what we have, at least the principles

Oooh. There's a question. How do you tackle control?

Right now we have chat commands. If the SE adapter is just going to be a thing that posts to chat rather than a part of the detection engine... what do we do?

Some commands become inapplicable

Some we could keep

Or we could invent some sort of integrated control system

[ SmokeDetector | MS ] Bad keyword in answer, blacklisted username, offensive answer detected, potentially bad keyword in answer, toxic answer detected: Cooking for the next day on yom tov by Adolf Hitler on judaism.SE

tpu- by ArtOfCode

@ArtOfCode Seems like a business problem to solve, then maybe open source if one wanted to

How so?

As I see it, we can kinda split commands into two groups: chat helpers, and engine control

Mobile, few minutes

Chat helpers are things like !!/test and !!/alive. Engine control is !!/watch or !!/reboot or !!/standby.

Chat helpers could be a part of the SE adapter, just interfacing with the engine as they do now.

Engine control... could, but then we have to build deeper access control into the engine and decide whether or not it can/should be part of every adapter

Which brings up the question of how do we control SmokeDetector on other sites?

Seems like it'd be easier in some ways to do a rewrite, converting everything to a database, and at the same time write a control webapp that writes to that database.

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, phone number detected in body, phone number detected in title: This Is Binance NumBer [1.888.883.0233] Binance Phone NUmber + Binance Number + Binance Support Number by Peter Land on askubuntu.com

@SmokeDetector k

tpu- by John Dvorak

I've got a one on one in an hour-ish, depending on how on time I plan on being. After that I'm going to work on some of these more detailed proposals. Right now I have something for each of the three sections in my image (Input, Analysis and Output), one for Infrastructure and one for Organization. Thoughts on what you'd like to see first? (Not infrastructure). If I do Organization, Can I get a quick summary of the options we've talked about? There have been a few and I want to include those

Hey what did I miss?

everything

:P

All right give me a few minutes

I finally got adequate sleep so maybe I can answer questions

@Andy I think Input and Output are the most interesting technically to implement; maybe one of them? We all know how Smokey analysis works, but polymorphing IO to suit multiple platforms... less so.

OK I knew it was only a matter of time effort someone wanted to make money off of it

@ArtOfCode I already did Input for Inferno

Output is easier

@quartata that was just for SE wasn't it

It could do other sites. You specified the route

Although I did append api.stackexchange.com to it

Yeah... probably gonna need to be more complex than that.

Not really. Most APIs still fall into the two categories, polling and enqueueing

Two subclasses of Bodyfetcher

In the configuration you can specify things like your max quota, the parameters for the key, anything to write to the WS on startup, etc

It should cover almost everything, and if it doesn't then you make a more specific subclass

Output needs only a few methods implemented really - tell_rooms, is_privileged and get_report_data

[ SmokeDetector | MS ] Pattern-matching website in body: Can anyone tell me the important of h1 tag? by Ghost stories on stackoverflow.com

SE falls in both classes, polling for SO, enqueent over the websocket for the other sites

Indeed.

@quartata You're still thinking too narrow here. Input might have more data than we do now. Input might have more methods than we do now. Output might not tell any rooms. Output might do other things. Output might not have privileged people. Etc, etc.

Money wise, I think what people need to realize is what we bring to the table is not code but the support

@quartata and existing experience

I really don't care at all what you do with that though

@ArtOfCode tell_rooms is misleading. Try broadcast on for size

@ArtOfCode what if we just made an e.g. Slack backend for chat?

You specify what your message is, it's the Output's responsibility to put it somewhere

And yes there will absolutely be people with privileges

@NobodyNada Or IRC. Or email. We can do them all, if we make it extensible

This was my plan from the beginning

Now I have not read Andy's proposal yet so hold up

TL;DR: Words. Lots of them. And more on the way.

OK @Andy I think you get the general breakdown of the systems

I like

But I think there's one thing we need to get out of the way

One SmokeDetector instance monitors all the sites it has configured. This is what you're saying right

I don't have any implementation details in there right now.

At least I don't think I do.

I did mention it'd need to be able to handle input from multiple sources and output to multiple sources at the same time. Those sources could be SE, Medium, Bob'sDiscourse, Betty'sWordpress and Linus'sGitHub though. We don't want a deluge of posts on one to slow to processing of the others.

I intend to make Smokey take advantage of multiple cores

Two of those 5 instances are system wide things (SE/Medium) and will have a decent volume of traffic. The other three are small to medium things and will have a handful of posts per day.

Additionally, the first two are probably custom integrations that we run. The other three are plugins or bots (Github? not sure how that works right off hand) where it would just get the output of our middle step and then it implements the logic they want.

So, the work load for a post from entry to exist is going to very greatly.

More so when we add in logic that says "Bob'sDiscourse allows Pornhub links but Linus'sGithub does not"

So I've discussed this a bit but wanted to illustrate it better

Making each site as configurable as possible

For what you're describing, this is what I was envisioning

tp by WELZ on Why is the Tetragrammaton unutterable [MS]

There might be a directory checks/ and each file in it is a .yml

It looks something like this

....crap I can't put in newlines on my phone

@quartata copy+paste from e.g. notes

Don't have time I guess I'll talk about it later

That's cool. No rush.

I seem to have assigned myself lots of homework anyway

[ SmokeDetector | MS ] Mostly non-Latin answer: Download dSYMs from Xcode Organizer i get this error "Missing App Record" by user8669097 on stackoverflow.com

naa by Zoe

[ SmokeDetector | MS ] Link at end of answer: How to start java programming from a to z by AlgirdasM on stackoverflow.com

fp- by doppelgreener

@SmokeDetector k

[ SmokeDetector | MS ] Misleading link: Find current state of all properties in java.security.Security by jr10 on stackoverflow.com

fp- by NobodyNada

fp by Zoe

[ SmokeDetector | MS ] Link at end of answer: Why can't I read data from file where i wrote text. C by mario on stackoverflow.com

naa- by Stephen Kennedy

what happened to the metasmoke review queues?

[ SmokeDetector | MS ] Bad keyword in body: Boost C++ asio deadline_timer и tcp::socket отослать асинхоронно по tcp по таймеру by Mihail3200 on ru.stackoverflow.com

tp by Vogel612 on What is the best Android Rooting Tool? [MS]

@Vogel612 We now have an Untagged Domains review queue, so that we can tag domains with no tags

(assuming you're asking why there's 1800 reviews)

that explains. I was kinda worried stuff blew up...

@NobodyNada yes, that's exactly what I was asking about :)

CI on e07c131 succeeded.

!!/pull

[ SmokeDetector ] SmokeDetector started at rev e07c131 (NobodyNada: Merge pull request #2178 from NobodyNada/master) (running on teward/Solar Flare)

Restart: API quota is 11651.

fp- by rene

tp by Vogel612 on Can anyone tell me the important of h1 tag? [MS]

tpu- by Vogel612 on Can anyone tell me the important of h1 tag? [MS]

grumble grumble

Can we get SD to flag in my name if I tp from MS review?

^

second report is from FDSC

If you review from FIRE as tpu it will flag

^

Alternatively, visit the post on MS and there's a spam flag link

[ SmokeDetector | MS ] Potentially bad keyword in answer: Create sitecore template field with specific condition by s_magon on sitecore.SE

naa- by Zoe

[ SmokeDetector | MS ] Bad keyword in body: How can I find the best SEO service company in Cheltenham? by Fanny Warrick on superuser.com

!!/unwatch playonlinux\.com

@quartata Removed playonlinux\.com from watchlist

@gparyani @tripleee well known wrapper for Wine

[ SmokeDetector ] SmokeDetector started at rev dc8b818 (SmokeDetector: Auto unwatch of playonlinux\.com by quartata --autopull) (running on teward/Solar Flare)

Restart: API quota is 11279.

What should we do about companies that produce good software, but use a very wrong way to promote their software (e.g. Paragon, etc.)

tp by Zoe

@SmokeDetector likely spam seed

^

@gparyani this is open source tho

Other such software?

Eh

I would generally assume it's not intended to be spam

You have no particular reason to assume affiliation

Unless it's disclosed in which case it's probably OK

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, phone number detected in body, phone number detected in title, blacklisted user: Hello Freind 1.888.883.0233 Binance NumBer ♂ Binance PHone NUmber ! BInance Support NUmber by Peter Land on askubuntu.com

tpu- by Ferrybig

sd - k

@Art I've made a Metasmoke PR with a review history page

[ metasmoke ] ci/circleci failure on 9583ad6: Your tests failed on CircleCI

[ metasmoke ] ci/circleci failure on 9583ad6: Your tests failed on CircleCI

!!/location

@Zoe teward/Solar Flare

@SmokeDetector Oh good, I only violated one style check

oh yeah, that's the one that doesn't allow one-line ifs under any circumstances :/

prime directives: 1. ruin your code

@quartata ?

stand by, dealing with fires, will review in a while

@ArtOfCode No rush

@ProcessedMeat ...that's gotta be a dupe

[ metasmoke ] ci/circleci failure on 02c83eb: Your tests failed on CircleCI

[ metasmoke ] ci/circleci failure on 02c83eb: Your tests failed on CircleCI

@ProcessedMeat Found it

[ metasmoke ] ci/circleci failure on 4d0d3fe: Your tests failed on CircleCI

[ metasmoke ] ci/circleci failure on 4d0d3fe: Your tests failed on CircleCI

[ metasmoke ] ci/circleci success on 3f89870: Your tests passed on CircleCI!

[ metasmoke ] ci/circleci success on 3f89870: Your tests passed on CircleCI!

[ SmokeDetector | MS ] Bad keyword with email in answer: How does a hacker in a public WIFI actually view your computer? by L Mason Biddle on superuser.com

tpu- by NobodyNada

@SmokeDetector Wow, loan spam on a hacker question

I think you copied the wrong one, spammer

!!/watch hduncan\.al

@Makyen Added hduncan\.al to watchlist

[ SmokeDetector ] SmokeDetector started at rev 47c7699 (SmokeDetector: Auto watch of hduncan\.al by Makyen --autopull) (running on teward/Solar Flare)

Restart: API quota is 10911.

Possible business dynamic: Hiring folks to get issues fixed. Like "I really need this done, no time to do it myself, would pay a few bucks to get it done"

... mobile

@Undo Hurting? Like, we'll go out and knee cap spammers that hit your site? Or something else. Because, if it's the first one, I'm calling dibs on the title of Chief Kneecapper.

That is another option

is that a thug?

thugs, consultants, much of a difference?

@Undo who's hiring who to do what?

no

Unrelated: Has anyone done any analysis on word length of the Markov spam we get? I half suspect there'll be patterns there, the spammers might be pretty lazy on setting that up.

As in "always 100 words" or something

@ArtOfCode E.g. of

I need smaller thumbs

I like how Jeff does it with Discourse. They have a core team and then they have an official marketplace to solicit improvements/plugins.

That's not the way I want to go initially, but it's a thought. scribbles note to make a mention of it later

E.g. if I'm working with Medium and really need more concurrency or something, I might bribe quartata to work on that. Then it's a simple 1099 for me, I know I'm getting competent work, and quartata gets time compensated.

[ SmokeDetector | MS ] Few unique characters in body: Merge two dictionaries if the keys are equal by SleepinYeti on stackoverflow.com

i only take payment in dogecoin or zimbabwean dollars

No stickers?

Joke's on you, I actually have zimbabwean dollars

actually i probably would take stickers, it'd just be easier

@ArtOfCode wait really

yup

somewhere

nice

Like five hundred million of them?

@SmokeDetector v

@Undo probably, they're not worth much

A sheet of toilet paper

OK I figured out how to do newlines

sites/
    stackexchange/
    medium/
        fetcher.yml
        checks.yml
        sender.yml
checks/
    ...
    health.yml
    blogspam.yml
    offensive.yml
    trolls.yml

Now for the contents

checks/*.yml I envision as being fairly similar to the dictionaries we have now in findspam, just split up into categories

[ SmokeDetector | MS ] Bad keyword with email in answer, username similar to website in answer: infinite scroll on squarespace get category filter by Squarepaste on stackoverflow.com

Maybe looks a bit like this:

tpu- by Makyen

[ metasmoke ] ci/circleci success on 4012bca: Your tests passed on CircleCI!

[ metasmoke ] ci/circleci success on 4012bca: Your tests passed on CircleCI!

Does the AutoReviewComments userscript break with the non-inline editor? The option for auto-comments in editing don't appear in the non-inline editor

blogspam:
    - method: blogspam_checks.bad_keyword_with_email
       name: "bad keyword with email in {}"
       max_rep: 20
       ...
    - regex: "^asdf$"
       nwb: true
       ...
    - regex_list:
           - 1
           - 2
       nwb: false
       ...

Sorry for bad indentation

Then in sites/SITE/checks.yml

checks:
  - blogspam

We could do something like from blogspam import * vs from blogspam import specific_checks if we wanted more control, but in those situations we should probably just make a separate file

→ 1 message moved to Physics Meta

Internally I can do the work to make it so that regexes with the same names and same conditions e.g same max rep, score get grouped together into one for efficiency

If we take Andy's proposal, we're going to need to develop it layer-by-layer - i.e. we need to do Input first

[ metasmoke ] ci/circleci success on e19d4cf: Your tests passed on CircleCI!

[ metasmoke ] ci/circleci success on e19d4cf: Your tests passed on CircleCI!

[ metasmoke ] ci/circleci success on 295274f: Your tests passed on CircleCI!

@ArtOfCode I think a general pattern for configuring those things should be:

[ metasmoke-deploy ] deploy started by ArtOfCode

blah_handler:
    class: <classname>
    args:
        - ...

That is, if we need more flexibility we can inherit from whatever the generic class is

Or just change whatever configuration goes into that generic class

Now that being said I don't actually think findspam needs input to be made first -- they should not depend on each other

uh

they kinda have to

where else is it gonna get input from

Not what I meant sorry

I mean that findspam should be able to be called into outside of the bodyfetcher workflow

It should not care where the data came from, aside from whatever configuration it needs to use

Yeah, of course

Hey @Art why is this a TP? (if you somehow happen to remember from 2 years ago)

Hmm, I think stuff like max_rep should actually be specific to the category and not the checks

but to design the analysis phase we need to know what input it's going to get, and we don't know that without designing the input phase

A more generic idea:

@ArtOfCode I was getting to that

So for stuff like max_rep and stuff I think instead it ought to be specific for a category of checks and not an individual check, which is how it is anyways usually

That means it would go into site-specific configuration

fp by NobodyNada on What secret is this massive technical puzzle hiding? [riddles and data puzzles] [MS]

It could go like this perhaps:

@NobodyNada not a clue. I can't invalidate it for some reason, though.

There's 10 reports...should I fp them all?

checks:
    - category: blogspam
       import: "*"
       constraints:
           - user_reputation: "<20"
           - post_score: "<1"

Those keys are the actual properties you get in the response back from the API

@NobodyNada sure

fp by NobodyNada on What secret is this massive technical puzzle hiding? [riddles and data puzzles] [MS]

OK...sorry everyone

As for what should be scanned, it could be as simple as

fp by NobodyNada on What secret is this massive technical puzzle hiding? [riddles and data puzzles] [MS]

fp by NobodyNada on What secret is this massive technical puzzle be hiding? [riddles and data puzzles] [MS]

fp by NobodyNada on What could the hidden message be?[now involving 8 clues - riddles, and data puzzles] [MS]

fp by NobodyNada on What could the hidden message be?[now involving 8 clues - riddles, and data puzzles] [MS]

fp by NobodyNada on What could the hidden message be?[now involving 8 clues - riddles, and data puzzles] [MS]

fp by NobodyNada on What could the hidden message be?[now involving 8 clues - riddles, and data puzzles] [MS]

fp by NobodyNada on What could the hidden message be?[now involving 8 clues - riddles, and data puzzles] [MS]

fp by NobodyNada on What could the hidden message be?[now involving 8 clues - riddles, and data puzzles] [MS]

That should be it

heck

[ SmokeDetector | MS ] Potentially bad keyword in answer: Binance volume monitor by Burak Ereno on stackoverflow.com

[ metasmoke-deploy ] deploy started by ArtOfCode

fp- by J F

[ metasmoke ] ci/circleci success on c294e2d: Your tests passed on CircleCI!

That will not do what you want @Art :P

bah

I think

Maybe it's right

wait

why not

It was no method >=, not no method created_at... so f has a value, but created_at doesn't

Yeah, might be right. Cause it'll fall through to the invalidate thing

Deployed, anyway, so let's find out :)

yup, works

I though you had it backwards. Guess not.

I caught myself before getting it backwards :P

fp by ArtOfCode on How to: New order Binance API via RStudio [MS]

fp by ArtOfCode on Can anyone tell me the important of h1 tag? [MS]

fp by ArtOfCode on How to start java programming from a to z [MS]

tp by ArtOfCode on Merge two dictionaries if the keys are equal [MS]

@Art I've opened a PR to fix a bug -- I accidentally made an item unreviewable if it was completed but all the reviews were skips, e.g. if you add a domain tag on a review others have skipped

fp by ArtOfCode

[ metasmoke ] ci/circleci failure on 68f3469: Your tests failed on CircleCI

[ metasmoke ] ci/circleci failure on 68f3469: Your tests failed on CircleCI

crap...I accidentally removed a v from a div tag

I got it

Oops, sorry for the extra PR then