Charcoal HQ

2:01 PM

tpu- by tripleee on A bizarre slot machine [MS]

[ SmokeDetector | MS ] Manually reported answer: A bizarre slot machine by user33559 on puzzling.stackexchange.com

tpu- by Glorfindel

[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, link at end of body, repeated URL at end of long post: skin is due to immoderate of elastin and collagen by user71740 on drupal.stackexchange.com

tripleee

!!/blacklist-username Pro Max

SmokeDetector

tpu- by Floern

@tripleee Blacklisted Pro, Max

[ SmokeDetector | MS ] Blacklisted user: A bizarre slot machine by user33559 on puzzling.stackexchange.com

tpu- by Floern

tripleee

uh, not what I wanted

Magisch

@tripleee yah

Lemme fix that

SmokeDetector

CI on 360f55e succeeded. Message contains 'autopull', pulling...

tripleee

2:04 PM

@Magisch sure, thanks

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 360f55e (SmokeDetector: Auto blacklist of Pro, Max by tripleee --autopull) (running on Undo's EC2)

Restart: API quota is 5940.

[ SmokeDetector | MS ] Blacklisted user: A bizarre slot machine by user33559 on puzzling.stackexchange.com

tpu- by tripleee

tpu- by Magisch on A bizarre slot machine [MS]

CI on ffc4c9f succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev ffc4c9f (Thomas Ward: Merge pull request #458 from magisch/patch-6 --autopull) (running on Undo's EC2)

Restart: API quota is 5913.

tripleee

2:41 PM

!!/alive

SmokeDetector

@tripleee Of course

Undo

@MarkYisri We don't generally nuke accounts on request. Go through /contact for that

@Glorfindel Invite sent, confirm receipt.

SmokeDetector

[ SmokeDetector | MS ] Few unique characters in answer: How to make a Yes/No Message Box while clicking on button by Marcel Dierckx on stackoverflow.com (@dorukayhan)

Glorfindel

@Undo thanks, done.

SmokeDetector

[ SmokeDetector | MS ] Manually reported answer: Struts2 xml validation is not working by iutg on stackoverflow.com (@dorukayhan)

tpu- by Glorfindel

Undo

2:53 PM

also @angussidney, you should be able to manage the userscript team on github.com/orgs/Charcoal-SE/teams/userscripters

Magisch

The auto flagging used 9 flags from me today

Is it currently chosen at random between eligible flaggers as to who flags?

Undo

@Magisch Yep, looks like that didn't explode overnight.

Glorfindel

@Magisch looking at the flag logs, I think so.

Undo

@Magisch Yes

Glorfindel

I'm at 18 currently.

Undo

2:57 PM

Could probably switch that over to a more better system, but for now I'd say random works.

Magisch

Are only sites you have an account on in the site selection

Glorfindel

@Magisch ~~Yes~~. No, see below.

Undo

no, all the non-meta sites are.

Magisch

~~good~~ what happens if you've selected a site that you don't have an account on?

Undo

But if it tries to flag something on a site you don't have an account on, it should just error out and move on to the next eligible user.

Glorfindel

2:57 PM

But I assume you have an account on most sites, especially those which are spammed regularly.

Undo

@angussidney done, cc whoever

ArtOfCode

@Undo you had it right the first time :P

That's a pretty nice UI

Andy

@Magisch Yeah. I got 2 random flags :(

Random doesn't love me

ArtOfCode

I've had 17 today

Glorfindel

@Undo random will even out in the end.

Thomas Ward

3:11 PM

@Undo not sure if you saw my commits yesterday, but we now have error checking on the blacklist functions to prevent invalid patterns from being entered.

(confirmed working)

Undo

@ThomasWard Yep, saw it

Magisch

@Glorfindel Ideally we'd have it select so the "I went to flag but the system already did with my account" events are minimized

So essentially the users who are currently actively flagging in the room don't get autoflags so we can kill spam faster.

tripleee

I put in so strict requirements that it will not pick me any time soon

Magisch

@tripleee like 600 score?

I used 270

tripleee

@Magisch no, just a really constrained set of sites

Magisch

3:19 PM

The minimum is 99%, how much score would that be?

Glorfindel

200 IIRC

tripleee

@Magisch if you have enabled autoflagging, you can use the Preview button to play around with different thresholds without committing

and IIRC there was a sandbox where you could try this too, but I'm not sure where that was

Thomas Ward

@Undo cool, but at least we're now making sure it's valid regex, so we limit the number of broken patterns that could get added :P

saves regexparser CI errors, by doing that step earlier.

Glorfindel

@tripleee @Magisch metasmoke.erwaysoftware.com/flagging/conditions/sandbox

tripleee

@Glorfindel thanks, actually the preview button in the real thing doesn't seem to work very well (or maybe I was too impatient)

Mithrandir

3:26 PM

@Floern nice flag thing

SmokeDetector

[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, repeated URL at end of long post: cluesforhealth.com/garcinia-slim-fit-180/ by user84307 on graphicdesign.stackexchange.com

tpu- by Glorfindel

tripleee

the current 100% threshold is 341

Magisch

The current min accuracy we're going for is 99% right?

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, pattern-matching website in body, repeated URL at end of long post: How does vlamorous capacity? by user220401 on apple.stackexchange.com

tpu- by Glorfindel

ArtOfCode

@Magisch that's what the setting says

Glorfindel

3:27 PM

@SmokeDetector autoflagged

SmokeDetector

@Glorfindel That post was automatically flagged, using flags from: NobodyNada.

Glorfindel

Hmm ... when I opened the question, it was still on 0 score.

Magisch

@ArtOfCode That means I could set it to weight:140, max_rep:10, reasons:2

Thats 99,02%

ArtOfCode

I believe you can set it to 135/1/1 and get 99.02% as well

Magisch

what happens when a FP causes the score threshold to shift

will your preferences be adjusted automatically?

I think i'll set it at 190, 10 max rep, 2 reasons. That'll encompass all the spam with only 2 high accuracy reason triggers, and still be at 99,72% accuracy

Undo

3:33 PM

Minor todo: We should probably re-validate all the flag conditions when the min accuracy is changed

tripleee

I think (or at least certainly hope) that the expectation is for the system and community to work monotonically towards fewer false positives, as practices and processes improve

Undo

@tripleee At this point, we're probably going for more true positives. We already get very few false positives (as raw numbers) anyway

More true positives without more false positives.

Jan Dvorak

@Undo so that we hit all sorts of rate limits whenever it's decreased?

tripleee

@Undo but in order for this to be acceptable to the growing group of stakeholders, there is less margin for experiments which might produce false positives

which is one of the reasons it would be nice to have a staging copy where you could try out a rule for a while before you migrate it to the production version with alerts and flagging

Undo

@JanDvorak Not sure what you're saying there.

@tripleee Yeah, I've been thinking about that.

It'd be really nice if we could track those things inside Smokey itself, and not have to run two copies

tripleee

3:37 PM

@Undo yeah, true

Jan Dvorak

Unless I (probably) misunderstood you?

Undo

@JanDvorak Oh, context-sensitive. Sorry. Metasmoke does automatic validation of accuracy and post count on all FlagConditions before allowing them to be created/edited.

If we change the minimum accuracy, those validations should be re-run and conditions that don't match disabled.

Jan Dvorak

Oh. I thought you meant un/casting some spam flags retroactively

tripleee

six more autoflags to set another new record since last week's all-time high

Undo

Nope

Jan Dvorak

3:41 PM

how long until we switch Smokey into full-auto mode (six autoflags and don't even bother telling us)?

Undo

heh, that'll be staged

But we could do that now, if SE were to give us a thumbs up on that

tripleee

or if you didn't care about the consequences /-:

Undo

Crank min accuracy up to 99.95%, set max_flags to 5/6, let it go. See what happens.

tripleee

@Undo you could go for 100.0%, it would still flag quite a lot of what it currently does

Undo

At 100% accuracy, you lose a ton of volume just to get rid of single-digit numbers of false positives.

Jan Dvorak

3:46 PM

Maybe you could cast more autoflags at higher accuracy

Undo

That'd be an option

tripleee

@Undo I speculate that the ton of volume is mainly very old messages which do not trigger current rules, can you do a similar plot for messages less than three months old, for example?

SmokeDetector

[ SmokeDetector | MS ] Offensive answer detected: What does the word "pinche" mean? by Eric Lee on spanish.stackexchange.com

fp- by Glorfindel

Undo

@tripleee could, requires a bit of time to put together

Can't right now.

tripleee

@Undo speculative anyway, just thinking out loud

Glorfindel

3:52 PM

@tripleee I support this POV. Since you (the 'development team') have been making concrete progress on autoflagging, tripleee, Magisch and I (and probably a few others) have been adding blacklist entries not only to detect previously undetected posts, but also increasing the score of the vast majority of posts.

Thomas Ward

while I've been poking things to improve the chances that user error won't torpedo smokey

(and other optimizations)

we're all doing our part in making everything work smoother :)

tripleee

the gaffe I had earlier with !!/blacklist-username, is that something which should be fixed as well?

I would never have imagined that it would break two tokens up into two separate blacklist entries

Thomas Ward

@tripleee what broke?

oh

tripleee

I did !!/blacklist-username two parts and it added "two" and "parts" as distinct entries to the username blacklist

Glorfindel

I think that happens to all blacklist operations, right?

Thomas Ward

3:54 PM

that's by-design, the inputs are split by spaces

Glorfindel

I mean, this works the same as your average *nix console.

tripleee

well, I never would have thought of trying that

Jan Dvorak

your average *nix console accepts quoted arguments

tripleee

@Glorfindel yeah, granted, but it's kind of clunky to have to backslash literal whitespace in the shell, too

since nobody ever used that before (to my knowledge) maybe the design could be rethought?

Glorfindel

@tripleee yeah, I agree with this.

Thomas Ward

3:56 PM

I think we're going to have a design conflict then

what if we want to blacklist five things in one go

Glorfindel

That doesn't happen.

Thomas Ward

do we really want to have to execute five separate git requestsw

Jan Dvorak

Then we issue five commands

Thomas Ward

the only other thing I can think of is try to not split on spaces, but that'll be some work

Andy

@ThomasWard Does it matter that we do 5 in a row?

Jan Dvorak

3:56 PM

Smokey could wait a minute before firing off a git push

Glorfindel

It happens sometimes that I want to blacklist a keyword and a website, but that's two separate commands anyway.

Thomas Ward

@JanDvorak won't work by design, each command creates its own blacklist branch, which is then pushed if code powers are available into Master, or pushed into its own branch for a PR on GH

if you don't have code power

Jan Dvorak

I guess we're fine with pushing multiple times if we're pushing to GIT anyways

tripleee

why optimize a case which nobody has ever seen in practice?

ArtOfCode

^ he's got a point

tripleee

3:58 PM

there should probably be a way to do multiple blacklist entries in one go but my current approach to that would be a pull request anyway

Jan Dvorak

maybe a separate command?

Thomas Ward

I think message parsing would need to be handled differently then possibly

@ArtOfCode where do we parse the message into message_parts for a given command?

Jan Dvorak

or, since the argument is a regex anyways, you could just use pipes to blacklist several keywords at once

Thomas Ward

'cause that seems to be where the core issue might arise

ArtOfCode

@ThomasWard we do that globally for every command. If we were doing this, we'd have to join them.

tripleee

4:00 PM

@JanDvorak yeah, though we should probably try to make the blacklist files more regular, not less

ArtOfCode

On the other hand, you can just use \s for spaces.

Thomas Ward

@ArtOfCode I'd rather force the use of \s for a space then, and put that in the wiki about how to use said commands

rather than try and beat the hell out of the parser system just for blacklist functions

tripleee

@ArtOfCode the reason I brought this up is that if your expectation is for one blacklist entry to land more or less verbatim in the file, you could create really FP-prone phrases like "or" in the blacklist by mistake

Jan Dvorak

@ThomasWard the stage after parsing could join the arguments by space

tripleee

or the sanity check should simply disallow multiple arguments and say use \s if you mean space

Thomas Ward

4:03 PM

@JanDvorak By the time it gets to the blacklist commands in the code if there's any 'spaces' they're already lost by the split

Andy

@ThomasWard I don't like this idea. You are depending on people reading the documentation and then remembering to use \s every time

Thomas Ward

no way to know where the space was supposed to be, unless we use \s to denote the spaces in regex

@Andy It'd need a massive refactor then so everything can handle a joined string instead

at least AIUI

Undo

Executive decision: We should probably just assume one blacklist entry. Create a special separator if you really really want.

5

Magisch

@Undo That'd be higher then our 280 score now

Thomas Ward

still doesn't tell me where the message parsing into message_parts is done, I'll go hunting

Magisch

4:04 PM

I'd say 99,8 some % is good enough to go for 5/6 flags

Andy

@ThomasWard it's in chatcommunicate.py

Magisch

@Undo Agreed. There has, so far, maybe been 1 use case total where you'd need that seperate things blacklist BS

Andy

Starts here but then is reassigned a few times

Undo

@Magisch 280/1/1 gives us 99.98 or 99.99, depending on your sites.

Magisch

On the other side, you're relying on people reading nonobvious documentatio

Thomas Ward

4:06 PM

hmm

message_parts = re.split('[ ,]+', content_source)

^ that's where we currently split the message into parts

for everything that goes to a command

Undo

any reason we can't just do " ".join(message_parts[1:]) as a quick-n-dirty solution?

Andy

Keep reading though. message_parts is reassigned later

Jan Dvorak

oh boy, that's not good

Magisch

@Undo Hmm. Then no objection. Maybe make different thresholds for different flag amounts

Like 99,0% 1-2 flags by us, 99,5% 3-4 and 99,9% 5

Undo

@Magisch yeah, that'd be possible

Thomas Ward

4:07 PM

@Andy if it's sd in the first location (index 0) yeah, it goes into a different block of code

Glorfindel

@Magisch we need to account for sites like Workplace and EL&U which (currently) have a nuke-level of 3 instead of 6.

Thomas Ward

otherwise it's passed as-is to the command handlers

Magisch

@Undo So we're still eagerly awaiting SE's word on this

has anyone heard from pops recently?

Jan Dvorak

Another option might be to defer the autoflags until sd k's come

Thomas Ward

@Undo the hackish dirty way is that join

Undo

4:08 PM

@Magisch I suspect you'll see it here when someone does

Thomas Ward

in the blacklist commands

Magisch

Thats actually a pretty nice idea

Thomas Ward

if that's what we wish to do, we can.

Undo

@JanDvorak That'd be an option, yes, but at these kinds of accuracies...

Magisch

How about we do 5 autoflags but only when the threshold matches and on one TP

So we'd essentially delegate the one-button nuke to a user in here in cases that are statistically impossible to mis-judge

Undo

4:09 PM

dogs

Glorfindel

dogs on keyboard?

Undo

@Magisch Remember, all that work would be to avoid literally one false positive.

Magisch

I mean if you say it like that

leave it auto

:D

Undo

Two if you squint just right

Magisch

I'm curious what those 2 famed FPs are

do you have links?

Undo

4:10 PM

I pulled them up once.

Jan 1 at 21:45, by Undo

I'm going through the false positives caught by that condition.

Magisch

I'm very curious how someone manages to trip 3 high accuracy spam filters of ours on a FP

metasmoke.erwaysoftware.com/post/24413

is not a fp

in fact we should invalidate that FP

because it isn't one

Undo

Problem is, those are actually Korean characters we can't see

That was back before we handled Unicode with some semblance of sanity.

Magisch

do we have them raw

Undo

also, it's on Ask Patents and that's a weird site.

Magisch

or should I get someone from ask patents to give me the raw text

Undo

4:13 PM

I could probably acquire a screenshot, gimme a sec

Magisch

If we're dealing with numbers this low, it's very important our FPs are actually fps

so as to not unduely skew our numbers towards needing a higher weight

we're essentially, even in the worst accuracy currently allowed score, talking about <30 FPs in 40k+ posts

Thomas Ward

@Undo @ArtOfCode github.com/Charcoal-SE/SmokeDetector/pull/459 for your review

Undo

@Magisch Pending me finding someone with deleted-post-viewing powers on AP, of which there are like two.

Magisch

@Undo site moderators?

Undo

There's only one, because AP is weird

CMs are the best bet. I'll find one eventually

Thomas Ward

4:18 PM

what's AP?

Undo

Ask Patents

Magisch

I see 2 site moderators (excluding joel) and 1 10k user

Thomas Ward

ah

Magisch

right

Undo

IIRC George actually work(s|ed) at the patent office.

Magisch

4:19 PM

would make sense

Undo

@ThomasWard Looks good to me

Magisch

> "I am a former entrepreneur, an electrical engineer, and a now a patent agent located in Southern California."

Undo

!!/pull

SmokeDetector

@Undo CI build is still pending, wait until the build has finished and then pull again.

Undo

details, details

SmokeDetector

4:20 PM

CI on 28d3171 succeeded.

Thomas Ward

heh

Undo

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 28d3171 (Undo1: Merge pull request #459 from Charcoal-SE/blacklist_join_messageparts) (running on Undo's EC2)

Restart: API quota is 4892.

Thomas Ward

I bet you guys enjoy having someone who can stab code while you guys work on autoflagging, etc. :P

makes things move a little faster

Undo

aye

Thomas Ward

4:22 PM

though a slightly higher chance of incendiaries going off in the code base.

Andy

I don't know how yet, but I'm going to slip in a change to make Smokey sarcastic (like this (youtube) or this (reddit))

Magisch

I'm in way over my head & training discussing autoflagging with undo

Thomas Ward

heh

Magisch

< Second year programming apprentice with very little idea of anything outside c and vba

Thomas Ward

@Magisch i'm self-taught in python, so... :P

gotta know some for Metasploit stuff anyways

SmokeDetector

4:24 PM

[ SmokeDetector | MS ] URL in title: VVVArt.com reviews and reviews vvvart.com by user349227 on meta.stackexchange.com (@dorukayhan)

Andy

@ThomasWard :)

Thomas Ward

gotta know some for Ubuntu packaging too, all the apport bug handler hooks are Python :P

wow that was blown away fast o.O

@SmokeDetector tpu-

Glorfindel

@SmokeDetector k

Undo

Possibly one of the best XKCDs:

Plus, time's all weird in there, so most of it probably broke down and decomposed hundreds of years ago. Which reminds me, I've been meaning to get in touch with Yucca Mountain to see if they're interested in a partnership.

2

Jan Dvorak

I love the last panel

Glorfindel

4:27 PM

@tripleee I uploaded my userscript.

Thomas Ward

@Undo do let me apologize far in advance though, it's entirely possible the codebase'll be littered with IDE suppressions eventually. (Not my fault I like PyCharm :P)

Undo

@ThomasWard Nah, that's fine

Mithrandir

@Undo chat.stackexchange.com/transcript/message/34827761#34827761 .

Why is everybody linking that?

Glorfindel

@Mithrandir because it's XKCD.

Thomas Ward

!!/blacklist-keyword this is a test of the emergency broadcast system this is only a test

SmokeDetector

4:32 PM

@ThomasWard Blacklisted this, is, a, test, of, the, emergency, broadcast, system, this, is, only, a, test

Thomas Ward

(and will be reverted later)

oop?

lol?

that's interesting

SmokeDetector

CI on b1a5c63 failed.

CI on bfad7ae succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev bfad7ae (Thomas Ward: Revert "Auto blacklist of this, is, a, test, of, the, emergency, broadcast, system, this, is, only, a, test by Thomas Ward --autopull") (running on Undo's EC2)

Restart: API quota is 4793.

[ SmokeDetector | MS ] Manually reported answer: Java: Enemy follow Player by Deez Noots on stackoverflow.com (@dorukayhan)

tpu- by Glorfindel

Thomas Ward

@Undo so, that didn't work...

oh i figured why

DERP

SmokeDetector

CI on 0a092f0 succeeded.

Thomas Ward

i feel silly since I didn't actually use the joined pattern heh

!!/pull

SmokeDetector

4:37 PM

[ SmokeDetector ] SmokeDetector started at rev 0a092f0 (Thomas Ward: Actually use joined blacklist pattern instead of the parts.) (running on Undo's EC2)

Restart: API quota is 4760.

Thomas Ward

!!/blacklist-keyword This is a test of the Emergency Broadcast System.

what died...

SmokeDetector

Restart: API quota is 4757.

Thomas Ward

did I break it

yes I did fakkk

Mithrandir

!!/alive

SmokeDetector

@Mithrandir Of course

Thomas Ward

4:38 PM

!!/errorlogs 35

SmokeDetector

  File "/home/ubuntu/SmokeDetector/ChatExchange/chatexchange/browser.py", line 224, in _load_user
    user_id, user_name = self.user_id_and_name_from_link(user_link_soup)

  File "/home/ubuntu/SmokeDetector/ChatExchange/chatexchange/browser.py", line 232, in user_id_and_name_from_link
    user_id = int(link_soup['href'].split('/')[-2])


TypeError: 'unicode' object does not support item assignment
2017-01-17 16:38:23.300836 UTC
  File "/home/ubuntu/SmokeDetector/excepthook.py", line 47, in run_with_except_hook

Thomas Ward

wat

wtf type error

oh

yeah I did break it, because we have gitmanager designed to handle multiple objects for a blacklist, not a single item.

That's... going to be interesting. And probably going to need secondary code changes after the one I just pushed.

SmokeDetector

CI on a2c88ad succeeded.

Thomas Ward

don't pull it yet

i removed something that I had to add back in

SmokeDetector