Reason count, though

reason count is rarely a deciding factor

and... remember, dissecting autoflagging parameters is exactly what we shouldn't do. The system works on large numbers, not on individual cases.

Don't adjust behavior for autoflagging. It's calibrated for what we already do.

(whether that calibration needs re-calibrated is another question)

@JF yes. I'd like to extend it, but never did

If your SE account is compromised, we've already lost.

[ SmokeDetector | MS ] Blacklisted user: Error ! Unable to fetch video, Invalid link or Video Doesn't Exist. Youtube API key? by Leo Luan Elezi on stackoverflow.com

[ SmokeDetector | MS ] Blacklisted user: How to create Youtube API key? for YouTube Converter script? by Leo Luan Elezi on stackoverflow.com

fp by ArtOfCode

fp by ArtOfCode on Error ! Unable to fetch video, Invalid link or Video Doesn't Exist. Youtube API key? [MS]

!!/rmblu stackoverflow.com/users/9355587/leo-luan-elezi

@Makyen User removed from blacklist (9355587 on stackoverflow.com).

fp- by Makyen on Error ! Unable to fetch video, Invalid link or Video Doesn't Exist. Youtube API key? [MS]

This reminds me...

Is there anything to prevent autoflagging such posts in the future?

github.com/Charcoal-SE/SmokeDetector/issues/1779

Yes, I saw that and posted there.

@SFTP sorely tempted to take away the ability for people to set rock-bottom flag conditions

@Undo stuff like 1/3/1 is legit tho

also tempted to say "that's not a system problem, that's a people-taking-on-risk problem"

I think maybe the problem lies in the site selector

People taking on risk and losing has implications for the autoflagging process in general.

It doesn't concern me, because... I know how this thing works. But it's pretty clear that I'm unable to explain how it works to the concerned people.

People only see one layer: "You detect stuff, and now you want to cast five flags on everything"

well no

Maybe judge all the conditions as a whole?

Art says it's too slow, although I suspect it would be fine once optimized

That doesn't mean it can't be done on a rolling basis, during periods that there's a light load. Sure, that doesn't give people immediate feedback, but it would disable the conditions.

From users unfamiliar with Charcoal, they would see any activity carried out on its behalf (including autoflags) as effectively condoned & permitted if it is allowed to happen.

So we effectively have ownership & responsibility for activity a user carries out through the autoflagging system, from a PR perspective, because it's our systems that enabled it & carried it out

For Ask Different by itself, the historical data supports (barely) using 1/1/2 conditions at 99.75%.

@doppelgreener I agree. That is an issue.

People own & are responsible for their own flags cast through our flagging system. That's the CM perspective and our perspective. But it's not the outside perspective, because they're not across the nuances, and stuff like that doesn't matter compared to "charcoal HQ is flagging stuff they shouldn't be at all so smoke detector is unreliable"

Those people don't need to be happy.

@Undo if we want to expand beyond 3 flags I'd argue it's fairly essential

It sounds harsh, but... seriously, we've got to make some tradeoffs. Folks who don't make an effort to understand how this works don't need to be dictating how we're run.

And yes, PR is important, but I won't let it be the overriding factor in anything. We have one goal; PR is part of attaining that goal but is not everything.

It's unclear what solution would be best. Perhaps: all conditions must be both: A) for any sites on which the condition is enabled, where the site has enough detections to be statistically relevant (i.e. > 1000 TP+FP+NAA), the condition must meet 99.75% separately on all such enabled sites; and B) the condition must meet 99.75% when calculated across all sites (not just the ones on which it is enabled).

That meta post laid out a roadmap. I fully intend to follow it, and fully intend to go to five once we're there. Most of it is technically unnecessary, but I get why people are nervous. Just need to find the time to get it done.

I'd rather not have to opt out of autoflags because of too low accuracy...

@JohnDvorak Don't manually change your flagging conditions to specify low accuracy, and you'll be fine :P

@Makyen I like this.

280/1/1 - the default - is currently 40128/1 tp/fp

Oh, OK. Thanks

@Makyen said differently, that sounds like "any filter must be 99.75% accurate for all sites, and have >1000 relevant posts on the sites where it's enabled"

@doppelgreener No, that's not the same thing.

That's less restrictive, or at least that's how I read it at first.

oh! is it: 99.75% accurate taking global statistics for every site into account together, plus, 99.75% on each individual site for which the filter is enabled (considered independently of all others)?

(plus the 1000 posts thing)

@Undo By the way, this is absolutely insane. Three years ago, I'd have said you were crazy if you claimed an automated system even could classify spam at this accuracy. We have it now.

@doppelgreener I read "99.75% accurate for all sites" as only a check across all sites, not on each individual site.

However, I wouldn't require that conditions separately meet 99.75% on sites where it isn't enabled. The reality is that sites have different types of content. Our current filters only weight based on their effectiveness globally. That means that their weight numbers don't necessarily reflect how they actually perform on any single site.

@doppelgreener Might run some data on that - my initial thought is that you'll end up not allowing any flags on small/low-spam sites. Those are, arguable, one of the places we provide the most value.

@Makyen that is what i read into your suggestion's part B), that it must meet 99.75% against global stats from all sites

It would only take one false positive on a site out of ~300 reports to cut a site off entirely. Pretty high bar for small sites.

it'll still get caught & reviewed here though even if autoflags can't get cast

At significant inconvenience for both us and the site when a spammer suddenly discovers a new site

Or, say, our new friend.

@doppelgreener My comment was 1 behind yours. Your comment just above what I wrote agrees with what I was attempting to communicate. I also diverged onto an issue I've been thinking about and ways to resolve it, overall.

@Makyen Cool :)

One thing that I'd like to see data on though - is this lower accuracy being caused by global conditions, or individual ones?

I think it's because of individual ones - so no need to worry too much about the global percentage calculation

(Lmk if I'm getting the wrong end of the stick - joined halfway through the convo)

@CalvT븃 i think you've got more or less the right end of the stick

@Undo At least as I understood it, we have only been discussing applying the per-site requirement on sites with enough data, where "enough data" isn't yet defined, but generically assumed to be at least > 1000.

a message that wasn't really worth flagging got autoflagged probably specifically because someone had a 1/2/1 condition set for that site and for that site that's enough

@Undo how easy would it be to get a percentage of users that have 2 or more conditions?

Or even better - a graph with how many users have x conditions

@CalvT븃 Are you interested in total conditions, or total conditions which are currently enabled? For example, I have multiple disabled conditions, which I've played around with to see what conditions hit various accuracy points. I have no idea if anyone else does, but that might skew the data, depending on what you're looking for.

@Makyen That's a good point. I'm interested in enabled conditions.

Because I'd like to see if it's just a couple of users that have a lot of conditions, or if there's a trend among the more regular people to have a lot of conditions.

Or if I'm wrong and there's actually no users with more than say 5 conditions

I personally have 2, and the second I am a bit worried about, as it's a 1/3/1

Get a bunch of potentially bad title, potentially bad body, and luncheon meat (which are all pinned at 1 iirc) and it'll flag them

But I can't be that worried as it's still enabled

I'm wrong about luncheon meat actually - it's not pinned

[ SmokeDetector | MS ] Offensive answer detected, potentially bad keyword in answer: Why does Candace never take a photo of boys' inventions? by Random guy on scifi.SE (@Mithrandir)