@JohnDvorak Hmmm... I hadn't thought of that issue. It'd be interesting to see how long such an effect takes to occur. Assuming we move to fewer things being actionable (i.e. 4/5 autoflags), then we will need some way to make it clear to people when there's something for them to act on, or that there's nothing currently actionable.
OTOH, it's been said that it's desirable to have a minimum of 2 feedbacks per post. We appear to end up with a significant number that only have one feedback.
@undo I have been concerned that the prominent reminders in here that the test is ongoing might skew the results. The data is currently matching our expectations, so it's probably not that much of an issue, but...
@Undo My main concern is for the numbers are where the # autoflags are low (i.e. 1 or 2). Those require either higher participation levels here, or action by people from within the site. I see a lot of those be deleted prior to getting the level of feedback here which would indicate they are deleted due entirely to people via MS (i.e. most get participation from in-site users or moderators). But, there are a few outliers which either very rapidly get large numbers of feedback from here.
My concern is that the test is generating more participation here, but that may be an artifact of people not normally providing confirming feedback when they see the post is deleted.
@JohnDvorak I'd assumed that there would be a moderate delay in such reporting to allow those that are actively watching the various rooms to take care of a report and only notify someone who doesn't have the tab as the active tab when something still needs action.
@Undo That could give an idea as to there being any skew. Probably only comparable on those posts with 3 autoflags: response within test period vs. "normal" response. Hmmm... even just comparing the average TTD number for the 3 autoflag posts in and out of the test period would probably give an indication.
Totally random musing: The main benefit from five flags might be on the outliers - the spam that lasts way longer because there's only one or two people around to flag it now gets deleted with the first flag.
@Undo Question. If a site comes to you with a clear meta consensus that they don't want it, will you still disable it? Even if you, after having considered it, still think it is better for the site to have it?
You see, part of the problem here is that you (charcoal), perhaps without meaning to, give out the vibe that you get to decide whether your system is active on a site instead of the site's users. That is the most important issue here, I think, and the one that's at the core of all the recent tensions.
I mean, if the users are idiots (and this has been known to happen) and want to do all the spam handling themselves, they should have the right to decide that.
I think it's fair to say we want to control how we detect spam, mostly from the concerns in that GH comment. That can pretty easily look like wanting to control everything.
But yes, if there's a clear consensus to turn the whole thing off, or tone down autoflagging, both of those are easily configurable and easy to respect in the future.
@terdon Worry: would it be difficult to get such a meta consensus, given that Charcoaliers have accounts on all sites (therefore, can at least upvote, even if not downvote) and monitor all metas for posts about spam? IOW: if you post a meta about turning off Smokey, what's to stop a bunch of Charcoaliers voting on it even if they aren't actually active users of the site?
@Randal'Thor But why expend the time of site-users to do drudgery which can be automated? One way to look at that is "Please let our users do more unpleasant work, while still more are exposed to unpleasant things." This is a bit tongue-in-cheek, but there's an underlying issue there.
[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +3 more: dietplanusa.com/up-a-cup/ by noshibiker on graphicdesign.SE
@Makyen There's a difference between the users saying "we want more drudgery work" and moderators saying "our users want more drudgery work". We have an allergic reaction to the second; I think we could do just fine if the users actually said that.
@terdon There might be another aspect hidden in there. Many users actually enjoy flagging such things and would not like it when the opportunities are taken away by some unbeatable FGITW flagging machine.
@Undo The problem there is that i) those outliers are veeery few, ii) I suspect they are borderline spam where people might have had differing opinions on whether it is even spam and iii) I dunno if that's site specific or network wide and the numbers vary enormously between sites.
@Undo I'd be interested to see more info on those outliers. Why are they taking so long to be nuked - is it because they're on tiny inactive sites, because they were less clearly spam, because of some other case-specific issues?
@Randal'Thor They're smokey true positives. I think they're all autoflagged but one of the charcoal folks would understand if my SQL query actually does what I think it does.
@Undo Well, that seems to be a good number of outlying datapoints: enough to draw some kind of broad conclusions about reasons, but few enough to maybe look at each one individually?
@Randal'Thor It should be the time from creation to deletion (time to deletion) among those posts marked as true positives, not "not an answer", and posted since February 2017 on the selected sites.
So... I was just thinking about this over Saturday. Is there any specific reason we allow people to pick autoflag weights that have a less than 100% accuracy?
@Undo The leading cause of false positive autoflags is people picking less than recommended. If we enforce that recommendation, we cut down on false positive autoflags and I don't think we lose many true positive autoflags.
I'm confused: why does the regex \bkys\breturn no results while the regex kysreturns plenty, several of which have kys surrounded by word boundary characters?
[22:01:43] AttributeError: 'NoneType' object has no attribute 'items'
2018-03-03 20:01:43.803599 UTC
File "ws.py", line 112, in <module>
chatcommunicate.init(username, password)
File "/home/[redacted]/SmokeDetector/chatcommunicate.py", line 78, in init
parse_room_config("rooms_custom.yml")
File "/home/[redacted]/SmokeDetector/chatcommunicate.py", line 109, in parse_room_config
for site, site_rooms in room_dict.items():
[22:01:49] metasmoke host not found. Set it as metasmoke_host in the config file.See github.com/Charcoal-SE/metasmoke.
