@ArtOfCode Yeah, that's the term we're using right now. The feasibility of moving forward with any kind of real project at all. The exact shape of that real project isn't defined yet. In other words, does such a project exist.
@Pops It works like a real time review queue for spam, which is what its best at. We catch a broad net of posts, and let those that aren't spam through. The strenght of it lies in the fact that we can have human eyes on all potential spam with minimum hassle
@Pops the real power of smokey is that anyone can help adjust its DB. That would need to be factored into any SE implementation for it to be effective.
In essence, its all just regex matching. But having 10+ people keep ontop of any spam on the network and blacklist websites and patterns as they are used, thats what makes it effective
I see two categories of 'integration' here: One is minor nice-to-haves that'd give us a good deal more potential for catching stuff with minimal developer work. The other is 'real' integration - two-way communication, feedback stuff back to the system itself.
@Seth just noting that because I think that's also a strength - it's not literally everyone and their dog, it's people who we think are sane. Most people, in effect, but there are social filters in place.
I'd think that something like allowing metasmoke to feed into spamram or reducing the number of spam flags needed for nuking when a post gets feedback on MS would be possible
In the middle, it'd be nice if we had a realtime feed of spam we missed. That's stepping into some iffy territory around deletion, but it should be surmountable if we really wanted to do it.
Aye, that seems to have been the dominant integration idea: if Smokey says "this post is spam", the system says "okay, so we're only gonna require 3 spam flags on that to nuke it instead of 6".
We're already pretty good at identifying spam and getting eyes on it. Its just a number from then on out. Many posts get from 0 score to nuked in <30s when people get on it
@Pops Going back to this, I believe it'd be feasible to do with less keyboard-poking. By nature of the system, we have false positives. I don't want to turn this thing loose with power to unilaterally block stuff, there should still be a human involved somewhere.
But this clearly shows the effectiveness of the smokey system on stackoverflow, where there is also a large group of flaggers in the SO Close Vote Reveiwers. However on the smaller sites, deletion is usually to late for compared to google indexing
I actually don't have any questions right now. Was happy to listen to what you had to say, all sounds good so far. I had intended to read up on smokey before coming in here but between out-of-office backlog and the usual pop-up tasks, got swamped with other stuff so far this week. So you have some idea, I am a former CS person but it's been a few years since I've done anything code-y (plus networks were never my strong suit) so I may have lots of stupid questions.
We've got a plan sitting around somewhere for automated flagging, which would effectively send signal to the system without any dev work on the SE side. All we need is the go-ahead on that.
where metasmoke can put feedback it gets, and if there is positive feedback, the amount of spam flags it needs to delete is reduced. That would help to bring the time-to-deletion during prime time for smaller sites down
I'm one of the people who doesn't write code for smokey. I just feed back throughout the day. I have it set up so I get every SD report as a desktop notification while at work and then act on it
As I see it, we've got two potentially separate ideas. One, we automatically cast a spam flag on posts we think are spam (we've got a list of accurate criteria we can use) - that's no dev work, but only weak signal. Two, we set up between SE and metasmoke, and when MS tells SE that we think something's spam, SE reduces the number of spam flags required to, say, 3 - that's more dev work on SE side, but more signal.
@ArtOfCode We could expand on your last idea, more reasons a certain post is caugth by (smokey works with a few rule sets, like "blacklisted user", or "blacklisted website"), the more automatic flags are thrown, up to the maxium of 3
I was going to go looking, but I guess I might as well just ask, how are you set up for outside interaction, on a scale of "you'd have to scrape the page" to "fully documented API and 24/7 customer service with a real human on the phone"?
It really depends on how many resources ya'll are willing to devote to this project. We can go anywhere from us handling everything (automatic flagging) to you handling everything and us just providing the data.
Essentially you could have that in any way you like. You can have SE get the feedback from the site, or we could send it to SE directly via an api. Its open to use for you guys
Okay. This will probably be at least 50% old info to me at 100% old info to you but can you walk me through what happens how smokey handles a single post from start to finish (or tell me why that question doesn't make sense, and what a better question would be)?
@Pops, can you provide a bit of info on how spam detection on your side works? Are all detection rules built by SE, or do you use third party services too?
@Andy You know, I had a moment of mild panic when I came in because I thought I should explain that and then I realized I wasn't sure if that was one of the few non-public things about our company or not.
@Pops Smokey watches the realtime websocket. When it gets a post, it retrieves the full details of the post from the SE API. Runs that through our extensive regexes and check methods. If they come up positive (i.e. this is spam), sends it to metasmoke and posts a report in chat.
1. We get it from the websocket (the one that feeds the realtime page on stackexchange.com) 2. We run it through a bunch of filters, some regexes and some code things. 3. If it matches, we post it to chat and send it to metasmoke. 4. People tell the system if we got it right or not. We send that to metasmoke too.
Step 1.5: the post is put in a queue, and posts are batched up before asking the SE api, because else we wouldn't have any api quota left at the end of the dat
Just popping in here and having to leave again: How about making Smokey “official” by having it feed an SE-wide spam/rude/abusive queue? Should have some strong audits though to kill the robo-reviewers.
And the chat works how, someone on the site manually sets up a bot account and you just say "hey, from now on, if there's a hit on site X, tell chat account Y"?
Would be quite nice for smaller sites, though, and if we could do it network-wide so that reviewers from any site see posts on any site, then we could dramatically reduce spam lifetime on small sites.
A while ago, Tim Post started an initiative to improve spam protection on SE. One improvement that was almost immediately suggested was to add network IDs to the API's shallow_user object.
Doing this would allow existing tools such as SmokeDetector to better pick up on network spammers, which wo...
basically, most people here have chipped in something, whether that's just feedback, or pull requests, or userscripts, or extensions, or hardware and servers to run it on...
we also have people spam flagging the reports (most of us have accounts on nearly all SE communities, mostly with 101 rep. Some people have 500+ flags on tons of sites due to smokey
Yeah its a very iterative effort but it has extended to dozens of people devoting time to nuke spam on all of SE, not just the sites they're usually active in, as fast as possible
Nor do you realise the interactions with site specific chat rooms, until you see the history of smoke detector's messages, he posts in many different rooms
Well okay! This all sounds amazing, and that's really saying something because knowing you all I had high expectations to begin with. (And it makes me feel even worse for not having much to say back to you today.) I'm going to keep poking around both at smokey and on our end. Feel free to ping me if you have further questions/requests/ideas, I will definitely be back in the coming days with more info about my thoughts.
@Pops There are a few more events (undeletion, for one) that would be nice to have in the websocket. It'd also be nice to have a bigger API quota (20-30k would be awesome, although not entirely necessary). Probably the cheapest thing is adding the network ID thing ^.
Network ID would let us track spammers' accounts across the network, instead of just on one site. Don't really have data on how many, but I see them somewhat often.
The whole point of it having minimal data was for the masses: It would be nice to get an official websocket endpoints used for developers who need access to full questions/answers.
@GregP Yeah... I was just thinking about that yesterday. That's really a failure on our end as builders of the system; in a practical sense, of course you're right, but on the other hand, it's technically within the correct definition for close voting (and downvoting).
There really isn't a frame limit on websocket connections. They're only limited, because less data going out to the public the less strain on the servers. 10 bots/developers using this special websocket wouldn't add that much more load
The ideal solution probably involves some dev work from both sides, to make sure that we've got a flexible system that's powerful enough and accurate enough to get rid of things fast. It's what undo said - it really depends how much work SE are willing to put into it.
@ArtOfCode Would it be possible to remove the error messages of NAA answers that I mark that aren't in metasmoke? API should return like a 200 response, not in metasmoke, and should fail gracefully.
The client didn't mess up. It was attempting to submit a report of a possible post on meta smoke. Meta handled the response, and IMO should respond with a 200 and a json object with {"error": "no_post_related"} or something similar. I'm against using standard error codes like 404, when simply moving a folder could trigger apache to server 404's, which would cause the app to fail silently, when there was indeed an error server side.
@zaq Fight spam for Stack exchange just crashed, google chrome showed an error message that it got stuck, and this tab showed the google chrome crash page, no detailed error message
