@iBug You are not a privileged user. Please see the privileges wiki page for information on what privileges are and what is expected of privileged users.
YO. I may have just made a breakthrough with Smokey+ML, thanks to @thesecretmaster. Ya know how we've never managed to build a classifier better than 50/50 before? I have a 95% accurate classifier.
and that's without any attempt to improve it off the baseline
of course manual efforts would be able to be more accurate with tp’s, but the fact that the computer can reach that accuracy without telling it the logic to use is still impressive
@Rob Don't you? I don't use autoflagging myself and it's been a long while since I looked at the tools, but it was in the form of userscripts once upon a time. Check out the wiki for what's current.
@BrockAdams Post 1: Could not find data for this post in the API. It may already have been deleted.
> Would not be caught as a post, title or username.
> Would not be caught as a post, title or username.
> Would not be caught as a post, title or username.
> Would not be caught as a post, title or username.
> Would not be caught as a post, title or username.
:41731324 > Repeating characters in body, repeating characters in title, title has only one unique char ---------- Title - Repeated character: *aaaaaaaaaaaa* Title - Position 1-13: aaaaaaaaaaaa Post - Repeated character: *aaaaaaaaaaaa*
:41731327 > Would not be caught as a post, title or username.
:41731340 > Would not be caught as a post, title or username.
> Offensive body detected, offensive title detected ---------- Title - Offensive keyword: *private show* Post - Offensive keyword: *private show*
@iBug Post 1: Could not find data for this post in the API. It may already have been deleted.
tpu- by ArtOfCode
Recovered from BrokenPipeError: [Errno 32] Broken pipe
@Undo I think we just repro'd your weird reply thing by the way
Looks like it's caused by a failover
@ArtOfCode @thesecretmaster OK. I've had an idea for an ML approach that's pretty different from what we've done in the past. I've pitched this to @tripleee before but I put it to the side when working on chatcommunicate
Here's the thing: once we discover a spam domain, nothing will ever be as accurate as the blacklist. That's why we make reasons based on discovering new domains and keywords: things like link at the end of post, pattern matching product, bad NS...
I'm thinking an ML approach should focus on the keyword or domain name level, not the whole post level:
Basically we extract out really rare or never before seen keywords, or URL domain names and tails (kind of like what Half-Life) does and run them through something that goes character by character -- I was thinking some ensemble form of NB
When you see the really blatant spam stuff you can tell just by reading the product name that it's spam. it's marketing health buzzwords
Currently you're operating on a list derived from each word of the post body. What I was talking about was data based on what characters are present in the URL domain we extracted
@ArtOfCode sklearn has a random forest I believe
but heres how it works:
the models themselves are simple, constructing is more complicated. at each node of the "tree" (really a graph) the mdel compares one of the input variables with some value, then takes the appropriate branch. the leaves (endpoints) represent the possible output labels
most of the construction algorithms are greedy -- they pick a variable and comparison that best splits the set of data that's taken the previous branch into the next two branches
hmm. something recurrent might be better though to factor in the order better
the hard part is getting data and extracring the urls and keywords to be honest. we can try a lot of types of models from there
Would it make sense to use regular string parsing to extract the main portions of the urls before using ML to analyze keywords in the domain name and the rest of the post?
Okay. I have SU loaded. Next problem: extracting URLs takes 5 minutes on the MS dataset, about 80k posts. SU has several hundred thousand, if not over a million already. Yeah.
@Undo so... every Smokey has access to the classifier, by making an API request to metasmoke, which forwards it to a capable instance, back to MS, back to requester?
@Undo nah, go totally async and even more crazy: API request made, MS sends 202 Accepted, drops the request. Forward to a capable instance in the background, wait for response. Once response is available, throw it down smokedetector_messages.
@ArtOfCode this is how the (now dormant) DeepSmoke was set up; I have a sturdy EC2 running the classifier and accepting posts to classify over a simple HTTP API
@tripleee Yeah, I'm transporting models around in binary files. Training is intensive, but keeping it loaded in memory takes a significant chunk of RAM
