I hawk every other cent when it comes to personal expenses.

yay my massive stockpile of water I ordered is on its way

and so's my upgraded internet modem yay

It's come so far that I developed a liking to beans and rice

@Magisch Hi Magisch; I'm not replying to your message, but rather pinging you to discuss the case of Thierno Sow. I'm a moderator at MathOverflow. I have some noob questions. How close is the relationship of SD or Charcoal to SE Community Management? If I want to ask that certain items get blacklisted, is it considered the done thing to get CM oversight first, or do SE site users normally interact with SD more independently? Can character strings get blacklisted? More questions to come.

I discovered easy budgeting software, and it's completely changed our lives around money. It's amazing how having the dollars all mapped out into the future (including plans for how to pay off debts) makes money not stressful anymore.

@ToddTrimble Hey

I'm not an admin here, you're looking for @Undo or @ArtOfCode or @ThomasWard

I can blacklist websites keywords and usernames though

A few people can

@ToddTrimble So for a quick overview of what this thing is, have a look at charcoal-se.org

we have many different filters, and the project is always open to pull requests

TL;DR: SmokeDetector scans all new and updated content network-wide, trying to detect spam based on common characteristics and a number of blacklists.

So if you go to our github you can suggest a filter. If it makes sense, we'll add it

This is basicly a real time spam review queue

I think Thierno Sow is already a blacklisted username

Smokey scans all posts and reports the ones that match one of our filters

Right?

Our filters range from simple keyword based regex blacklisting to complicated regexes to catch some of our more ... evasive spammers

@Magisch It's amazing how accurate that analogy is

And it's also amazing how we keep having this conversation every time

Restart: API quota is 2687.

@ToddTrimble It's exciting to have a published and accomplished Math guy in here :D

[ SmokeDetector | MS ] Few unique characters in body: how can i safely write to a file when using Flask? by zaphod on stackoverflow.com (@dorukayhan)

[ SmokeDetector | MS ] Few unique characters in body: using regex with jQuery to select options from a dropdown menu by zaphod on stackoverflow.com (@dorukayhan)

[ SmokeDetector | MS ] Few unique characters in body: jquery ajax call to a remote php file failing by zaphod on stackoverflow.com (@dorukayhan)

@SmokeDetector tp-

SD 3V

[ SmokeDetector | MS ] Few unique characters in body: title is missing by zaphod on stackoverflow.com (@dorukayhan)

@SmokeDetector v

[ SmokeDetector | MS ] Few unique characters in body: title is missing 123 by zaphod on stackoverflow.com (@dorukayhan)

Somebody stop this guy

I've modflagged

Bhargav is already on it

@SmokeDetector v

Mod-flagged

Oh

[ SmokeDetector | MS ] Blacklisted website in body: Can you recommend me other way for easy Chine languages learn? by karamel on chinese.stackexchange.com

tpu- by Kyll

Chine? Seriously?

API quota rolled over with 2300 requests remaining. Current quota: 9999.

[ SmokeDetector | MS ] Bad keyword with email in body: failed to run pytomo crawler by Beta on stackoverflow.com (@dorukayhan)

[ SmokeDetector | MS ] Blacklisted website in answer: Where can I find a well-organized free french lessons website? by karamel on french.stackexchange.com

sd k

is my presence still needed or no?

I was pinged during class :P

No, not really. A MathOverflow mod got interested in our little project (after we encountered a rather odd spam post this morning).

Cool

goes back to braving the winter storm

@Glorfindel any chance you could give me a link to that meta post you made? On mobile and I don't want to trawl through the transcript

crap

@DavidPostill thx

!!/blacklist-website one-zero\.eu

@Glorfindel Blacklisted one-zero\.eu

^ seems to be the main website of said crank/impostor/troll.

We've encountered it once before, although in the form of a shortened URL.

CI on d4e724e succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev d4e724e (SmokeDetector: Auto blacklist of one-zero\.eu by Glorfindel --autopull) (running on teward/Aurora)

Restart: API quota is 9467.

@Glorfindel Yeah, I've got two different references (URLs) to it in my browser history :)

Charcoal HQ - this spammer: one - zero.

!!/rev

@ThomasWard d4e724e (SmokeDetector: Auto blacklist of one-zero\.eu by Glorfindel --autopull)

!!/location

@ThomasWard teward/Aurora

good

that is still working

@ToddTrimble I don't think anyone answered your first question about our relationship with SE; basically we're in good enough terms with them that they're considering integrating our system into theirs. Right now however nothing much has happened apart from some basic talks, as @Pops (the CM who deals with us) has been a bit busy lately.

^ that, plus tl;dr: none of us are staff, and this isn't an official SE project

We have complete control over how our system works, so there isn't any need for you to ask the CM team or anything before we blacklist keywords or add new checks.

@ArtOfCode yeah, that

Also @ToddTrimble, sorry if it looked like a bunch of us jumped in and let you know about our opinions etc, we just like to keep close tabs on everything happening around the network related to spam.

:)

@angussidney for the future: what should I do to avoid that? Ideally, I'd like to just have posted a question and allow the 'local' community to react. I understand you're all eager to vent your opinions about spam, but if I'd be interested in them, I would just ask it here :)

I keep in mind the little problem we had with Graphic Design after one of the mods locked a spam post and some of us got a little angry about that.

@ArtOfCode I wonder, do CMs have pull access to this?

@Magisch to the !!/pull command? Yes

So if for instance shog came in here and wanted to BL a keyword, could he?

@Magisch No.

He could use the command, but it'd go through PR; or he could make a PR on GH directly.

Hmm

I wonder if we want that

or not

Blacklist control is managed by the code admin permission, which is applied via metasmoke.

@Magisch I'd say probably not, for the same reason moderators don't have access to it. There's probably context or experience or convention you need to be able to accurately blacklist with Smokey, which CM's and/or moderators don't have.

@ArtOfCode Ik, just thought there was a similar mod exception there

point taken

So basically I have more privileges than Shog? o_O

@Glorfindel Yep.

O_o

o_O

Is Shoggy like the epitome of a CM? He's always mentioned as an example.

Shog is the shog

Don't call him shoggy if you like your account

Which isn't surprising, really, given that we own all the infrastructure this runs on - it just interacts with SE systems.

Shog is the team member with overall responsibility for Q&A.

I don't own anything of this

I'm basicly only in here flagging things and playing with regexes

@Magisch "we" as in charcoal the organisation

All of us? Or all of the admins?

Are we an official organization even?

@Magisch define official

registered somewhere

if official is just has some sort of structure and has a website, then yes we are

we're not registered, no

@Magisch have a look at /people on the website

I like charcoal

@ArtOfCode .html

Undo runs metasmoke; a bunch of us have creds to run Smokey, but anyone can run it if they have a server

@Mithrandir also works without, actually

@angussidney Angus, it's totally cool. It looks like you and @Magisch and @Glorfindel and @ArtOfCode were all being circumspect, and anyway you made absolutely the right call on this Thierno M. Sow. (What he calls mathematics "isn't even wrong"; it's gibberish on so many levels, like type mismatches, that you can't even assign a truth value to it.) Anyway, glad you all stepped in.

Welcome, glad to flag stuff. :P

@ToddTrimble we're all glad to help and make Stack Exchange a better place. We are pretty good at catching spam once we see a pattern, but we sometimes do need help in identifying those patterns. As I said on Meta MO, if you ever see such a pattern, please post it here and help us increase our awareness level.

[ SmokeDetector | MS ] Manually reported answer: How to set up Spark on Windows? by Chad Mcvay on stackoverflow.com (@dorukayhan)

tpu- by Mithrandir

Why can't I flag that? The only option is "In need of moderator intervention"

ah, it looks like it was deleted but it didn't notify me

@NobodyNada Was already deletd

@NobodyNada Race condition :)

Maybe that happens if the post is deleted in between the server generating the HTML and my browser connecting to the notifications websocket

You must have made the request literally milliseconds before the post was deleted for that to happen

I've seen it a few times, so it's not that rare.

New metasmoke user 'Jamal' created

@ArtOfCode Re the conversation we had, the denizens of the main RPG chat are cool with having Smokey post messages there, especially since the RPG spam is fairly low-volume. Do we need to do anything special to set that up?

@SevenSidedDie I add a couple lines of code to some config, deploy Smokey, and it happens :) Gimme about three minutes

@ArtOfCode Sweet! Much appreciated.

CI on b67c64b succeeded.

!!/pull

[ SmokeDetector ] SmokeDetector started at rev b67c64b (ArtOfCode: Add RPG special room) (running on teward/Aurora)

@Seven done

Restart: API quota is 8887.

@ArtOfCode Cool. Now to wait for spam. (… It's bad that I'm hoping for some spam soon to see it in action, isn't it. Nerd problems.)

If I'm mostly contributing to flags from posts generated in this room, then do I not need to bother with autoflagging?

@SevenSidedDie Nerd problems are the best kind of problems to have :)

@Jamal What do you mean?

I don't think you're eligible for autoflagging sign up at the moment, actually

I figured, since I didn't see a checkbox on the page. I guess I'm also not exactly sure how that works.

@Jamal there's a specific permission that has to be manually applied to your metasmoke account before you can sign up; we add the permission when you reach 50 feedbacks, if you want it.

In other words, when I reply to a SD message with the valid review for a post?

@Jamal Yeah, in effect

@SmokeDetector k

^ that's feedback

Okay. I did do that once, and it acknowledged that it was my first one.

@Jamal Ah yeah, I remember that. Not entirely sure what's happened to it, because it isn't associated with your metasmoke account. Did you go through SE authentication?

Probably because it was done before I created my account (today). No big deal. I was told that the post would've been deleted anyway.

CI on 6f0c4aa failed.

@Jamal possibly - we might also have a bug around retroactively associating feedback with accounts there, but I'd have to look into that

@SmokeDetector Looks like the link is goo.gl’d now.

CI on fbab672 succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev fbab672 (Undo1: Revert "Change 'repo' link to point to info page by default (#488) --autopull") (running on teward/Aurora)

Restart: API quota is 8812.

@ArtOfCode pretty sure the bug is "we don't"

@Undo that could be called a bug, yes

On /status:

read as "we have a long stretch where we do nothing"

time stamps?

also is that 1 day or ...

one request

metasmoke.erwaysoftware.com/status?pp=flamegraph

Also, homepage:

@Undo that gap is Passenger: RequestHandler#wait_until_termination_requested. Sounds like a bug?

@ArtOfCode Puma does the same thing locally

puma-3.6.2/lib/puma/single.rb:38:in Puma::Single#run'`

Huh. What's it doing for so long, I wonder?

that's the golden question

I half wonder if it's a SQL thing. Was seeing the same gaps in prod on 14.04, though

What's the timespan of the full graph?

/status shows miniprof timings of 200ms-1s, depending on the weather

So if those gaps are SQL, that's a 150-750ms long SQL query, which is insanely long for SQL.

'specially on an SSD. I see query times between 0.1-3ms locally on my HDD

not an SSD right now, been running on a magnetic drive for a long time with no performance issues

until 16.04, of course.

@Undo Aren't all EC2 machines on SSD?

Not necessarily

Huh, I thought they were

You can have gp2 (ssd), pIOPS (something fast), or magnetic (spinny disk, or at least simulated spinny disk)

btw, is it a bad thing to be bored enough to find the first commit of smokey's repo?

> Amazon EBS provides four current generation volume types to best meet the needs of your workloads: Provisioned IOPS (SSD), General Purpose (SSD), Throughput Optimized (HDD), and Cold (HDD). Provisioned IOPS (SSD) are SSD-backed volumes and offer storage with consistent and low-latency performance, and are designed for I/O intensive applications such as large relational or NoSQL databases.

> General Purpose (SSD) are general purpose volume types suitable for a broad range of workloads, including small to medium sized databases, development and test environments, and boot volumes. Throughput Optimized (HDD) are HDD-backed volumes, best used for frequently accessed, throughput intensive workloads. Cold (HDD) volumes provide the lowest cost per gigabyte of the EBS volume types and are ideal for workloads where data is accessed infrequently.

@ArtOfCode here's a page with no SQL queries

@Undo shoulda called it /dev/null

:)

lol

@Undo what type of EBS volume is it running at the moment?

@ArtOfCode 'standard'

not gp2

We could try it, but I'm skeptical

I thought gp2 was standard...

Historical naming, probably

It does the same thing on my dev machine, with SSD and all

@Undo I dunno, actually. The DB size isn't trivial any more - we're talking probably 200,000+ rows. SSD might make a difference.

heck, we can try. Hold on.

Even if not to the graph, it might make a difference to overall speed

metasmoke is down and I know about it

!!/report scifi.stackexchange.com/a/151536/31394

[ SmokeDetector | MS ] Manually reported answer: What exactly is cellular peptide cake? by biosyn on scifi.stackexchange.com (@Mithrandir)

Restart: API quota is 8643.

snapshotting takes far too long. It's one of the things I most dislike about EC2.

Restart: API quota is 8588.

@ArtOfCode done. I actually think it's slower

shouldiblame[thelackof]caching.com?

Amazon’s probably got some kind of optimizations on the filesystem.

^

@Undo by about 150-250ms, yes. The question is whether the DB indexes came over to the SSD?

If they didn't, I think MySQL builds them as it goes

computers use spare RAM as a disk cache; Amazon probably has a lot of caching

amazon probably isn't doing anything special like that

@ArtOfCode I just snapshotted the whole thing

like, the whole disk

I'm going to try MariaDB

shrug I'm kinda stuck why it's slower on an SSD, then, but the profiler doesn't lie

@ArtOfCode suspect it's 16.04 being evil, somehow

It's possible that we actually have fewer IOPS on an SSD than a magnetic

dunno

I still think it's because of caching; I'd check again in a few hours.

it shouldn't ever be this slow

Restart: API quota is 8537.