Charcoal HQ

tripleee

1:00 PM

!!/errorlogs

SmokeDetector

Restart: API quota is 6146.

tripleee

!!/errorlogs

still no?

SmokeDetector

Restart: API quota is 6144.

Glorfindel

!!/errorlogs 50

SmokeDetector

  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)

  File "/home/ubuntu/SmokeDetector/ChatExchange/chatexchange/browser.py", line 709, in _runner
    self.on_activity(json.loads(a))

  File "/home/ubuntu/SmokeDetector/ChatExchange/chatexchange/rooms.py", line 85, in on_activity
    event_callback(event, self._client)

  File "/home/ubuntu/SmokeDetector/chatcommunicate.py", line 170, in watcher
    message_id=message_id)

  File "/home/ubuntu/SmokeDetector/chatcommunicate.py", line 237, in handle_commands

Glorfindel

1:01 PM

@angussidney see above, the optional parameter for !!/errorlogs might not work as intended

SmokeDetector

[ SmokeDetector | MS ] Link at end of answer: C# Project Google map by hjojohan on stackoverflow.com (@dorukayhan)

tripleee

@SmokeDetector very Swedish

SmokeDetector

fp- by tripleee

Mithrandir

Isn't it naa as a link-only answer @tripleee ?

tripleee

@Mithrandir the other answer is a library recommendation too, not sure really

SmokeDetector

1:06 PM

[ SmokeDetector | MS ] Link at end of answer: Google Maps for ASP.NET c# by hjojohan on stackoverflow.com (@dorukayhan)

naa- by Mithrandir

tripleee

@Undo @ArtOfCode the error log above is uninformative at least to my untrained eye, any insights as to what went wrong there? I can see how the blacklist attempt would have been problematic (after all, that's what I was trying to test) but why did it crash again after that?

and sorry for making a mess, I didn't quite expect it to be this bad

Petter Friberg

maybe report? stackoverflow.com/a/41719069/5292302, it translates to nothing... gibberish?

Mithrandir

As what? Rude or abusive?

tripleee

looks more like SOCVR fodder but yeah, flagged and downvoted

Google Translate does give something for Hindi, there are a couple of perhaps crucial words it doesn't know but something about a crazy brother

Mithrandir

mhm

i did naa

tripleee

1:17 PM

yeah

Mithrandir

i like the naa flag

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, pattern-matching website in body, repeated URL at end of long post: kpiling cells and beg by user220563 on apple.stackexchange.com

tpu- by Glorfindel

Glorfindel

@Mithrandir I like all flags.

Mithrandir

@SmokeDetector autoflagged

SmokeDetector

@Mithrandir That post was automatically flagged, using flags from: ArtOfCode.

tripleee

1:18 PM

some Unicode fixes again github.com/Charcoal-SE/SmokeDetector/pull/461

Mithrandir

oh cool I crossed into 800 helpful flags

tripleee

meh flake8 failure

Glorfindel

@tripleee do you need to tell Python every time you're using a Unicode string?

tripleee

@Glorfindel in Python 2 unfortunately yes

Cerbrus

I'm kinda confused how the autoflagging works

tripleee

1:19 PM

the whole code base should move to Python 3 before py2 dies but we still have a few years

Cerbrus

My flags aren't being used

Glorfindel

It's random.

Mithrandir

It chooses a user and uses their flags based on what they set their preferences too

tripleee

@Cerbrus mine neither, I have a fairly conservative filter in place

try the settings Magisch reported earlier today to stand a better chance of being drawn in the big lottery

Cerbrus

Min weight 280, max rep 100

Got a link?

Mithrandir

1:21 PM

I have min 200, max rep 21

And no flags

tripleee

"under-indented for visual indent" just sucks as a reason for rejecting a PR

Mithrandir

It's random

tripleee

particularly when I kept the indents that were already there

Mithrandir

(And a minimum of 2 reasons)

Cerbrus

Does it select a random user from those that can flag that post, or does it get a random user, then check if he accepts the flag?

Mithrandir

1:22 PM

idk

Glorfindel

I think the latter.

Cerbrus

That's kinda silly

Glorfindel

Do you have an account on all major spam targets?

Cerbrus

I doubt I do

Mithrandir

I think I do...

Cerbrus

1:23 PM

@Magisch: What's your autoflagging settings?

Magisch

@Cerbrus Does it say "enabled and flagging" on the dashboard?

More specifically, did you click the big red button here @Cerbrus? metasmoke.erwaysoftware.com/flagging/preferences

Cerbrus

It does, I did

Magisch

It's been known to not work the first time / only work after you set preferences and condition

Cerbrus

Maybe my preferences are too restrictive for sites

Magisch

@Cerbrus My prefs are 180 score - 1 rep - 1 reason, that comes out to a 99,74% accuracy rating over 18k posts.

It's also entirely possible that right now it'll take a while before you get a flag

tripleee

1:25 PM

yeah, it's awfully quiet this time of day

Magisch

we're still running with only 1 (looking to increase soon) and we're in the quiet time of day

tripleee

pessimistic whether we can reach a new daily autoflag record after all

but we beat yesterday's total already at least

Cerbrus

metasmoke.erwaysoftware.com/flagging/conditions/new <-- preview isn't showing

Glorfindel

@tripleee we'll get ±5 more today, I guess (based on past data).

tripleee

@Cerbrus works for me once I fill in the fields

SmokeDetector

1:27 PM

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, pattern-matching website in body, repeated URL at end of long post: Do whatever alpha monster advanced by luminajami154 on stackoverflow.com (@dorukayhan)

Cerbrus

Oh, sites field broke

tripleee

but even if I don't I get NaN%

SmokeDetector

tpu- by Glorfindel

Restart: API quota is 5976.

Mithrandir

Yes the sites field is showing up blank

Cerbrus

Works now after reload

Mithrandir

1:29 PM

There we go.

Magisch

99.74% is good enough for me

tripleee

!!/blacklist-website supplementsoffer\.com

SmokeDetector

@tripleee Blacklisted supplementsoffer\.com

Glorfindel

@tripleee that one is already pattern-matched.

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 6ffaf83 (SmokeDetector: Auto blacklist of supplementsoffer\.com by tripleee --autopull) (running on Undo's EC2)

tripleee

1:32 PM

108 TP, 20+ below threshold metasmoke.erwaysoftware.com/…

SmokeDetector

Restart: API quota is 5939.

Glorfindel

Yeah, but what this effectively does, is give that blacklist entry a weight of (almost) 200 instead of 100.

tripleee

@Glorfindel yes but for a reasonably heavy hitter who avoids the autoflag threshold otherwise, adding a specific blacklist entry with 0 FP seems not only reasonable, but desirable, don't you think?

Cerbrus

Oh association bonus...

Mithrandir

o_o

Magisch

1:35 PM

@Cerbrus DId you go on an account creating spree?

Cerbrus

Ehhh, no?

Glorfindel

@tripleee I'm still not sure. It just doesn't feel right.

Magisch

I'm not that sure either

But whatever floats your boat I guess

Cerbrus

Free rep

SmokeDetector

[ SmokeDetector | MS ] Link at end of body, pattern-matching website in body: Apex Plump Review by Narutto456 on drupal.stackexchange.com

Glorfindel

1:37 PM

I mean, when a post has two bad keywords, it doesn't trigger the filter twice.

Cerbrus

I was missing a couple of sites

SmokeDetector

tpu- by Mithrandir

Glorfindel

@SmokeDetector k

tripleee

to me, "this site's name matches some keyword" is a much weaker heuristic than "this specific site has an all-spam track record"

Cerbrus

So, more newb questions: Why does SD report some users' feedback?

Glorfindel

1:38 PM

!!/blacklist-keyword apex\Wplump

SmokeDetector

@Glorfindel Blacklisted apex\Wplump

Btw @Cerbrus

aye?

Do you know what giving the write key to MS enables it to do?

Cerbrus

1:38 PM

@Mithrandir hax

Mithrandir

@Cerbrus ^

Cerbrus

@Magisch Basically everything.

Magisch

You're basicly trusting undo with your whole account

:p

Mithrandir

I earned a few bounties and joined 100 communities...

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev f5c09ff (SmokeDetector: Auto blacklist of apex\Wplump by Glorfindel --autopull) (running on Undo's EC2)

Restart: API quota is 5894.

tripleee

1:39 PM

@Cerbrus not sure really about "why" but I guess it's occasionally useful to see whether a post gets feedback and what it is

the mechanism is the first feedback from FDSC (and Metasmoke's Review pane I guess?)

Mithrandir

Did Smokey not have ROship before, or did I just notice the italiics now?

Glorfindel

@Cerbrus that's not a newb question. We have had a discussion before if they're really useful. They can be even harmful, as they induce a 'gang' effect.

tripleee

when somebody posts feedback in another room it's not visible here, only in that room

Cerbrus

@tripleee Ah, so if the first feedback from a report is a flag FDSC feedback, it gets mentioned in chat?

Glorfindel

@tripleee feedback from the /review in MS doesn't appear here.

tripleee

1:41 PM

@Cerbrus basically yeah

@Glorfindel right, okay

Magisch

@Cerbrus yeah

Cerbrus

@Glorfindel I'd say, leave'm in.

Magisch

It's so that people don't feel the need to extra feedback

Cerbrus

They induce as much of a "gang" effect as me saying "sd k"

Maybe make the message a little clearer

Magisch

There are people on the network who see us as a malicious voting ring

Mithrandir

1:42 PM

poor them

Glorfindel

@Magisch I thought we are.

We're only malicious for the spammers.

Magisch

I guess if you're very empathethic for spammers we kind of are

We destroy their pesky spam before it can savor the views

Glorfindel

@tripleee I agree with that, but I just don't like this solution for 'fundamental' reasons, whatever those may be. This might be the best solution for now, but what I'd really like to see it something like 'this URL has been TP-ed so many times already, so its weight increases'.

@Cerbrus true, we should abolish that as well, then.

Note that the option to feedback via chat is a lot older than feedback via FDSC.

Cerbrus

I know, that's how we used to feedback...

Magisch

I dont think we have issues with gang effects

lets cross that bridge when we get to it shall we

Andy

1:47 PM

@Glorfindel I have a PR in to fix this.

!!/errorlogs 100

Glorfindel

Speaking for myself: when I see a tpu- on a non-obvious spam post, I'm slightly less inclined to do thorough research on it.

@Andy OK, thanks.

Andy

!!/errorlogs 500

And it looks like the other PR may fix tripleee's issue already

SmokeDetector

[ SmokeDetector | MS ] URL in title, pattern-matching website in body, pattern-matching website in title, repeated URL at end of long post: newsupplements2017.com/somabiotix-reviews/ by Somabiotix Reviews on apple.stackexchange.com

tpu- by rene

Magisch

:o

flower inbound

Mithrandir

@SmokeDetector autoflagged

SmokeDetector

1:50 PM

@Mithrandir That post was automatically flagged, using flags from: ArtOfCode.

Cerbrus

If I want to submit a FR to make feedback messages like this clearer, does that go into the SD repo or in the sdml one?

SmokeDetector

[ SmokeDetector | MS ] Bad keyword with a link in answer: Equalizer levels for minimal hearing loss by Richard Cates on sound.stackexchange.com

tpu- by Cerbrus

naa- by Mithrandir

Cerbrus

Ah kewl

@Mithrandir: why didn't you flag that as spam?

Mithrandir

@Cerbrus Because it doesn't look like spam, it looks like a link-only answer.

Cerbrus

The link doesn't answer the question, though

Mithrandir

1:56 PM

I didn't bother clicking on the link....

Cerbrus

*shrug*

Mithrandir

Let's put it this way: NAA is less likely to be declined :P

Cerbrus

... Depending on the position of the moon.

Glorfindel

@SmokeDetector f

Magisch

@Cerbrus Yer a Lycanthrope, Harry.

Cerbrus

2:06 PM

@Magisch wizard a Harry, yer.

SmokeDetector

[ SmokeDetector | MS ] URL in title, pattern-matching website in body, pattern-matching website in title, repeated URL at end of long post: newsupplements2017.com/somabiotix-reviews/ by Somabiotix Reviews on graphicdesign.stackexchange.com

tpu- by Mithrandir

Magisch

> This is the flag you currently have raised

rip

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, pattern-matching website in body: Some of the pleasant used citrus fruits that Alpha Max No2? by LucyJohanson on graphicdesign.stackexchange.com

Mithrandir

!!/blacklist newsupplements2017.com

SmokeDetector

@Mithrandir The !!/blacklist command has been deprecated. Please use !!/blacklist-website, !!/blacklist-username or !!/blacklist-keyword. Remember to escape dots in URLs using \.

tpu- by Jan Dvorak

Glorfindel

2:08 PM

@SmokeDetector k

Mithrandir

Hey that was my autoflag

!!/blacklist-website newsupplements2017.com

SmokeDetector

@Mithrandir You don't have code privileges, but I've created a pull request for you.

[ SmokeDetector | MS ] Few unique characters in answer: Submit button to submit a form in dreamweaver cs4 by danish zahir on stackoverflow.com (@dorukayhan)

tpu- by Mithrandir

Glorfindel

@Mithrandir please use a \ to escape the .

Mithrandir

!!/blacklist-website newsupplements2017\.com

SmokeDetector

@Mithrandir You don't have code privileges, but I've created a pull request for you.

Mithrandir

2:14 PM

@Glorfindel Like that?

Glorfindel

Yes, that's better. Still, I'm not convinced about BL'ing websites which are already pattern matched. I'd like to hear more opinions about this. It might also hurt performance.

SmokeDetector

[ SmokeDetector | MS ] Blacklisted website in answer: Follow First 5 Pages of Google Results Python Mechanize by Hemant kumar arya on stackoverflow.com (@dorukayhan)

Jan Dvorak

@Glorfindel I think SE runs on a dedicated rPi with about 3% CPU utilization ;-)

Mark Yisri

@JanDvorak are you sure? That sounds a bit small

Glorfindel

EC2

Jan Dvorak

2:20 PM

Why? Evaluating a bunch of regexes against two strings per second doesn't take that much power.

SmokeDetector

Restart: API quota is 5557.

CI on 5152d24 succeeded.

Mark Yisri

@Glorfindel why is that necessary?

Glorfindel

It's regex and a . matches everything

Magisch

@MarkYisri you're entering a regex and . is a functional regex wildcard

Mark Yisri

I see

Can we not just ignore that for websites?

SmokeDetector

2:24 PM

CI on e7b96c3 succeeded.

Thomas Ward

no

Jan Dvorak

can, shouldn't

Thomas Ward

@MarkYisri it's coded to be processed as a regex currently

and since it's handled as regex in the captures against filters, it must remain Regex

at least currently.

If you read the specifics on the wiki for the blacklist commands you'll note it refers to adding regex not just a plain website.

so regex is important.

Unrelated, ignore the burst of evil CI builds I've made run.

!!/rev

SmokeDetector

@ThomasWard d7f56e7 (Thomas Ward: Merge pull request #462 from AWegnerGitHub/master --autopull)

Thomas Ward

!!/pull

SmokeDetector

2:25 PM

[ SmokeDetector ] SmokeDetector started at rev e7b96c3 (Thomas Ward: Merge pull request #463 from Charcoal-SE/auto-blacklist-1484748527.62) (running on Undo's EC2)

Restart: API quota is 5529.

Thomas Ward

@Undo @ArtOfCode github.com/Charcoal-SE/SmokeDetector/pull/461 is on your table for review, looks sane to me, but not gonna OK that one without more spotchecks. And/or tests. :)

@ArtOfCode I also assume your internet was nonfunctional before you went to bed, so therefore your trimming of lines down to 120 in length didn't get included

@angussidney just an FYI, last night we kinda decided to start obeying some more of PEP8, with line length restricts. Not enforced in Flake yet, but just wanted to give you that heads up - we all kinda went hackathon-ish at the code and made the line lengths <= 120 chars long.

Magisch

Also if we didn't make blacklist entries regex we would forego some of the cool things we can do with them

like luma(genex|generic)

Thomas Ward

^

and this kind of capturing is exceptionally useful for things like keyword patterns

(back in a bit, work calls)

and we're back :P

SmokeDetector

2:56 PM

[ SmokeDetector | MS ] Phone number detected in title: xlwings range.api.validation.add throws com_error -2146827284 by sanzoghenzo on stackoverflow.com (@dorukayhan)

Glorfindel

@SmokeDetector f

Thomas Ward

sd fp-

SmokeDetector

3:23 PM

[ SmokeDetector | MS ] Manually reported answer: Background color for Tk in Python by Joe on stackoverflow.com (@dorukayhan)

Floern

thank you, edited in grace period

SmokeDetector

naa- by DavidPostill

rene

@Floern what was it? spam or abussive?

Floern

@rene abusive, see metasmoke report

rene

yeah. I just noticed

SmokeDetector

3:32 PM

CI on 8a2e1a9 succeeded.

Thomas Ward

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 8a2e1a9 (Thomas Ward: FindSpam: Enforce max code line length of 120 chars) (running on Undo's EC2)

Restart: API quota is 4985.

Thomas Ward

@ArtOfCode I got bored, so I did the findspam.py line length refactor. Had to move some stuff around a little to make things look nice, but otherwise it's the same file.

(And now implemented)

Undo

@tripleee Just curious, why the indentation change on github.com/Charcoal-SE/SmokeDetector/pull/461/…?

Thomas Ward

@Undo Flake8

visual indent requirements

part of the style guides

Undo

3:42 PM

Was it failing before?

Thomas Ward

yes

check github.com/Charcoal-SE/SmokeDetector/pull/461/commits

the red x on the commit shows the failure cases

travis-ci.org/Charcoal-SE/SmokeDetector/builds/193036654#L214 specifically in the CI data

Undo

I know that, but why wasn't it failing earlier?

Thomas Ward

it was. he has a second commit in the branch.

Undo

oh, he just went the other way

github.com/Charcoal-SE/SmokeDetector/commit/… succeeds fine

... no

bah, I'll figure it out when I'm more awake. After processing nightly pings.

Thomas Ward

@Undo it's not failing because he has the fix commit. Which is how the PR worked. Trust me, the commit that has the flake8 fail had PEP8 violations which i caught

from a code sanity perspective though is what the review request was for

not for the fact he had to fix the flake8 indentation :P

Undo

3:52 PM

Yeah, I know. Just trying to understand what happened there

@Magisch Github issues please.

SmokeDetector

[ SmokeDetector | MS ] Offensive answer detected: Why do no user-agents implement the CSS cursor style for video elements by pissed off on stackoverflow.com (@dorukayhan)

Undo

!!/test luckypatcher.ucraft.com

SmokeDetector

> Url in title
----------
Title - Position 1-32: https://luckypatcher.ucraft.com

[ SmokeDetector | MS ] Link at end of answer: Calories burned by mac456 on fitness.stackexchange.com

Mithrandir

@SmokeDetector spam

Thomas Ward

@Undo Yeah that confuses me too :P

@Undo good news though is everything seems to be working even with my findspam line length refactor in place

and our refactoring last night shed light on other issues, which are all nice and solved it seems now.

maybe

Undo

4:06 PM

Yep, seems to be stable enough

Thomas Ward

@Undo now that we're line length compliant do we need to push a change to tox.ini to enforce in Flake8?

Undo

You can, yeah

Thomas Ward

i forget, what do we need ot change again?

wait you had one in the history somewhere... checks

18 hours ago, by Undo

  1 [flake8]
  2 ignore = F403,E402,F405,N804
  3 exclude = ChatExchange/*,.idea/*,.git/*
  4 max-line-length = 120

SmokeDetector

CI on e3c03fb succeeded.

Thomas Ward

@Undo is that still valid to put into place or no?

Undo

4:11 PM

I'm running it locally, I'll just push it @ThomasWard

SmokeDetector

CI on 89222c3 succeeded.

Thomas Ward

OK

SmokeDetector

CI on e9deb84 failed.

Undo

heh, apparently we're not compliant

@ThomasWard Any reason not to just exclude tests from that check?

Thomas Ward

@Undo for the visual indentation?

Undo

4:16 PM

For line length

Thomas Ward

what do you mean by exclude tests?

you mean those long regex reasons?

Undo

circleci.com/gh/Charcoal-SE/SmokeDetector/4046

Thomas Ward

yeah ignore them for test/*

Undo

Now the question is how

Thomas Ward

I'm not picky about the tests.

@Undo same way other Flake ignores are done

just add it to the exclude list in tox.ini for the tests

SmokeDetector

4:19 PM

CI on d8dfd0c succeeded.

Undo

Then you're excluding that directory from all checks, not just E501

Thomas Ward

that'll stop flake8 tests, but any syntax errors, etc. will still throw when the tests run

@Undo I'm not sure we can exclude from just E501

Undo

Yeah, doesn't look like we can

Thomas Ward

it's all or nothing

unfortunately.

unless we modify the CI tool chain setup to run a separate Flake8 command with a different config on tests

Ferrybig

We can run 2 instances of flake, 1 with E501 for the main code, and 1 without E501 fr the tests

Thomas Ward

4:21 PM

yeah that

which wouldn't necessarily be a bad thing

Undo

also, we still have some failures in ./chatcommunicate.py and ./deletionwatcher.py

@Ferrybig Yeah, probably more work than it's worth

Thomas Ward

@Undo I had fixed those, rebase your branch?

or, I can stab those

Undo

I'm on master

SmokeDetector

[ SmokeDetector | MS ] URL in title, bad keyword in body, blacklisted website in body, blacklisted website in title, repeated URL at end of long post: cluesforhealth.com/Derma-Reflexion/ by user84383 on graphicdesign.stackexchange.com

Undo

Clean tree

SmokeDetector

4:23 PM

tpu- by Undo

Thomas Ward

I'll poke a fix in then

SmokeDetector

CI on b8f97f2 succeeded.

Undo

Yep, that works

Thomas Ward

two commits sent in for both of those, @Undo

Undo

Yep. Enabling Flake again...

SmokeDetector

4:26 PM

CI on 8c0f504 succeeded.

Thomas Ward

god I like pycharm, it first gets a list of origin changes, and if it differs from local, it stashes my changes, rebases the branch, and then pushes

makes it wonderful for fixing things. Except when conflicts happen.

then i just start over :P

Undo

I really should start using Pycharm

Thomas Ward

not cheap though

Glorfindel

So basically it works like a charm?

Thomas Ward

PyCharm Pro has the VCS support, community doesn't

Glorfindel

4:27 PM

Or just in theory?

Thomas Ward

@Glorfindel Pretty much

at least in theory

SmokeDetector

CI on a2d8dcd succeeded.

Undo

@ThomasWard open source license, but yeah

!!/pull

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev a2d8dcd (Undo1: Enable Flake's 120-char check) (running on Undo's EC2)

Restart: API quota is 4544.

Thomas Ward

@Undo I assume we don't want to run a second separate Flake test then for test/* ?

we could always add that to the CI

Undo

4:29 PM

If you want to take that on, it'd be fine.

Not a priority, though. It's not like our test code has to be pristine.

Thomas Ward

indeed.

I'm going to add it as an enhancement idea on GitHub, and self-assign

then poke

and i'm out of power, back later.

SmokeDetector

4:56 PM

[ SmokeDetector | MS ] Link at end of answer: What IoT services are available for storing/sending/publishing generic data in the cloud? by Matt on iot.stackexchange.com

NobodyNada

meh, question asks for it

sd f

ArtOfCode

@ThomasWard you don't actually need the VCS support, though, if you're used to using git through terminal commands. PyCharm will let you open a docked terminal at the bottom of the window, so you can just use git in there.

Transcript for