Charcoal HQ - 2018-03-13 (page 2 of 7)

« first day (1577 days earlier) ← previous day next day → last day (2243 days later) »

4:00 AM

!!/listening

@quartata {'1-question-49215463': ('49215463', 'stackoverflow.com', 'question', '//stackoverflow.com/q/49215463', []), '1-question-None': ('49247558', 'stackoverflow.com', 'answer', '//stackoverflow.com/a/49247558', [(<bound method Message.delete of <chatexchange.messages.Message object at 0x7f99780169b0>>, 1520911856.8232858)])}

It's a form of communication on the interwebz

2 emails inc

@FrankerZ What does it say on the back of the letter that Art sent you?

SOBotics x4

That's why. @ArtOfCode messed up :D

I'll mail you the extra four I have for Charcoal.

4:04 AM

[ SmokeDetector | MS ] Repeated URL at end of long post: How would I install Budgie in 16.04 without using a PPA? by David Alvarado on askubuntu.com

sd f

!!/listening

!!/alive

@quartata Watching this endless list of new questions never gets boring

CI on 01dbc6b succeeded.

!!/pull

[ SmokeDetector ] SmokeDetector started at rev 01dbc6b (quartata: cull posts older than 2 hours) (running on teward/Solar Flare)

!!/listening

just please

4:06 AM

@quartata {}

Restart: API quota is 7776.

ok good

!!/report meta.stackexchange.com/a/305588

[ SmokeDetector | MS ] Manually reported answer: Formatting Sandbox by quartata on meta.SE

!!/listening

@quartata {}

???

Oh.

sd -f

sd - f

4:10 AM

Conflicting feedback across revisions: current, #1, #2

!!/report codegolf.stackexchange.com/questions/73104

[ SmokeDetector | MS ] Manually reported question: Grocery Store Micromanagement by quartata on codegolf.SE (@Mego @quartata)

!!/listening

@quartata {'200-question-73104': ('73104', 'codegolf.stackexchange.com', 'question', '//codegolf.stackexchange.com/questions/73104', [])}

!!/reboot

4:11 AM

Goodbye, cruel world

[ SmokeDetector ] SmokeDetector started at rev 01dbc6b (quartata: cull posts older than 2 hours) (running on teward/Solar Flare)

!!/listening

@quartata {}

Restart: API quota is 7747.

[ SmokeDetector | MS ] Bad keyword in body: he to can recommend that supplement and by MatteHine on drupal.SE

tpu- by Undo

[ SmokeDetector | MS ] URL-only title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +4 more: www.healthsupplementsreviews.info/vidhigra/ by Patsykrawczyk on askubuntu.com

tpu- by John Dvorak

[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title: fornatgaex.com/primacin-xl/ by yuor on puzzling.SE (@Mithrandir @micsthepick)

tpu- by PeterJ

@SmokeDetector wtf is this?

!!/listening

@quartata {'89-question-1014398': ('1014398', 'askubuntu.com', 'question', '//askubuntu.com/questions/1014398', [(<bound method Message.delete of <chatexchange.messages.Message object at 0x7fd27c2d3ef0>>, 1520914818.981948)]), '559-question-61819': ('61819', 'puzzling.stackexchange.com', 'question', '//puzzling.stackexchange.com/questions/61819', [])}

4:23 AM

@iBug Standard response to death

CI on d28cc01 succeeded.

!!/pull

[ SmokeDetector ] SmokeDetector started at rev d28cc01 (quartata: check for FP) (running on teward/Solar Flare)

!!/listening

Restart: API quota is 7671.

@quartata {'559-question-61819': ('61819', 'puzzling.stackexchange.com', 'question', '//puzzling.stackexchange.com/q/61819', [])}

4:24 AM

good

uh

Wait

Uh oh

I forgot it shortens it

I don’t think Metasmoke will take that

well we’ll find out

!!/listening

@quartata {}

Yep

@Undo can you check the MS logs to verify

Aye

(Looking for posts to /deletion_logs.json)

I think that’s what it’s called

@quartata Most recent one was for Workplace

4:28 AM

so it didn’t request at all?

Morning all!

@quartata That was the last one, 2018-03-12T18:31:08.429308

//workplace.stackexchange.com/questions/108085

Oh sorry I didn’t mean like that

I'm looking at logs, not objects

Hrmm

Search for //puzzling.stackexchange.com/questions/61819?

Er sorry

//puzzling.stackexchange.com/q/61819

4:32 AM

No hits

The /questions variation does, though

Assuming it logs the params

Oh?

It does, no hits

Can you show me that

@quartata Just the standard logs for a post. Feedback, API, etc.

No deletion_logs.json at all

4:34 AM

Not that I'm seeing

Well it is definitely keying on /questions and not /q

so let’s do that

I'd be seeing the request if it came through at all, even a 404

CI on d699cc1 succeeded.

!!/pull

[ SmokeDetector ] SmokeDetector started at rev d699cc1 (quartata: appease Metasmoke) (running on teward/Solar Flare)

4:36 AM

!!/listening

@quartata {}

well I need to sleep

Restart: API quota is 7611.

hopefully someone can keep an eye on it

fp by quartata on Formatting Sandbox [MS]

Conflicting feedback across revisions: current, #1, #2

Conflicting feedback across revisions: current, #1, #2

4:38 AM

@Undo can you destroy all my test posts

Conflicting feedback across revisions: current, #1, #2

fp by quartata on Formatting Sandbox [MS]

Aye

Conflicting feedback across revisions: current, #1

just search by username

I did some on different sites

Conflicting feedback across revisions: current, #1

fp by quartata on How would I install Budgie in 16.04 without using a PPA? [MS]

4:39 AM

I still don't get what use is Smokey posting these conflicts multiple times a post

!!/rmblu codegolf.stackexchange.com/users/45151

Metasmoke post 109891 (Grocery Store Micromanagement) destroyed by Undo

Metasmoke post 109890 (Formatting Sandbox) destroyed by Undo

@JohnDvorak It's a bug

hello ratelimiting

!!/rmblu meta.stackexchange.com/users/310369

Metasmoke post 109880 (Formatting Sandbox) destroyed by Undo

4:40 AM

do you think the !!/report URL is still wrong and needs a fix?

Oh, thanks. I once was told it's deliberate. I assume that was in jest then.

Metasmoke post 109878 (Formatting Sandbox) destroyed by Undo

I'm looking for something to do and if you think so, that'll be my next small work.

Metasmoke post 106806 (The Programming Language Quiz, Mark II - Cops) destroyed by Undo

@iBug Wrong how?

4:41 AM

@quartata User removed from blacklist (45151 on codegolf.stackexchange.com).

jeez

@Undo wait a moment

I think that’s the worst I’ve ever seen it

It's exponential now

Metasmoke post 106805 (The Programming Language Quiz, Mark II - Cops) destroyed by Undo

4:42 AM

will catch up eventually

Metasmoke post 106804 (The Programming Language Quiz, Mark II - Cops) destroyed by Undo

in SOBotics on Stack Overflow Chat, 19 hours ago, by Bhargav Rao

@SmokeDetector Ey, I reported an answer https://stackoverflow.com/questions/26905589/phpshow-one-row-only-in-same-id/49‌232017#49232017

@quartata User is not blacklisted.

[ SmokeDetector | MS ] Bad NS for domain in body, bad keyword in body, bad keyword in title, blacklisted website in body: Vidhigra Wellness Benefits of Blueberry Pineapple by Oliviayru on stackoverflow.com

That'd be a good one to take a look at, yes

I know what the problem is

4:43 AM

[ SmokeDetector | MS ] Bad keyword in body, blacklisted website in body, pattern-matching website in body: The Trustworthy And Genuine Solution by Bkiroewx on puzzling.SE (@Mithrandir @micsthepick)

A quick hexdump shows there are 6 bytes of invisible texts (UTF-8)

tpu- by PeterJ on Vidhigra Wellness Benefits of Blueberry Pineapple [MS]

[ SmokeDetector | MS ] URL-only title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +3 more: growxlsite.com/mega-boost-perform-xl/ by ranalotqik on askubuntu.com

Yep, and those are turned into HTML entities which the old regex can’t handle IIRC

@quartata Read my idea from this message:

2 hours ago, by iBug

@quartata The URL bug just occurred again yesterday.

@SmokeDetector no autoflag?

4:45 AM

I’ll fix it tomorrow

tpu- by PeterJ on The Trustworthy And Genuine Solution [MS]

@AJ It flagged

@AJ norepro

tpu- by A J

yeah page refreshed now.

4:45 AM

@quartata (little iBug wants something to do)

Hey @ArtOfCode Good Luck!!

@AJ There's a small delay before MS sends flags.

Restart: API quota is 7544.

ok so the HTML entities should be gone actually

Hm

A hexdump in !!/report would help probably

@quartata if they're gone, how would that be a problem actually

4:49 AM

> Stop being a mod on every site! Jeez, Art, you're making the rest of us look bad. – Nic Hartley 7 hours ago

LOL

@paper1111 I mean that they should be unescaped by that point

~ $ echo 'https://stackoverflow.com/questions/26905589/phpshow-one-row-only-in-same-id/49‌232017#49232017' | hexdump -vC
00000000  68 74 74 70 73 3a 2f 2f  73 74 61 63 6b 6f 76 65  |stackove|                              00000010  72 66 6c 6f 77 2e 63 6f  6d 2f 71 75 65 73 74 69  |rflow.com/questi|                              00000020  6f 6e 73 2f 32 36 39 30  35 35 38 39 2f 70 68 70  |ons/26905589/php|
00000030  73 68 6f 77 2d 6f 6e 65  2d 72 6f 77 2d 6f 6e 6c  |show-one-row-onl|                              00000040  79 2d 69 6e 2d 73 61 6d  65 2d 69 64 2f 34 39 e2  |y-in-same-i…

API quota rolled over with 7525 requests remaining. Current quota: 19999.

@quartata Copied from Bhargav's message ^^^

math: 5
stackoverflow: 5
boardgames: 2
serverfault: 2
askubuntu: 1
unix: 1
ux: 1
bitcoin: 1
english: 1
electronics: 1
raspberrypi: 1

4:54 AM

URL_CHAR = r"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789.?:-/=_#%"

def rebuild_url(s):
    return ''.join([c for c in s if c in URL_CHAR])

very good function. total use friend

it filters arbitrarily any garbage in URL

I'll do a PR soon.

applying this function to report and scan commands

yay

[ SmokeDetector | MS ] Shortened URL in answer: Information Security Risk Analytics by Utsav Vyas on security.SE

tpu- by A J

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, link following arrow in body, pattern-matching website in body, potentially bad keyword in body, +1 more: Nuallura There are unlimited alternatives with regards to wrinkle creams by Marvilcott on graphicdesign.SE

tpu- by A J

@iBug Through which method/route/API are you obtaining the message data? Basically, I'm expecting I'm going to have to deal with the issue in parsing SE URL also, at least in some of the scripts I maintain. I'm just trying to get a feel for what's going to be necessary on my end. I've known about the issue of chat inserting these characters. I should have thought about it affecting the URLs.

@Makyen SE has long been inserting invisible characters in URLs

[ SmokeDetector | MS ] Bad NS for domain in body: Woocommerce search form issue by Asif Mughal on stackoverflow.com

@SmokeDetector not sure, but CVed

@Makyen message data? I copy it by hand.

5:07 AM

@SmokeDetector That looks like spam by looking at the profile.

Copying by hand just gives you the verbatim message, so you can inpect the characters.

[ SmokeDetector | MS ] Shortened URL in answer, blacklisted user: SIEM and Security Analytics by Utsav Vyas on security.SE

tpu- by tripleee

!!/watch lts\W?secure

@tripleee Added lts\W?secure to watchlist

@iBug Yes, I know that it sometimes inserts those characters into chat messages. I've investigated it a bit previously, due to other issues.

5:09 AM

@Makyen Try my python code given above. It's simple and easy

!!/blacklist-keyword nuallura

@tripleee Blacklisted nuallura

@tripleee duplicate?

!!/test nuallura

> Potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username
----------
Title - Position 1-9: nuallura
Body - Position 1-9: nuallura
Username - Position 1-9: nuallura

oh, I only watched it

5:10 AM

!!/blacklist-website supplements2017\.com

@tripleee That pattern looks like it's already caught by Pattern-matching website in body and Pattern-matching website in answer; append -force if you really want to do that.

in Charcoal Test, 3 mins ago, by PulseMonitor

halflife:2018-03-13 05:06:14,890:109899: regex nuallura search: 4/4 hits

Restart: API quota is 19899.

!!/blacklist-website-force supplements2017\.com

@tripleee Already blacklisted on line 804 of blacklisted_websites.txt

5:10 AM

huh? okay

!!/test-a spam.supplements2017.com/spam-spam-spam

> Pattern-matching website in answer
----------
Body - Position 13-31: supplements2017.co

LOL.

hmmm, fine

[ SmokeDetector | MS ] Shortened URL in answer, blacklisted user: How to demo a new 0-day security product? by Utsav Vyas on security.SE

sd k

5:11 AM

tpu- by tripleee

@iBug OK. Do you mean that for the data you presented, you copied it out of the message which you were viewing in the chat room or transcript as displayed by your browser?

brb

!!/blacklist-website goo\.gl/7J79so

@tripleee Blacklisted goo\.gl/7J79so

@Makyen Both seems the same.

you get identical stuff when you copy it from room or from transcript.

[ SmokeDetector | MS ] No whitespace in answer: tensorflowで作ったネットワークに値を強引に代入するには by user27723 on ja.stackoverflow.com

[ SmokeDetector ] SmokeDetector started at rev 984023e (SmokeDetector: Auto blacklist of goo\.gl/7J79so by tripleee --autopull) (running on teward/Solar Flare)

Restart: API quota is 19870.

5:14 AM

@tripleee chat.stackoverflow.com/transcript/111347?m=41585835#41585835

@iBug yeh, just tracing back myself, but it should be in the PR for context

In response to your comment on the GH PR.

OK, I'm adding it in.

@SmokeDetector needs flags

@tripleee also, do you think I'm missing any characters?

@iBug is this for Stack Overflow URLs specifically, or URLs in general? in the latter case at least &

and ~

5:21 AM

@tripleee OK, I'll add the ampersand. It's meant for SE url only

tilde?

and ,

oh, i remember tilde

but if you want to normalize Stack Overflow question URLs, why don't you just grab the id after /question/ and scrap everything after that? are there instances where this doesn't work?

channels?

I think I've seen URLs with ; in parameters too

oh and + obviously!

5:22 AM

[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title, +2 more: nutrahealthtrimsite.com/golden-farms-forskolin/ by Pitenodex on english.SE

tpu- by John Dvorak

!!/blame

It's meant for SE only

@micsthepick It's Murch's fault.

soooo ... at this point maybe look at an authoritative resource rather than hope to rely on my memory

@JohnDvorak Channels isn't in the public API

5:23 AM

and AFAIK plus sign is filtered out by SE

[ SmokeDetector | MS ] Potentially bad keyword in body: Where can I find best German language classes in Pune? by user384752 on meta.SE

Then there are question titles such as "could not understand stackoverflow.com/questions/13371337/what-is-an-array-eli5";

tpu- by tripleee

don't worry, I whitelisted the percentage sign

!!/blame᠎ ‌⁣‍᠎‍᠎᠎᠎‍᠎᠎

5:24 AM

@micsthepick I hit an error while trying to run that command; run !!/errorlogs for details.

I added tilde, comma, ampersand and semicolon

!!/watch (?:7\W*)?914\W*611\W*04\W*44

@tripleee Added (?:7\W*)?914\W*611\W*04\W*44 to watchlist

it’s not fixed

!!/errorlogs

File "/home/smokey/SmokeDetector/deletionwatcher.py", line 68, in subscribe
post_id, post_site, post_type = fetch_post_id_and_site_from_url(post_url)

TypeError: 'NoneType' object is not iterable
2018-03-13 03:11:47.315062 UTC
File "ws.py", line 76, in <module>
GlobalVars.deletion_watcher = DeletionWatcher()

File "/home/smokey/SmokeDetector/deletionwatcher.py", line 33, in __init__
self.subscribe(post, pickle=False)

File "/home/smokey/SmokeDetector/deletionwatcher.py", line 68, in subscribe
post_id, post_site, post_type = fetch_post_id_and_site_from_url(post_url)

5:26 AM

@iBug Why can't you use string.printable?

@iBug I was meaning that you weren't getting the data from how SD/metasmoke gets the message data. You were effectively scraping the page, which may be different information than is delivered programmatically to SD. I have not investigated where in SE's dataflow those get inserted. What I was actually asking was what methodology does SD use to get chat messages. I'd guess that it listens to a WebSocket, but I don't know.

I assume that the post where the report was made was this one. The history for that post implies that the characters you're seeing are not in the actual data that SE keeps.

should be 100% enough for an SE link

!!/blacklist-website 3peartechnologies\.com

@tripleee Blacklisted 3peartechnologies\.com

@Makyen Smokey uses ChatExchange, which is more or less the same

5:26 AM

CI on 187dfee succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev 1aa0862 (SmokeDetector: Auto blacklist of 3peartechnologies\.com by tripleee --autopull) (running on teward/Solar Flare)

@Makyen but if they end up being copy/pasted into !!/report that's still something we need to cope with, isn't it? I'm just not sure ditching nonprintables is the correct way to cope

Restart: API quota is 19785.

@Andy lemme learn that first

docs.python.org/3/library/string.html#string.printable

@tripleee Yes, we should deal with it, as it's possible. They will only be there if the copy is made from a chat message, not URL-text on a main site. I'm not sure it's the right way either.

5:30 AM

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, bad keyword in username, blacklisted website in body, link at end of body, +2 more: What is Vivrax & How does it work? by Vivrax on superuser.com

well, good suggestion but I don't think it's needed

tpu- by tripleee

@iBug Is it really the same as a human viewing what's displayed in their browser? That seems to be quite a bit of extra cruft to have designed in, rather than just listening to the data.

!!/watch golden\W?farms

@tripleee Added golden\W?farms to watchlist

5:31 AM

@Makyen I suppose so.

@iBug Why? You are guessing what to include right now. I'm offering you a suggestion that has all the printable characters already. No need to guess.

[ SmokeDetector ] SmokeDetector started at rev a94556e (SmokeDetector: Auto watch of golden\W?farms by tripleee --autopull) (running on teward/Solar Flare)

Restart: API quota is 19752.

!!/watch rose\W?diamond

@tripleee Added rose\W?diamond to watchlist

in before Titanic plot

5:34 AM

@Andy I wanna exempt Tab and Newline etc. from that list

CI on 098bbad succeeded. Message contains 'autopull', pulling...

[ SmokeDetector ] SmokeDetector started at rev 098bbad (SmokeDetector: Auto watch of rose\W?diamond by tripleee --autopull) (running on teward/Solar Flare)

Restart: API quota is 19741.

@Andy I can copy the string from that and manually remove them

But I still doubt it's necessary.

I did look through my keyboard and determined whether each character is useful.

perhaps a better progression of work would be to figure out why reporting answers ends up reporting the question they're on, then deal with any formatting

@tripleee the hexdump is pretty much the answer.

[ SmokeDetector | MS ] Bad pattern in URL body, blacklisted website in body, pattern-matching website in body: Our subconscious mind is Male Extra by pamelarivera on graphicdesign.SE

tpu- by NobodyNada

5:40 AM

https://stackoverflow.com/questions/26905589/phpshow-one-row-only-in-same-id/49‌......232017#49232017

three invisible chars that broke Smokey

@iBug Where did you get that? I don't see that in the transcripts

@iBug Not necessarily. What you've demonstrated is that those characters exist in the page you are viewing. That doesn't mean it's what SD sees when it parses the message.

I'm also pretty sure we've circled this issue before

@Makyen wait for me to make some tests.

@Andy that's why they are called invisible

5:41 AM

chat.stackexchange.com/message/<id> will probably get you exactly what Smokey saw

20 hours ago, by K.Dᴀᴠɪs

in SOBotics on Stack Overflow Chat, 49 secs ago, by Bhargav Rao

@SmokeDetector Ey, I reported an answer https://stackoverflow.com/questions/26905589/phpshow-one-row-only-in-same-id/49‌232017#49232017

I can confirm that copy/pasting that gives me invisible characters after /49

@tripleee sigh I mean, I looked at the report that was sent and the transcript doesn't show the invisible characters when I check the hex dump

@Andy chat.stackoverflow.com/message/41585827

wait when I test it somewhere else.

@NobodyNada Convincing!

$ perl -e 'printf "\xe2\x80\x8c\xe2\x80\x8b\n"' | iconv -f utf-8 -t utf-16be | xxd
00000000: 200c 200b 000a

so the invisible characters are U+200C U+200B U+000A

5:44 AM

um

fileformat.info/info/unicode/char/200c/index.htm zero-width joiner

etc etc

paper1111 made an attempt earlier

pretty sure this is an artefact of copy/pasting from a place where this was somehow wrapped or etc

should be a PR already merged some time ago.

@WELZ looks great

5:45 AM

@tripleee IIRC chat does that automatically to make the words wrap properly, but I can't tell you for sure

Restart: API quota is 19656.

IMO we should use chat.stackexchange.com/message/<id>?plain=true, which gives us the Markdown of the message (example: chat.stackoverflow.com/message/41585827?plain=true)

so anyway perhaps a better fix would be to chuck that precise sequence and not do any more involved heuristics?

@NobodyNada but if you have a user copy/pasting something from somewhere you have no control over where they get that from or how they paste it

@iBug But the issue isn't that there are invisible characters. It's that there are & encoded characters. Are you dealing with the message that has had those decoded, or are they still &zwnj;?

[ SmokeDetector | MS ] URL in title, bad keyword in body, bad keyword in title, blacklisted website in body, blacklisted website in title: READ==> fornatgaex.com/primacin-xl/ by GlorYate on puzzling.SE (@Mithrandir @micsthepick)

tpu- by John Dvorak

5:48 AM

@Makyen They are not encoded.

@tripleee But how often does that happen? I think the only way that could possibly happen is if someone copied and pasted an URL from a Stack Exchange comment, by highlighting and copying the link text instead of the link target

@NobodyNada apparently it was a recurring problem for Bhargav at least

If you copy and paste a URL from the browser's address bar, which AFAIK is the only way we've ever run into this issue, the link should be clean

@tripleee That comes from chat, which throws garbage in the chat messages to make them wrap

The plaintext version, which contains exactly what Bhargav typed into the textbox, does not have funny characters

GlorYate, that name seems familliar... :P

so if I copy-paste that message into a chat message it will acquire these control characters in the message that a bot receives if it's listening to chat messages?

5:52 AM

wait guys

@tripleee It depends on how it's listening.

I'm adding a show_url command locally

Restart: API quota is 19604.

my local build will be available in room 65945

you can issue commands there.

I'll tell you when it's ready

My local instance will be run under my account.

I do not have Smokey credentials

can you capture a misspelled message like !!/reprot instead of !!/report ?

so I don't trip the real Smokey in that room

5:55 AM

!!/iReport

@JohnDvorak No such command 'ireport'.

Yes that shouldn't be hard to implement

wait i'm still cloning the repo

Apparently we get these zero width messages frequently: chat.stackexchange.com/search?q=zwnj&user=&room=11540 (ignore some of the more recent ones because of this conversation), there are several !!/ commands with these characters added

@Andy Yes, chat adds them in a misguided, ineffective attempt to cause things to wrap, rather than just relying on appropriate CSS.

naa by JAD on tensorflowで作ったネットワークに値を強引に代入するには [MS]

5:59 AM

Nice going, chat

« first day (1577 days earlier) ← previous day next day → last day (2243 days later) »

Transcript for

Mar '1813

Charcoal HQ

Where smoke is detected, diamonds are made, and we break thing...

dev-on-prod flag-based-fraternity programmatic-crowbars

join this room
about this room

00:00

06:00

12:00

18:00

all times are UTC

site design / logo © 2024 Stack Exchange Inc; legal

mobile