Charcoal HQ

12:16 AM

[ SmokeDetector | MS ] Potentially problematic ns configuration in body (1): woocomerce wordpress product quality restriction by Fatih Ylmz on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Windows 10 wifi not turning on Acer Aspire laptop by Burza on superuser.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): How to log a custom Event in Facebook Analytics SDK 2020 for swift Documentation is deprecated ✏️ by Daira Bezzato on stackoverflow.com

[ SmokeDetector | MS ] Bad keyword in title, potentially bad keyword in body, potentially bad keyword in title, toxic body detected (162): Is touching your penis allowed? (No rubbing or masturbation) (please answer fast) by anonymousmuslim on islam.SE

[ SmokeDetector | MS ] Potentially bad ip for hostname in answer (1): Why can I not connect to my Bukkit server? by Jake on gaming.SE

fp- feedback received on Is touching your penis allowed? (No rubbing or masturbation) (please answer fast) [MS]

fp feedback on autoflagged post: Is touching your penis allowed? (No rubbing or masturbation) (please answer fast) [MS]

Autoflagged FP: flagged by @SmokeDetector, @Makyen, @Shree

[ SmokeDetector ] SmokeDetector started in standby mode at rev b2f025e (SmokeDetector: Auto watch of elmosoft\.net by Daniil) (running on Makyen/Windows)

fp- feedback received on How to get only 1 input work instead of all input in for loop? [MS]

@double-beep Your flag conditions were manually disabled by @SmokeDetector: Total accuracy must be >= 99.75%.

[ SmokeDetector | MS ] Link at end of answer (60): Construir, utilizando HTML, CSS e JavaScript puro, uma aplicação para a visualização de cores a partir da escala RGB ✏️ by frontend on pt.stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Procedure of web scraping as a job? by KODA EA on meta.SE

user12986714

1:11 AM

(@Makyen) Is there a specific reason we want from globalvars import GlobalVars rather than import globalvars; GlobalVars = globalvars.GlobalVars;? I am having terrible circular import issues.

SmokeDetector

1:29 AM

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): What programming languages would be best suitable to create an e-commerce site from scratch? Or is it just better to use a cms like Wordpress? by ALPHA 7 on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad ns for domain in answer (1): ln: failed to create symbolic link: Protocol error (in Vagrant) by Purnesh Dixit on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad ns for domain in answer (1): creating symbolic link: Protocol error by Purnesh Dixit on stackoverflow.com

fp- feedback received on What programming languages would be best suitable to create an e-commerce site from scratch? Or is it just better to use a cms like Wordpress? [MS]

fp- feedback received on ln: failed to create symbolic link: Protocol error (in Vagrant) [MS]

fp- feedback received

API quota rolled over with 5022 requests remaining. Current quota: 19999.

math: 337
stackoverflow: 219
askubuntu: 73
superuser: 59
physics: 51
electronics: 39
english: 39
ell: 39
unix: 35
gaming: 30
serverfault: 27
apple: 27
mathematica: 26
mathoverflow.net: 24
rpg: 20
diy: 19
chess: 19
worldbuilding: 17
dba: 16
space: 16
codegolf: 16
ja.stackoverflow: 16
es.stackoverflow: 14
aviation: 14
chemistry: 12
salesforce: 11
security: 10
christianity: 10
academia: 10
gardening: 10
gis: 9
workplace: 9
cs: 9
writing: 8
judaism: 8
hermeneutics: 8
datascience: 8
wordpress: 8
sharepoint: 8

Makyen

1:56 AM

@user12986714 I am not aware of a specific reason for doing so. OTOH, I'm not the one who chose to use one semantic over the other. Does making the change really resolve the issues you are seeing? Why are you seeing the issues now, when we weren't previously?

SmokeDetector

[ SmokeDetector | MS ] Repeating characters in answer (81): UIImageView instances of NSPathStore2 leaked by Tezza on stackoverflow.com

user12986714

@Makyen Won't resolve problems... :-( Mainly because I am trying to integrate the auto ms up/down switch into globalvars, which should send a chat msg, so now globalvars need to import chatcommunicate.

SmokeDetector

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): how to speed up org-babel-tangle? by Jerry Leibold on emacs.SE

user12986714

Because otherwise we have to implement the switch everywhere we adjusts failure counter.

Makyen

@user12986714 It sounds like MS stuff should stay in the metasmoke.py module.

user12986714

2:03 AM

@Makyen yeah. I am now trying to move those global variables into metasmoke

Makyen

@user12986714 Why would the failure counter be adjusted from anywhere other than the metasmoke.py module? Assuming, that is, that we're keeping all of the interaction with MS in that module.

@user12986714 If they are global variables, it sounds like they should be in global variables, where you have the getter and setter structure to enforce locking.

Unless we're going to have/need that structure elsewhere... which we might, or may need some reorganization, or at least a good think about threading.

user12986714

@Makyen My thoughts are that we don't need to keep all global variables in globalvars.py. There is nothing special about that file.

Socket science will switch ms up/down

so need to expose the up/down methods by some ways

also chatcommands

Makyen

@user12986714 Socket science is currently not in use. OTOH, that doesn't mean we don't want to be able to use it.

user12986714

Also the problem is that from recent issues we experienced, it appears that we forgot to check failure counter at some places adjusting it. Otherwise it should switch ms off when there is 28 failures

Makyen

@user12986714 Or just a metasmoke.set_metasmoke_up() and metasmoke.set_metasmoke_down(), which then handle any secondary effects.

user12986714

2:12 AM

@Makyen yeah, but are we sure that no other file is trying to directly communicate with MS? If then we can make the failure counter private to metasmoke.py

Makyen

@user12986714 There should be a choice made as to if the automatic MS up/down should be done at every failure, or only from within the ping code.

user12986714

@Makyen I suppose it should be for all tries

Makyen

@user12986714 Well, given that we have a metasmoke.py, everything to do with MS is supposed to be going through that. If it's not, then there should be a really good reason. Having everything have to go through metasmoke.py is one of the things that enables having the error counter.

user12986714

Also we get to expose failure counter to chatcommands.py

used by !!ms-status

Makyen

@user12986714 I'd definitely accumulate errors from all tries, but I can certainly see a case that only the pings should actually make MS up/down.

user12986714

2:19 AM

@Makyen 1) Implement another counter. 2) Accumulated count vs consecutive count?

Makyen

@user12986714 Which could be implemented with get_ms_error_count(), but if it's a variable that's global to the (any) module, then it should have the lock semantics.

user12986714

@Makyen I get some idea. Let the counter that will switch ms on/off be private to metasmoke.py, and implement the other counter in the MSStatus class

SmokeDetector

[ SmokeDetector | MS ] Blacklisted website in answer, link at end of answer, potentially bad ip for hostname in answer, potentially bad ns for domain in answer (155): Install Tidal App by kellyken on askubuntu.com

Makyen

@user12986714 I wasn't meaning another counter; still one counter. 2) I can see arguments for doing either, but net errors - success goes above threshold is usually good for declaring "down" with some lower magnitude in the opposite direction needed for declaring up (which should end up only coming from pings, because that's the only thing that is forced to be tried when MS is declared down).

SmokeDetector

tpu- feedback received

Makyen

2:26 AM

@SmokeDetector It's relatively clear your spamming when you recommend explicitly Windows-only software on an Ubuntu site.

user12986714

@Makyen So now my public interface for globalvar.msstatus looks like this

###Public interface
- `is_up()`: Query if metasmoke is up. Returns `True` if metasmoke is up and `False` otherwise.
- `is_down()`: Query if metasmoke is down. Returns `True` if metasmoke is down and `False` otherwise.
- `failed()`: Indicate a metasmoke connection failure.
- `succeeded()`: Indicate a metasmoke connection success.
- `get_counter()`: Get metasmoke connection failure counter value. The counter counts consecutive failures.
- `get_last_ping()`: Get time for last status ping to metasmoke. If no time was previously set, returns `None`. Otherwise returns last ping time set as class …

Makyen

@user12986714 get_counter() is not a good name. A counter of what? [Scratch the rest of what I said. I was looking at it as the metasmoke.py interface.]

user12986714

@Makyen get_failure_count

Makyen

@user12986714 better.

SmokeDetector

2:42 AM

[ SmokeDetector | MS ] Link at end of body (99): Remote SASL in Apache Kafka ✏️ by trangtriqc on superuser.com

[ SmokeDetector | MS ] Offensive body detected (43): React Test case using jest and enzyme ✏️ by aqqq on stackoverflow.com

Makyen

@user12986714 It feels like basically all of these should be in metasmoke.py, rather than globalvar. It feels like there's an artificial separation here (exiting from prior to your involvement), which may be being compounded by the overall architecture currently putting things that are really a part of the MS module into globalvars.

SmokeDetector

[ SmokeDetector | MS ] Offensive body detected, blacklisted user (114): Missing test cases for React (Jest + Enzyme) ✏️ by aqqq on stackoverflow.com

[ SmokeDetector | MS ] Blacklisted user (71): Missing test cases for React (Jest + Enzyme) ✏️ by aqqq on stackoverflow.com

user12986714

@Makyen in fact I feels like the majority of globalvars.py should be transferred to each specific class... By first refactoring them into inner classes, it will be easier to transfer them later.

it is always easier to implement getters/setters and new features than to hunt down old code and replace them with new ones...

SmokeDetector

3:03 AM

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Is there a way to delete trimmed parts of a video on iOS? by Robin Charlse on apple.SE

tp- feedback received on Missing test cases for React (Jest + Enzyme) [MS]

[ SmokeDetector | MS ] Potentially bad keyword in body (1): How difficult would it be to make a simple Hebrew/Jewish Calendar covering the last 25 years with no access to the Internet? by AmethystMaria on judaism.SE

Makyen

@user12986714 Well, right now it feels like you're building out GlobalVars, when it would be easier to just move the very limited number of things currently stored there for the metasmoke module into the metasmoke module, then build it out there, if needed. I could be wrong, and only looking at a small slice of what's being done. It could also be that I'm just seeing the process from the opposite POV.

SmokeDetector

3:36 AM

tp feedback received on Brush Cutters recomandation [MS]

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Architecture of video conferencing application by HOO on stackoverflow.com

fp- feedback received on Missing test cases for React (Jest + Enzyme) [MS]

[ SmokeDetector ] Blacklists reloaded at rev 63186b6 (tripleee: ! Blacklist NS doodledigital.net) (running on teward/Osiris)

fp feedback received on How difficult would it be to make a simple Hebrew/Jewish Calendar covering the last 25 years with no access to the Internet? [MS]

[ SmokeDetector | MS ] Bad keyword in answer, blacklisted website in answer, potentially bad keyword in answer (180): Reducing independent set to triangle-free subgraph by mejrometson on cs.SE

Nisse Engström

3:57 AM

@SmokeDetector tpu-

SmokeDetector

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Windows 2016 Server Core - Add node to a SQL Server Failover Cluster (NOT Working) ✏️ by CodingEE on serverfault.com

fp feedback received on Windows 2016 Server Core - Add node to a SQL Server Failover Cluster (NOT Working) [MS]

fp feedback received

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Python Beautiful Soup find_all ✏️ by JGW on stackoverflow.com

fp feedback received on What open-source video editors are available with 4k capabilities? [MS]

fp feedback received

user12986714

4:14 AM

@Makyen socket science is messing things up. Currently metasmoke calls socket science for send status ping, while socket science calls metasmoke for ms_down(), introducing circular import... Maybe I should let socket science be an exception to directly operate globalvars.msstatus.ms_is_up...

SmokeDetector

4:27 AM

[ SmokeDetector ] SmokeDetector started at rev 63186b6 (tripleee: ! Blacklist NS doodledigital.net) (running on teward/Osiris, Python 3.6.9)

Restart: API quota is 18998.

user12986714

(@Makyen) CI was not happy with my code. The reason appears to be that I changed how globalvars initialize. In the original code, globalvars.reload() will be called every time globalvars is imported. However, now I removed that instruction, and let globalvars to be initialized by ws.py, once. Hence the test cases were incredibly angry at my code...

Makyen

@user12986714 I'd need to look more directly at what socket science does and how it's integrated to really comment.

@user12986714 Tests don't necessarily need to be they way they were. There're tests that exist to verify functionality as it's actually used and tests which exist to verify functionality as it happens to be written. If how it's written changes, then it's fine to change tests which are testing how it's written. In general, if a test is testing how it's intended to be used, then usually the overall effect of the test should be kept. In other words, changing the tests is also an option, in many cases.

SmokeDetector

4:43 AM

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Please give me any suggestion about my site by Sarthak-dev on stackoverflow.com

[ SmokeDetector | MS ] Url in title (96): Is there no correct answer on this question? imgur.com/gallery/9yaFAcK by Moose Lovell on math.SE

user12986714

@Makyen the current version on PR is that I made socket science an exception... not through metasmoke.ms_down(), but directly interact with globalvars. I need to document that in refdoc tomorrow when I am on the computer again. (currently only documented as comments in socket science code)

SmokeDetector

fp feedback received

fp feedback received on Please give me any suggestion about my site [MS]

user12986714

4:59 AM

@user12986714 reminder to self: two bugs in current PR. chatcommands.py line 726 and line 728. Also grammar issues in commit message: new class --> new classes in title.

Additionally, add docstring to ms_down() and ms_up().

SmokeDetector

5:30 AM

tpu- feedback received on Is there a way to delete trimmed parts of a video on iOS? [MS]

tpu- feedback received on Can't Get Subtitles to Format Correctly [MS]

Glorfindel

!!/test-a 5kplayer.com

SmokeDetector

> Would not be caught as an answer.

Glorfindel

!!/watch- 5kplayer\.com

SmokeDetector

@Glorfindel The regex contains an unescaped "."; in most cases, it should be "\."; append -force if you really want to do that.

[ SmokeDetector ] Blacklists reloaded at rev 9602fdb (SmokeDetector: Auto watch of 5kplayer\.com by Glorfindel) (running on teward/Osiris)

Glorfindel

(from the same spam ring as videoproc.com - I found 1 FP but a couple of TPs (all already deleted) on Ask Different)

!!/report superuser.com/a/1147184/442468

SmokeDetector

5:36 AM

[ SmokeDetector | MS ] Bad keyword with email in body (57): How to convert JSONP response of an Sharepoint search API? ✏️ by suresh on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in title (1): Can somebody please help me decipher this retard's retard-English? by Hilberto on english.SE

[ SmokeDetector | MS ] Manually reported answer (94): Media Player Classic Crashes on Windows 10 on any file by Y.Christina on superuser.com

fp- feedback received on How to convert JSONP response of an Sharepoint search API? [MS]

fp- feedback received on Can somebody please help me decipher this retard's retard-English? [MS]

tpu- feedback received

[ SmokeDetector | MS ] Mostly punctuation marks in body (29): El problema es que declaro una variable tipo char pero al colocar (? y ¡) , no me reconoce como tipo char si no como entero y me manda un error by Manuel Castillo on es.stackoverflow.com

fp- feedback received on how to speed up org-babel-tangle? [MS]

fp feedback received

[ SmokeDetector | MS ] Toxic body detected (61): Is touching your penis allowed? (No rubbing or masturbation) (please answer fast) by anonymousmuslim on islam.SE

[ SmokeDetector | MS ] Bad keyword in answer, potentially bad keyword in answer (87): Can you get sufficient protein from a vegan diet? by Katty Ikramita on skeptics.SE

[ SmokeDetector | MS ] Bad keyword in answer, potentially bad keyword in answer (87): Is a vegan diet nutritionally complete and suitable for babies? by Katty Ikramita on skeptics.SE

tpu- feedback received on Can you get sufficient protein from a vegan diet? [MS]

tpu- feedback received

fp- feedback received on Is touching your penis allowed? (No rubbing or masturbation) (please answer fast) [MS]

Makyen

5:56 AM

@SmokeDetector ignore-

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in answer, potentially bad keyword in answer, blacklisted user (158): Is being a vegan more environmentally friendly than otherwise? by Katty Ikramita on skeptics.SE

tpu- feedback received

tripleee

!!/blacklist-website payhip\.com

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev c800e02 (SmokeDetector: Auto blacklist of payhip\.com by tripleee) (running on teward/Osiris)

[ SmokeDetector | MS ] Potentially bad keyword in body (1): New WebAuthor Responsive HTML MathJAX Editor by Danilo Manhica on tex.SE

Makyen

@Glorfindel Some old astroturfing?

Glorfindel

@Makyen yes, sometimes the mod tools reveal some other related accounts, and one had an old SU post lying around.

SmokeDetector

6:03 AM

tpu- feedback received

Makyen

@Glorfindel Should we be assuming that "5KPlayer" is something we don't want on SE (in URLs)?

tripleee

I guess payhip is merely an attractive nuisance, this particular user seems to be payhip.com/VeganDiet

but so far blacklisting all of payhip seems like a good idea

perhaps the domain should get the collateral tag, though

Makyen

Yeah, it looks like it's collateral.

SmokeDetector

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): How to get frames from a pygame game by Trevor Galivan on stackoverflow.com

fp- feedback received

Glorfindel

@Makyen hmm ... I only got one hit: stackexchange.com/search?q=%225kplayer.com%22

Have I been doing this wrong or has the functionality changed?

On Ask Different, the score is 6 TP vs. 3 FP, and the TPs are newer (but the last one is from 2018, so they've probably stopped).

The difference is probably that legitimate users use bare URLs less often.

SmokeDetector

6:23 AM

[ SmokeDetector | MS ] Offensive body detected, potentially bad keyword in body (44): making a seductive bard? by Kayley on rpg.SE

fp- feedback received

[ SmokeDetector | MS ] Url-only title, bad keyword in body, bad keyword in title, bad pattern in url body, blacklisted website in body, +5 more (783): charity.gofundme.com/o/en/campaign/keto-fit-pills-reviews by CientonRieds on askubuntu.com

tripleee

@SmokeDetector k

Makyen

@Glorfindel As far as I know, the functionality hasn't changed. SE search has a weird separation that unless you give it the url: directive it doesn't look in URLs. It also, sort-of, separates out things in code format, so to do an exhaustive search, you have to run at least three searches, one each for in normal text, in URLs, and in code.

It also tends to be a bit finicky about using quotes, or not, so I often repeat the searches with and without quotes. Overall, it's a bit of a pain.

tripleee

no MS?

SmokeDetector

tpu- feedback received

tripleee

works again, maybe it was just slow

Makyen

6:27 AM

@tripleee autoflagging? It's slow. Something happened in the infrastructure change that makes the autoflags quite slow to be applied.

There was a day and a half about a day ago where MS wasn't autoflagging at all, but that's been resolved.

SmokeDetector

[ SmokeDetector | MS ] Potentially bad keyword in body (1): When do you feel the benefits of using front-end framework? by James.D on stackoverflow.com

tripleee

@Makyen I tried to click the FIRE icon and it said no MS record for this post

user12986714

!!/ms-status

SmokeDetector

@user12986714 metasmoke is up. Current failure count: 0 (teward/Osiris)

fp- feedback received

user12986714

Osiris does not appear to send statistics

last one 2 hrs ago

Makyen

6:30 AM

@tripleee There is one. Maybe it was slow.

Glorfindel

@Makyen somebody should write a script for that...

Makyen

@Glorfindel That's a good point. I'll should adapt what I have for the URRS, as I've already implemented an OR operator for SE chat search.

Now that I think about it, I'm not sure why I haven't already done so. :;

Glorfindel

That would be great. I'd do it myself but I already have a rather large backlog :)

Makyen

I really know the feeling about backlog.

user12986714

!!/ms-status

SmokeDetector

6:33 AM

@user12986714 metasmoke is up. Current failure count: 0 (teward/Osiris)

user12986714

@Makyen @tripleee Osiris is not sending statistics to MS.

Makyen

@user12986714 Try rebooting. It's definitely scanning.

user12986714

yeah but no stat

!!/reboot

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev c800e02 (SmokeDetector: Auto blacklist of payhip\.com by tripleee) (running on teward/Osiris, Python 3.6.9)

Restart: API quota is 18087.

Makyen

I'd tested a few times over the last couple of days. It had looked like the WebSocket issue had been resolved on SE's side (i.e. a few/several hundred passes without a failure vs. the ~10% failure rate previously).

user12986714

6:37 AM

@Makyen great news.

let's see errorlog? quite weird

!!/errorlog 50

SmokeDetector

### WebSocketConnectionClosedException on teward/Osiris at 2020-05-19T23:32:28Z: Connection is already closed.
File "ws.py", line 240, in <module>
a = ws.recv()
File "/home/smokey/.local/lib/python3.6/site-packages/websocket/_core.py", line 313, in recv
opcode, data = self.recv_data()
File "/home/smokey/.local/lib/python3.6/site-packages/websocket/_core.py", line 330, in recv_data
opcode, frame = self.recv_data_frame(control_frame)
File "/home/smokey/.local/lib/python3.6/site-packages/websocket/_core.py", line 343, in recv_data_frame

Makyen

@user12986714 Woah. Don't use that high of a count, unless you need to. That will give you the last 50 errors, which you almost never would need.

user12986714

@Makyen Sorry. Why there are soooooo many connections closed error?

tripleee

there are probably some old instructions which say it's the number of lines, like it used to be

SmokeDetector

[ SmokeDetector ] SmokeDetector started in standby mode at rev b2f025e (SmokeDetector: Auto watch of elmosoft\.net by Daniil) (running on Makyen/Windows)

tripleee

6:41 AM

none of them very recent, mind

Makyen

Well, a lot of them probably came from when MS was down for a while.

user12986714

Awkward. Still won't send stat. last one two hours ago.

tripleee

perhaps the exception handler just below should simply reopen the connection without a traceback if it's a WebSocketConnectionClosedException

user12986714

388 1.06991 18804 about 2 hours ago

from metasmoke

Makyen

We've changed it so it should just reopen it, although it still has the traceback, IIRC.

user12986714

6:46 AM

Starts reporting now.

SmokeDetector

[ SmokeDetector | MS ] Url-only title, bad ip for hostname in body, bad ip for hostname in title, bad keyword in body, bad keyword in title, +6 more (890): slimketoboost.com/realtone-keto/ by kaqwhumbkpug on askubuntu.com

Makyen

That's what I'd expect. Giving it a good kick usually fixes most things.

SmokeDetector

tpu- feedback received

user12986714

@Makyen maybe we can consider adjusting auto reboot to every 3 hours. Those several tens of seconds spent on rebooting doesn't really matter.

Makyen

!!/watch-force- slimketoboost\.com

!!/watch-force- real[\W_]*+tone[\W_]*+keto(?:[\W_]*+(?:\d++|[\da-f]{4,}+))*

SmokeDetector

6:49 AM

@Makyen That pattern looks like it's already caught by Pattern-matching product name in body; append -force if you really want to do that.

[ SmokeDetector ] Blacklists reloaded at rev b0fd2db (SmokeDetector: Auto watch of real[\W_]*+tone[\W_]*+keto(?:[\W_]*+(?:\d++|[\da-f]{4,}+))* by Makyen) (running on teward/Osiris)

[ SmokeDetector ] Blacklists reloaded at rev 6670e59 (SmokeDetector: Auto watch of slimketoboost\.com by Makyen) (running on teward/Osiris)

Makyen

@user12986714 Maybe. I'd like to see a bit better recovery of both chat command and chat output state prior to that happening. In other words, SD looses where it is reading commands from when it reboots, which results in some commands being skipped, which can be a bit unexpected when you encounter it, and can result in commands being executed out of order, which might be bad.

It also looses the queue for things which it's supposed to be posting in chat. The latter is both a good and bad thing, but it would be better if the default was that it was retained. Note to self: It also needs to be more assured that the debug rooms are told that the reboot is planned.

user12986714

@Makyen we need interprocess communication to implement this. Have a process dedicated to chat, and implement a producer/consumer buffer. (Which may also solve a lot of other problems, such as merging chat msg to overcome throttling)

If implemented, this is a quite large change to architecture though

tripleee

I'd love to see separate components for analysis and chat, like Halflife does it

in fact Halflife would very much like to be able to ask Smokey things if there was a way to do that

Makyen

@user12986714 It can probably be done by just dumping the chat send queue to a pickle, which is how most of the similar stuff is being done. I haven't looked closely enough to determine what's needed.

user12986714

@Makyen iBug has complained about pickle wearing out of ssd super fast...

Martin Sleziak

7:04 AM

!!/report meta.stackexchange.com/questions/348205/…

SmokeDetector

[ SmokeDetector | MS ] Manually reported question (93): How to Get Best Discount Coupon at Kaspersky Antivirus? by marcosnic on meta.SE

tpu- feedback received

tripleee

you don't have to write them to SSD for communicating between two processes over the network though

!!/watch fortworthclearinternet\.com

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev f76d5c6 (SmokeDetector: Auto watch of fortworthclearinternet\.com by tripleee) (running on teward/Osiris)

Makyen

@user12986714 Yes, I know, but the issue there is how often that pickle is written, which is way more often then it needs to be (i.e. at each scan). That can be reduced by a factor of 10 by just writing every 10th scan. Just writing when you're rebooting is nothing.

user12986714

@tripleee true. Just use a producer/consumer buffer

SmokeDetector

7:08 AM

[ SmokeDetector ] Blacklists reloaded at rev f89687c (tripleee: ! Watch IP 107.180.48.209) (running on teward/Osiris)

tripleee

top 11 /16 netblocks in *_cidrs.yml:

... I've been thinking we should have a level below watch, for collecting possibly relevant posts for deciding on e.g. expanding an IP block

user12986714

@tripleee log?

tripleee

would be nice if we could say !!/collect 104.16.0.0/15 and have metasmoke collect posts which match this pattern, but not actually issue spam reports for them

Makyen

@user12986714 The logs aren't saved.

tripleee

we could arrange for that with something like cloudwatch though

user12986714

7:13 AM

@Makyen yeah. Implement a mechanism to save them, like !!/log bad_ip

tripleee

(actually make that 104.16.0.0/12)

SmokeDetector

[ SmokeDetector ] SmokeDetector started at rev 3fb949b (Makyen: Tell debug rooms when doing scheduled reboot -autopull) (running on teward/Osiris, Python 3.6.9)

Restart: API quota is 17713.

Glorfindel

!!/allspam security.stackexchange.com/users/234995/mukesh-shukla

Makyen

We have to have a place to save them. Right now we have MS, which I don't think we want to store them there, unless they are separate in some way, or as chat messages.

SmokeDetector

[ SmokeDetector | MS ] Manually reported answer (batch report: post 1 out of 2) (94): Where does tamper data extract the password string from? by Mukesh Shukla on security.SE

[ SmokeDetector | MS ] Bad keyword in link text in body, link at end of body, pattern-matching website in body, potentially bad ip for hostname in body, potentially bad keyword in body, +1 more (268): Where Is Noida Escorts Service? by Richa Desai on meta.SE

[ SmokeDetector | MS ] Manually reported answer (batch report: post 2 out of 2) (94): What are the security risks of using WhatSender tool? by Mukesh Shukla on security.SE

tripleee

7:15 AM

!!/watch webpositionexpert\.com

SmokeDetector

tpu- feedback received on Where does tamper data extract the password string from? [MS]

tpu- feedback received on Where Is Noida Escorts Service? [MS]

tpu- feedback received

[ SmokeDetector ] Blacklists reloaded at rev 4161a7a (SmokeDetector: Auto watch of webpositionexpert\.com by tripleee) (running on teward/Osiris)

tripleee

!!/watch noidaescortss\.com

!!/watch escortsgallery\.in

!!/watch nainitalescorts\.in

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev 8f7d814 (SmokeDetector: Auto watch of noidaescortss\.com by tripleee) (running on teward/Osiris)

user12986714

@Makyen can have a log room... Just thoughts

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev dfde445 (SmokeDetector: Auto watch of escortsgallery\.in by tripleee) (running on teward/Osiris)

[ SmokeDetector ] Blacklists reloaded at rev a113890 (SmokeDetector: Auto watch of nainitalescorts\.in by tripleee) (running on teward/Osiris)

Makyen

7:17 AM

@user12986714 We can, but what level of data is it desired to have stored? I don't think that it's only normal SD chat reports.

!!/bisect noida escorts service

SmokeDetector

@Makyen That pattern looks like it's already caught by Potentially bad keyword in answer and Potentially bad keyword in body; append -force if you really want to do that.

Matched by the following regexes:
escorts?\W?service on line 2630 of watched_keywords.txt
\L<city>\Wescorts? on line 3614 of watched_keywords.txt

tripleee

the nsXX.domaincontrol.com name servers would be good to get more data on, too; I have watched a few of them and subsequently retired a few of those watches and left others, but I really have no idea if those are prevalent in spam and less prevalent in legitimate posts, and/or if there should be others on the watchlist

user12986714

@Makyen Considering that even if stored on ms, it can still only be plain reports and needs manual analysis anyways,we can just dump whatever posts logged into chat (maybe using some encoding?)

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev 87188a3 (tripleee: ! Blacklist IP 148.66.136.66) (running on teward/Osiris)

tripleee

chat throttling could become a major bottleneck, Halflife already suffers from being several minutes behind whenever there are more than a few posts in quick succession

user12986714

7:25 AM

since this doesn't have to be real time (unlike normal reports), we can have sd dump multiple posts in one msg, like how it flooded errorlog out...

Makyen

Yes, if you are expecting significantly more traffic, then we would need to implement one of the alternatives for getting more chat throughput. Given the CM changes earlier this year, the likelihood of actually getting the rate limit reduced for SD is much lower at this point.

@user12986714 Yes, that's one of the proposed alternatives.

Glorfindel

!!/watch- coim\.in

(from the Security spammer's profile)

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev 99e6c65 (SmokeDetector: Auto watch of coim\.in by Glorfindel) (running on teward/Osiris)

[ SmokeDetector ] Blacklists reloaded at rev 8c808cc (tripleee: ! Watch IP 185.201.11.98) (running on teward/Osiris)

[ SmokeDetector | MS ] Potentially bad keyword in body (1): About Windows regedit and VBA Editor Add-In by jyoti on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Calculate number of battery cycles ✏️ by robmiller505 on electronics.SE

fp- feedback received

fp- feedback received on About Windows regedit and VBA Editor Add-In [MS]

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Fast streaming camera view to video for linux (v4l) system device by g3n35i5 on blender.SE

[ SmokeDetector | MS ] Bad keyword in link text in answer (81): This media file is write proteted by ablackack on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Animation when changing divs by Pinnci on stackoverflow.com

[ SmokeDetector | MS ] Mostly punctuation marks in answer, repeating characters in answer (117): Stacking compasses and what happens by lala on physics.SE

[ SmokeDetector | MS ] Bad keyword with email in body (57): How to restrict fields in jenkins signup page by Uday Kiran Reddy on stackoverflow.com

[ SmokeDetector | MS ] Bad keyword in answer, blacklisted user (157): Can I make Voice/Video Call in Desktop or Web App with Cisco Webex SDK? by Luca Pfeifer on stackoverflow.com

[ SmokeDetector | MS ] Username similar to website in answer (61): BigCommerce Marketing Banner - Change Size by Martechlive on stackoverflow.com

tpu- feedback received

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title (194): What is lucky patcher app? by BradfordGarcia on android.SE

[ SmokeDetector | MS ] Bad ip for hostname in body, blacklisted website in body, blacklisted website in title, link following arrow in title, pattern-matching website in body, +5 more (600): DEAL NOW>> supplementdare.com/test-infernox-uk/ by doroson on askubuntu.com

fp- feedback received on How to restrict fields in jenkins signup page [MS]

tpu- feedback received on Can I make Voice/Video Call in Desktop or Web App with Cisco Webex SDK? [MS]

[ SmokeDetector | MS ] Potentially bad keyword in body (1): npm is give me are error in laravel. i have are error 404 error from laravel webshoket. how to i fix it by Sanjaya Elapatha Arachchige on stackoverflow.com

naa- feedback received on This media file is write proteted [MS]

[ SmokeDetector | MS ] Bad ip for hostname in answer, bad ns for domain in answer, blacklisted website in answer, potentially problematic ns configuration in answer (267): How to disable restart print job which goes to error state? by Robert john on superuser.com

[ SmokeDetector | MS ] Potentially problematic ns configuration in body (1): Redirect homepage and specific url from old domain to new domain with htaccess by Simone Verstyle on stackoverflow.com

SmokeDetector

8:30 AM

[ SmokeDetector | MS ] Potentially bad keyword in body, potentially bad keyword in title (2): How to make a video meme in kinemaster for Samsung Note7? by Maxinegrant on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Plastic insert damaged when tried to open gravity wheelguard ixow by Mihai Mărgi on bicycles.SE

[ SmokeDetector | MS ] Blacklisted user (71): How to use content settings to set up repositories of standard and required values for email messages by Martechlive on stackoverflow.com

fp- feedback received on Plastic insert damaged when tried to open gravity wheelguard ixow [MS]

tpu- feedback received

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Ajax search box and dropdown filter in PHP and MYSQL by Eormo on stackoverflow.com

fp- feedback received on Fast streaming camera view to video for linux (v4l) system device [MS]

[ SmokeDetector | MS ] Bad ip for hostname in body, bad keyword in body, blacklisted website in body, body starts with title and ends in url, link at end of body, +1 more (589): resistance sponsor with counterfeit by NatilyWinsen on askubuntu.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Certainly, the mistakes that we male and female mortals make when we have our own way might fairly raise some wonder that we're so fond of it by davidconnor9573 on stackoverflow.com

tpu- feedback received

tpu- feedback received on resistance sponsor with counterfeit [MS]

SmokeDetector

9:06 AM

[ SmokeDetector | MS ] Offensive answer detected (78): What does 14px/26px font size in css do? by Matt Wells on stackoverflow.com

[ SmokeDetector | MS ] Mostly non-latin answer, potentially bad keyword in answer, shortened url in answer (105): FireWire or USB by سرور مجازی on sound.SE

tpu- feedback received

[ SmokeDetector | MS ] Potentially bad keyword in body (1): I wonder if my code can be more optimized by airbug on stackoverflow.com

[ SmokeDetector | MS ] Bad keyword in answer, potentially bad keyword in answer (87): Error: You need to load the kernel first in Grub by james scott on askubuntu.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Is There Working Spell for My Husband to Obey Me and Be Under Control? by Lena Mama on meta.SE

[ SmokeDetector | MS ] Link at end of answer (60): How to make an HTML GUI for PowerShell scripts? by Albert Jeremy on stackoverflow.com

tpu- feedback received on [ Error: You need to load the kernel first in Grub](//askubuntu.com/a/1241685) [MS]

user435118

9:40 AM

!!/test onesimcard.com

SmokeDetector

> Would not be caught as a post, title or username.

[ SmokeDetector | MS ] Potentially bad asn for hostname in body (1): Do you add extra oil when replacing eggs in baked goods? by Taylor on cooking.SE

user435118

!!/watch onesimcard\.com

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev b926c1c (SmokeDetector: Auto watch of onesimcard\.com by Daniil) (running on teward/Osiris)

user435118

!!/test onesimcard.com

SmokeDetector

> Potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username
----------
Potentially bad keyword in body - Position 0-14: onesimcard.com
Potentially bad keyword in title - Position 0-14: onesimcard.com
Potentially bad keyword in username - Position 0-14: onesimcard.com

[ SmokeDetector | MS ] Blacklisted website in body, pattern-matching product name in body, potentially bad keyword in body, potentially bad keyword in title (199): Blood Balance Formula Scam Finally Exposed By Verified Users: by formulaus20 on workplace.SE

tpu- feedback received

[ SmokeDetector | MS ] Bad ip for hostname in body, bad keyword in body, bad keyword in title, blacklisted website in body, pattern-matching product name in body, +2 more (589): Granite Male Enhancement :Avoid cardiovascular disease or brain fog by wooktom on superuser.com

Kulfy

9:48 AM

!!/bisect pnjsharptech.com

SmokeDetector

@Kulfy Matched by pnjsharptech\.com on line 15066 of watched_keywords.txt

Kulfy

Still alive

user435118

@Kulfy flagged, should be down soon

SmokeDetector

[ SmokeDetector | MS ] Bad keyword in body, potentially bad keyword in body (97): Has anyone heard about the world forum susudunia here? by Susudunia on stackoverflow.com

Kulfy

Gone. Thanks

SmokeDetector

9:51 AM

tpu- feedback received

[ SmokeDetector | MS ] Bad keyword in body (96): FSUM7343 cannot open "String" for output: EDC5129I No such file or directory by Tamilarasan on stackoverflow.com

user435118

!!/watch forum\.susudunia\.xyz

SmokeDetector

fp- feedback received

[ SmokeDetector ] Blacklists reloaded at rev dc9be10 (SmokeDetector: Auto watch of forum\.susudunia\.xyz by Daniil) (running on teward/Osiris)

[ SmokeDetector | MS ] Few unique characters in body (94): ............................................................. ✏️ by fraulifang on math.SE

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Why is the output DC voltage of full wave not equal to the RMS of the original AC input? by Abdallah Youssef on electronics.SE

tpu- feedback received on Granite Male Enhancement :Avoid cardiovascular disease or brain fog [MS]

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in title, bad keyword in username, blacklisted website in body, potentially bad ip for hostname in body, +1 more (385): Do you know about this product Pure Sol Keto? by Pure Sol Keto on askubuntu.com

tpu- feedback received

Glorfindel

!!/report meta.stackexchange.com/q/348216/295232

SmokeDetector

[ SmokeDetector | MS ] Manually reported question (93): How is social distance dead serious in Covid-19? by JenniferShoffner on meta.SE

Glorfindel

!!/watch wns\.com

SmokeDetector

10:04 AM

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Playing music over Bluetooth in Android car unit that only accepts Bluetooth phone call ✏️ by anonymous on android.SE

[ SmokeDetector ] Blacklists reloaded at rev 238ef03 (SmokeDetector: Auto watch of wns\.com by Glorfindel) (running on teward/Osiris)

fp- feedback received on Why is the output DC voltage of full wave not equal to the RMS of the original AC input? [MS]

[ SmokeDetector | MS ] Potentially bad ns for domain in body, potentially bad keyword in body (2): web ca cuoc uy tin 2020 by ASIA KUBET86 on meta.SE

tpu- feedback received

SmokeDetector

10:15 AM

tpu- feedback received on How is social distance dead serious in Covid-19? [MS]

tripleee

!!/watch kubet86\.asia

SmokeDetector

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Discord.js split the message by Anej on stackoverflow.com

[ SmokeDetector ] Blacklists reloaded at rev 5c8f5f6 (SmokeDetector: Auto watch of kubet86\.asia by tripleee) (running on teward/Osiris)

[ SmokeDetector | MS ] Few unique characters in answer, repeating words in answer (172): "We found extra characters at the end of json" Excel Power Query by kezix on stackoverflow.com

tpu- feedback received

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): The most basic way of building a simple html website by Peura on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): How do I stop my 2D Sprite from streching out in other aspect ratio? by Tamla Htoo on stackoverflow.com

fp- feedback received

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Has anyone successfully controlled sexual urges by fasting? by Kabir Sarkar on islam.SE

[ SmokeDetector | MS ] Potentially bad ns for domain in answer (1): How to creating a user profile (properties) for real estate recommendations engine by rumi on stackoverflow.com

tpu- feedback received

user435118

10:31 AM

!!/watch homelight\.in

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev a47f3a7 (SmokeDetector: Auto watch of homelight\.in by Daniil) (running on teward/Osiris)

[ SmokeDetector | MS ] Potentially bad ip for hostname in body (1): ExpressionChangedAfterItHasBeenCheckedError: Expression has changed after it was checked in OWLCool by Zafar HAYAT on stackoverflow.com

[ SmokeDetector | MS ] Pattern-matching website in body (97): The URL of the index.html file in my new Google bucket includes the google storage path by Richard on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Windows 10 shuts down instead of sleep by Dicse Gábor on superuser.com

Magisch

@ArtOfCode I saw some of it before it got deleted on MSE, it was an utter crapfest

two or three now deleted posts had a lot of answers and comments. Shog was really doing a kindness by saying they chose to delete it all instead of going through, you'd probably have seen dozens of suspensions.

SmokeDetector

10:55 AM

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Best way to find biggest & smallest number in a random list? ✏️ by Vinetos on scicomp.SE

[ SmokeDetector | MS ] Potentially problematic ns configuration in body (1): Object passed as parameters as if we are passing constructor values by shreyas kagalkar on stackoverflow.com

SmokeDetector

11:07 AM

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Why few of the executors are in idle state when i submitted a spark job in cluster by lakshmi n on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): how to speed up org-babel-tangle? ✏️ by Jerry Leibold on emacs.SE

fp- feedback received on Best way to find biggest & smallest number in a random list? [MS]

[ SmokeDetector | MS ] Mostly punctuation marks in answer (36): 35mm Black and White Film Artifacts by user92195 on photo.SE

[ SmokeDetector | MS ] Potentially bad ns for domain in body (1): Не могу подключиться к gitlab по SSH, почему? by Дмитрий Храмов on ru.stackoverflow.com

[ SmokeDetector | MS ] Potentially bad ns for domain in body, potentially bad keyword in body (2): Have you used this company for croydon taxis to gatwick airport transfers by chirshlips1 on travel.SE

[ SmokeDetector | MS ] Potentially bad ns for domain in answer, potentially bad keyword in answer (2): Does the taxi waiting time impact the taxi fare when taking a taxi from Taiwan Taoyuan International Airport (TPE)'s taxi stand? by chirshlips1 on travel.SE

tpu- feedback received on Have you used this company for croydon taxis to gatwick airport transfers [MS]

tpu- feedback received

fp- feedback received on Не могу подключиться к gitlab по SSH, почему? [MS]

[ SmokeDetector | MS ] Blacklisted website in answer, link at end of answer (153): How to install infura to a server runing ubuntu 16.04 by Coin Joker on ethereum.SE

tpu- feedback received

[ SmokeDetector | MS ] Potentially problematic ns configuration in answer (1): Trouble Connecting to sql server Login failed. "The login is from an untrusted domain and cannot be used with Windows authentication" by Shafeeque Mohammed on stackoverflow.com

[ SmokeDetector | MS ] Bad keyword in body, bad keyword in link text in body, bad keyword in title (263): How you can get IELTS trainer jobs in Chandigarh? by Speedy Jobs on stackoverflow.com

Robert Columbia

11:36 AM

@SmokeDetector tpu-

user435118

!!/test www,expresscouriercars.co.uk

SmokeDetector

> Potentially bad ns for domain in body, potentially bad ns for domain in title, potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username
----------
Potentially bad keyword in body - Position 4-28: expresscouriercars.co.uk
Potentially bad keyword in title - Position 4-28: expresscouriercars.co.uk
Potentially bad keyword in username - Position 4-28: expresscouriercars.co.uk
Potentially bad ns for domain in body - expresscouriercars.co.uk NS suspicious ns2.phase8.net.,ns0.phase8.net.,ns1.phase8.net.

[ SmokeDetector | MS ] Few unique characters in body (94): deleted......................... ✏️ by Karla on math.SE

user435118

@SmokeDetector self vandalism

user435118

!!/blacklist-website expresscouriercars\.co\.uk

SmokeDetector

[ SmokeDetector ] Blacklists reloaded at rev c4138ad (SmokeDetector: Auto blacklist of expresscouriercars\.co\.uk by Daniil) (running on teward/Osiris)

[ SmokeDetector | MS ] Potentially bad asn for hostname in body (1): Accordion via bootstrap in IE browser by Илья Покровский on stackoverflow.com

[ SmokeDetector | MS ] Potentially bad keyword in body (1): Can't run python in VScode after editing settings.json ✏️ by generationX on stackoverflow.com

tp- feedback received on deleted......................... [MS]

[ SmokeDetector | MS ] Potentially bad keyword in answer (1): Хочу стать web-разработчиком by Tatweer development on ru.stackoverflow.com

fp- feedback received on Trouble Connecting to sql server Login failed. "The login is from an untrusted domain and cannot be used with Windows authentication" [MS]

Transcript for