@Machavity That pattern looks like it's already caught by Potentially bad keyword in body and Potentially bad keyword in answer; append -force if you really want to do that.
> Potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username ---------- Potentially bad keyword in body - Position 0-11: bugbros.com Potentially bad keyword in title - Position 0-11: bugbros.com Potentially bad keyword in username - Position 0-11: bugbros.com
@Stormblessed Ahh, yeah, it's a problem there too (I had it on the list, but forgot to mention it in addition to FIRE). Yes, there's a problem with the code that's used to convert the adjusted HTML provided by SD (and stored by MS) into correct, displayable HTML. I have code that does a much better job for FIRE, but haven't pushed it yet. My intent was to adapt that for MS too.
Unfortunately, given what SD does to the post body text prior to testing/transmitting it to MS, it's not possible to perfectly recover the actual HTML, under some conditions. Thanks for mentioning the issue.
@Makyen HTML display on MS is handled by a gem, not by custom code. Should hopefully just require some adjustment of parameters... either that or it's a security issue in the gem.
@ArtOfCode The gem is likely doing the "right" thing with the HTML it's provided. The problem is (at least for FIRE) that SD mangles the HTML prior to testing/reporting it. It's necessary to convert what MS has stored in its database back into actual HTML. Unfortunately, given what SD does (converts HTML entities into characters) results in text that, in certain cases, can't be deterministicly converted back into the source HTML.
I haven't actually looked at what is happening on MS for display of the post, so I could be wrong. However, the display appears to look as if that is what is happening.
@Makyen What MS has in the database is HTML. I'd need to check to be sure, but I believe Smokey keeps an untouched copy of the HTML it gets from the API, and that's what gets sent to MS.
@ArtOfCode At least what MS delivers via the MS API in the body to FIRE is the text that was HTML, but which has been processed by SD to convert HTML entities to characters. FIRE must then reverse the processing that SD has done in order to correctly display the preview. It's definitely not an untouched version of the HTML delivered by the SE API for the post.
> Url in title, potentially bad keyword in body, potentially bad keyword in title, potentially bad keyword in username ---------- Potentially bad keyword in body - Position 16-30: abitshadow.com Potentially bad keyword in title - Position 16-30: abitshadow.com Potentially bad keyword in username - Position 16-30: abitshadow.com Url in title - Position 0-30: https://lending.abitshadow.com
> Potentially bad keyword in body, potentially bad keyword in title ---------- Potentially bad keyword in body - Position 0-1: 🍆 Potentially bad keyword in title - Position 0-1: 🍆
@ArtOfCode Yep. That's the processing I'm talking about. It converts HTML entities into characters, which is what the regexes we use expect. That conversion results in the text no longer being HTML which produces the same output. If that text is directly used as HTML for the DOM, then the display won't necessarily be correct (depends on what the HTML originally was).
There can also be security issues if that text is directly used as HTML source for a page, due to things such as <script>alert('executed')</script> now being <script>alert('executed')</script>, or other exploits.
If jQuery is used to add that text to the DOM, security issues are a bit worse than using vanilla JavaScript methods to add HTML text, as jQuery "helpfully" converts text like <script>alert('executed')</script> into a <script> that actually executes. If added as HTML text with vanilla JavaScript methods, then the <script> is created, but doesn't execute. However, there are other exploits which still could be used.
The issues (for SD, FIRE, and MS) are most commonly in <code> blocks, where all of the HTML code would have had the < characters as < prior to SD using html.unescape(). To convert it back to the HTML which can be correctly displayed requires reversing the html.unescape(), which can mostly be done correctly, but there are some cases where it's not possible to be sure to convert it back into the same original HTML.
I have code for FIRE which does this correctly (as much as is possible). I'd intended to also add it to MS, but it sounds like the conversion isn't done in JavaScript, so I'm unsure if I will easily be able to do that.
