It found this oddity: which contains: "wwwwwwweqr3r2qqerqr3wrefdfdf3ettghrgwxgesgh4ssegszgsrewrer" - which seems like a strange headbash, but probably too short; though better than quoting a full size example. ... Something to do when we are crashed and repairs are underway.
Writing a post with a repeated sequence of characters causes Regex to backtrack when parsing using an unreasonable amount of time and memory, paralyzing the system.
@EriktheOutgolfer SD gets the HTML rendered by SE. SE strips some tags in the Markdown➞ HTML conversion due to potential security issues. The issue is that SD doesn't also get the Markdown source and run, at least some, detections against the Markdown. Currently all of our detections are written to run against the HTML, rather than Markdown, so it's a bit difficult to cut-over to using just Markdown, which would be more assured of detecting everything, but require us to interpret Markdown.
@EriktheOutgolfer If this comment is in reference to Glorfindel's manual report, where you left a comment, it's a fair question: If we are catching "<a href=" not sure why we don't look for "<a title=" (followed by href shortly thereafter)?
@Rob SD detects against HTML generated by SE from Markdown. If there are HTML tags in the Markdown, then attribute order is enforced, which if not followed will result in the HTML tag being stripped prior to SD seeing it. Thus, it's not possible for the title to proceed the href, thus we don't test for it and use a regex that only looks for enough information to make it very likely that it's an actual <a> tag. example
This is one of a plethora of things that would need to change if we were testing only against the Markdown rather than the HTML supplied by SE. If we tested against only the Markdown, then we'd need to fully parse both Markdown and HTML, which we don't currently do. We could improve our detections by having a limited number of things run against the raw Markdown, but as I said earlier, fully switching over is non-trivial.
@Makyen - Interesting, I tested the text that @EriktheOutgolfer provided using an Online HTML2TXT Converter and indeed it does the same, which is probably the intended behavior. I found what is claimed (haven't found an online Proxy) to be a means to process the HTML using w3m, lynx, or links - we could fetch using my
earlier suggestion for catching ReDoS without dipping into our quota, though some more code would have to be written.
@Rob Fetching the Markdown in addition to the HTML costs us nothing wrt. SE API quota. It's just changing the filter provided with the SE API request. Doing so does increase bandwidth somewhat and would add additional required processing (i.e. it's more data). No one has raised objections on those grounds.
