Avoid rescanning for a comment close after unclosed comments

`HTMLExtractor.parse_comment` searches for `-->` (or `--!>`) from each
`<!--` to the end of the raw data. When there is none, it passes `<`
through as data and returns `i + 1`, so the next `<!--` repeats the
same failing scan. A run of n unclosed comments therefore costs O(n^2):
`'<!--' * 32000` (125 KB) takes about 5 seconds to convert and 250 KB
takes about 22 seconds.

If no close exists after some position in `rawdata`, none exists after
any later position in the same string either. The parser now records
the string and position of the last failed search and skips the search
when it would start at or after that position. The string is compared
by identity so the record is never reused for different raw data.
Output is unchanged.

Author: naaz234

.spell-dict

@@ -108,6 +108,7 @@ refactor
 refactored
 refactors
 registerExtension
+rescan
 resettable
 richeland
 RSS

docs/changelog.md

@@ -29,6 +29,8 @@ See the [Contributing Guide](contributing.md) for details.
   consisting only of `{` and some text with many trailing spaces (#1656).
 * Fix quadratic-time regex backtracking in hash headers when a long run of `#`
   characters is followed by more text on the same line.
+* Fix quadratic time when a document contains many unclosed HTML comments (`<!--`
+  with no matching `-->`).
 * Fix regression in `BacktickInlineProcessor` where text could be lost when opening
   and closing backticks did not match (#1668).
 * Escape the language name in the `codehilite` extension's non-Pygments output to

markdown/htmlparser.py

@@ -158,6 +158,8 @@ def reset(self):
         self._cache: list[str] = []
         self.cleandoc: list[str] = []
         self.lineno_start_cache = [0]
+        # The `rawdata` and position of the last failed search for a comment close.
+        self._no_commentclose: tuple[str | None, int] = (None, 0)
 
         super().reset()
 
@@ -359,8 +361,15 @@ def parse_pi(self, i: int) -> int:
     def parse_comment(self, i, report=True):
         rawdata = self.rawdata
         assert rawdata.startswith('<!--', i), 'unexpected call to parse_comment()'
-        match = commentclose.search(rawdata, i+4)
+        # If no close was found after an earlier position in the same `rawdata`, there
+        # is none after this one either. Don't rescan the rest of the document again.
+        searched, pos = self._no_commentclose
+        if searched is rawdata and i + 4 >= pos:
+            match = None
+        else:
+            match = commentclose.search(rawdata, i+4)
         if not match:
+            self._no_commentclose = (rawdata, i + 4)
             self.handle_data('<')
             return i + 1
         if report:

tests/test_syntax/blocks/test_html_blocks.py

@@ -1860,6 +1860,13 @@ def test_multiple_unclosed_comments_no_hang(self):
             '<p>&lt;!-- and &lt;!--</p>'
         )
 
+    def test_many_unclosed_comments(self):
+        """Test that each unclosed comment does not rescan the rest of the document."""
+        self.assertMarkdownRenders(
+            '<!--' * 50000,
+            '<p>' + '&lt;!--' * 50000 + '</p>'
+        )
+
     def test_no_hang_issue_1586(self):
         """Test no hang condition for issue #1586."""