[facelessuser]

Description

Due to the way Python Markdown processes content, we must safeguard
against content placed into tails that we've already processed.

1. Built HTML elements we plan to replace ahead of time and place in
   the cache. They are built as the text is when they were processed.
   Each entry in the cache will contain the element, the region it
   replaces, and the anchor (position of next delimiter Python Markdown
   will before that that region so we can account for new index of the
   modified buffer).
2. While the cache is not empty, we will sometimes receive tail content,
   already processed tail content so we can identify and ignore them.
3. Now we just return the pre-built HTML elements until the cache is
   spent.

Fixes #1694

AI Assistance Disclosure

• No AI tools were used in preparing this PR.

• AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

  AI tools were use to assist with the following:

  • Code
  • Code Comments
  • Documentation
  • Changelog Entry
  • Commit Message
  • The Description Above
  • Other: Debugging and fuzz testing dropped text cases

Checklist

• This PR follows the contribution guidelines.
• The code follows the Code Style Guide.
• The commit message follows the Commit Message Style Guide.
• I have added or updated relevant docs, including release notes if applicable which follow the Documentation Style Guide.
• I have added or updated relevant tests.
• I have not requested, and will not request, an automated AI review for this PR.

[facelessuser]

It seems in #1691, I had hand ported changes from BetterEm, which Python Markdown's solution is based on, over to Python Markdown. But I had missed that some of the tests used check_markdown instead of what this project uses, assertMarkdownRenders. That's on me, and it surprisingly didn't fail in CI, so I missed that we didn't have everything.

So, I:

• Fixed the tests here that weren't properly running that would have caught some of these dropped text cases.
• Brought over two lines missing from the "rule of 3" solution.
• Brought over some placeholder awareness that I thought was only needed if Punctuation rules were applied. It turns out we need to be aware of Python Markdown placeholders even without the Punctuation rules.

I've run a comparison of BetterEm logic to the logic here to make sure we aren't missing anything. The same tests that pass there pass here. There should be no more dropped text.

[facelessuser]

For reference, the "rule of 3" changes here were supposed to mirror the changes in:

• facelessuser/pymdown-extensions@0a5c403

And these placeholder changes mirror BetterEm changes here:

• facelessuser/pymdown-extensions@668694a

I have a couple extra things I have to handle in BetterEm due to some Pandoc style superscript and subscript, which is why I mistakenly didn't include some of the rule of 3 things here, and since some of the tests were silently not running here due to the test name mismatch, it got missed.

Had the tests been running properly, I would have also caught that we needed the placeholder changes as well. Live and learn, I guess.

[facelessuser]

Oh, and for completeness, the rest of the emphasis fixes here are based on the changes:

• facelessuser/pymdown-extensions@626fa38

If anything, the original changes were fully verified; it was just that when I ported them, I got burned by not properly updating the tests to the framework here and assumed all the tests were passing.

To be sure, I have now also hand-verified the test results in this PR to make sure they are correct, and double-checked the results in the CommonMark dingus.

I believe we have all the changes properly ported over.