Skip to content

Stop list nesting short of the recursion limit - #1685

Open
Steve-20-ai wants to merge 3 commits into
Python-Markdown:masterfrom
Steve-20-ai:fix/list-nesting-recursion-limit
Open

Steve-20-ai wants to merge 3 commits into
Python-Markdown:masterfrom
Steve-20-ai:fix/list-nesting-recursion-limit

Conversation

@Steve-20-ai

Copy link
Copy Markdown

Description

markdown.markdown() raises an uncaught RecursionError on a deeply nested
list, so any service rendering untrusted Markdown can be crashed by a small
input. 500 list markers on one line is about 1 KB:

>>> import markdown
>>> markdown.markdown('- ' * 500 + 'x')
RecursionError: maximum recursion depth exceeded

The same happens for an indented list of about 500 lines, and for a list inside
a blockquote. It is the class of problem #1534 covered, with a different
trigger.

A list nests one level per parseBlocks call, and the list processors had no
limit on that. BlockQuoteProcessor has had one since #799, which is why
>>>>>>... renders instead of raising:

def test(self, parent, block):
    return bool(self.RE.search(block)) and not util.nearing_recursion_limit()

This applies the same guard to the list processors. ListIndentProcessor.test()
and OListProcessor.test() now return False once nearing_recursion_limit()
is true, and UListProcessor inherits the latter. The block then falls through
to the paragraph processor, so the markers which no longer nest are kept as
text, exactly as the blockquote guard keeps its > markers. No content is lost:

Input Before After
'- ' * 500 + 'x' (1 KB) RecursionError renders, 0.04s, 447 nested <ul>
'1. ' * 500 + 'x' (1.5 KB) RecursionError renders, 0.02s
'> ' + '- ' * 500 + 'x' (1 KB) RecursionError renders, 0.04s
600 lines of ' ' * i + '- x' (354 KB) RecursionError renders, 0.7s

The guard alone is not enough to make that path reasonable, which is the part
of the report about rendering rather than crashing: get_items() appended every
continuation line with '{}\n{}'.format(item, line), copying the whole item on
each line, which is quadratic in the number of lines of an item. A 1200 level
list, which is one huge item once nesting stops, took 20.0s with the guard and
3.8s once the lines of an item are collected in a list and joined once. That is
the second commit of this change, and it also speeds up any list item which
spans many lines.

Output is unchanged for input which nests below the limit: all 1192 tests pass,
and rendering a corpus of 3000 random list documents produces byte-identical
output before and after.

The remaining time for a very deep list is spent in the block processors
re-scanning the tail once per level, which this change does not address.

Two notes for review:

  • The depth at which nesting stops depends on sys.getrecursionlimit(), as it
    does for blockquotes. The added test uses recursionlimit() to exercise the
    guard at a low limit, in the style of TestBlockquoteBlocks.test_nesting_limit().
  • I kept the changelog entry and the quadratic fix in this pull request because
    the report asks for both halves. Happy to split them if you would prefer to
    review the guard on its own.

fixes #1679

AI Assistance Disclosure

  • AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

    AI tools were use to assist with the following:

    • Code
    • Code Comments
    • Documentation
    • Changelog Entry
    • Commit Message
    • The Description Above
    • Other: tests

    The tool was the WorkBuddy coding agent. I reviewed the change and ran the
    checks below myself, and I am the one responsible for it in review.

Checklist

Checks run locally: python -m unittest discover tests (1192 tests, OK, 111
skipped), flake8 markdown tests (no new findings).

A list nests one level per `parseBlocks` call, so a deeply nested list --
500 markers on one line is about 1 KB of input -- reached the interpreter's
limit and raised `RecursionError` out of `markdown.markdown()`. Anything
rendering untrusted Markdown could be crashed by it.

`BlockQuoteProcessor` has been limited this way since Python-Markdown#799: its `test()`
stops nesting once `util.nearing_recursion_limit()` is true, and the
remaining markers are kept as text. The list processors had no such guard,
so `ListIndentProcessor.test()` and `OListProcessor.test()` now apply it,
`UListProcessor` inheriting the latter. Content is not lost, the markers
which no longer nest are rendered as text.

The guard alone left the degraded path slow, because `get_items()` appended
every continuation line with `'{}\n{}'.format(item, line)`, copying the
whole item each time and making it quadratic in the number of lines. It now
collects the lines of an item in a list and joins them once, which renders a
1200 level list in 3.8s instead of 20s. Output is unchanged on a corpus of
3000 random list documents.

fixes Python-Markdown#1679
@waylan waylan added the needs-review Needs to be reviewed and/or approved. label Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review Needs to be reviewed and/or approved.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Uncaught RecursionError (DoS) on deeply nested lists; blockquotes are guarded but lists are not

2 participants