Skip to content

fix(html2text): thread escape_backslash into escape_md_section() call#2082

Open
chuenchen309 wants to merge 1 commit into
unclecode:developfrom
chuenchen309:fix/html2text-escape-backslash-not-threaded
Open

fix(html2text): thread escape_backslash into escape_md_section() call#2082
chuenchen309 wants to merge 1 commit into
unclecode:developfrom
chuenchen309:fix/html2text-escape-backslash-not-threaded

Conversation

@chuenchen309

Copy link
Copy Markdown

Summary

escape_backslash is never forwarded into escape_md_section() at its call site in HTML2Text.handle_data() (crawl4ai/html2text/__init__.py), unlike its sibling flags escape_dot/escape_plus/escape_dash, which are all threaded through correctly. Because escape_md_section() has its own hardcoded default of escape_backslash=True, that default always wins — backslashes get escaped/doubled in the generated markdown regardless of what the instance (or CustomHTML2Text's own escape_backslash=False default, which is what the default markdown-generation pipeline actually uses) configures.

Fixes: n/a (no existing issue filed for this)

List of files changed and why

  • crawl4ai/html2text/__init__.py — pass escape_backslash=self.escape_backslash into the escape_md_section() call in handle_data(), the same way escape_dot/escape_plus/escape_dash are already passed.
  • tests/unit/test_html2text_escape_backslash.py — new unit test covering both escape_backslash=False (must not double backslashes) and escape_backslash=True (must still escape), using CustomHTML2Text directly.

How Has This Been Tested?

  • Added a failing test first (TDD): with escape_backslash=False explicitly set, converting <p>Use \* for multiplication</p> produced Use \\* for multiplication (doubled) instead of the expected Use \* for multiplication — confirmed red against current code.
  • After the fix, the same test goes green, and a companion test confirms escape_backslash=True still escapes as before (no default-behavior flip).
  • Ran the full non-browser test subset (tests/unit/, tests/deep_crawling/, tests/regression/test_reg_extraction.py -m "not network", tests/test_source_sibling_selector.py, tests/general/test_strip_markdown_fences.py, tests/general/test_async_markdown_generator.py): same pass/fail counts before and after the change (verified via git stash/git stash pop A-B comparison) — the only failures present are pre-existing environment failures unrelated to this diff (missing local Playwright browser binary, BrowserType.launch: Executable doesn't exist), not caused by this change.
  • python -m py_compile on both changed files: clean.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added/updated unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

AI disclosure

This fix was found, implemented, and verified with Claude (Anthropic) assistance in a human-in-the-loop workflow: I reviewed the exact code path, wrote/ran the failing test myself before the fix, confirmed the fix goes green, and ran the broader test subset before submitting. Commit includes a Co-Authored-By: Claude Opus 4.8 trailer for transparency.

Sibling flags (escape_dot/escape_plus/escape_dash) are all forwarded from
the HTML2Text instance into escape_md_section() at its call site in
handle_data(), but escape_backslash was never passed. As a result,
escape_md_section()'s own hardcoded default of True always wins, so
backslashes are escaped/doubled in generated markdown regardless of what
the instance (or CustomHTML2Text's own escape_backslash=False default,
used by the default markdown-generation pipeline) configures.

Fix: pass escape_backslash=self.escape_backslash alongside the other
escape_* flags, the same way the siblings are threaded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant