Skip to content

fix(link-checker): inspect HTML bodies after successful HEAD requests - #984

Open
baoypeng wants to merge 1 commit into
practical-tutorials:masterfrom
baoypeng:codex/2026-11-07-html-link-checking
Open

baoypeng wants to merge 1 commit into
practical-tutorials:masterfrom
baoypeng:codex/2026-11-07-html-link-checking

Conversation

@baoypeng

@baoypeng baoypeng commented Nov 7, 2026

Copy link
Copy Markdown

An HTML tutorial URL that responds to HEAD with HTTP 200 is currently accepted without fetching its body. Since HEAD has no response body, the existing soft-404 title detector never sees 'Page not found', and a missing tutorial can be reported as OK.

Fetch HTML/XHTML bodies with the existing bounded GET path after a successful HEAD. Also fetch when Content-Type is absent. Non-HTML resources keep the HEAD-only path; redirects and unsupported HEAD requests retain their existing handling.

Add nine offline regression tests using a local HTTP server. Cover soft-404 detection, normal HTML, missing Content-Type, XHTML, binary resources, redirects, HEAD fallback, domain-policy overrides and HTTP 410. Run the tests in the validation workflow and trigger that workflow for tooling changes.

Validation:

  • On the unchanged upstream implementation, six of the nine tests fail.
  • With this patch, all nine pass.
  • README lint passes: 457 entries, 0 errors, 0 warnings.
  • Patch applies cleanly to the recorded upstream snapshot.

Tradeoff: successful HTML links now require one extra bounded request. No third-party dependencies are added. Hosted GitHub Actions has not been run locally; only its workflow configuration was updated.

This is a tooling-only change. AI assistance: Codex investigated, implemented and validated it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant