Fix compute_head_fingerprint folding case-only title/meta changes#2036
Conversation
compute_head_fingerprint lowercased the entire <head> (`head_html.lower()`) before extracting the title and meta values, so the captured signal values were lowercased too. Two heads that differ only in the case of a title or meta value (e.g. "iPhone" vs "IPHONE", "Buy Now" vs "BUY NOW") therefore hashed to the same fingerprint. CacheValidator treats an equal fingerprint as unchanged, so a genuinely updated page was reported FRESH and stale cached content was served. Match tags/attributes case-insensitively (re.IGNORECASE) against the original head instead, so the extracted values keep their original case. Tag/attribute case-insensitivity is preserved; identical content still hashes identically. Adds regression tests: a case-only value change now changes the fingerprint (fails on the old code), while tag/attribute-only case differences do not.
chuenchen309
left a comment
chuenchen309
left a comment
There was a problem hiding this comment.
Confirmed the bug against main — the analysis is right:
compute_head_fingerprint("<head><title>iPhone</title></head>")
== compute_head_fingerprint("<head><title>IPHONE</title></head>") # True on main
compute_head_fingerprint("<head><title>iPhone</title></head>")
!= compute_head_fingerprint("<head><title>Android</title></head>") # True — so it's the case-folding specificallyhead_lower = head_html.lower() was doing double duty: making the tag match case-insensitive (which you want) and lowercasing the captured value (which you don't). Splitting those with re.IGNORECASE + matching against the original is the right fix, and the meta patterns still match PROPERTY="OG:TITLE" style markup the same as before.
One upgrade effect worth putting in the PR description / release notes. This changes the fingerprint of essentially every page, not just case-only-changed ones — the extracted signal itself is different now:
head = "<head><title>My Product Page</title></head>"
# main extracts: 'my product page'
# this PR extracts: 'My Product Page'cache_validator.py:131 and :163 compare new_fingerprint == stored_head_fingerprint, and every stored fingerprint out there was computed by the lowercasing version. So on the first check after upgrading, any cached page whose title has an uppercase letter compares unequal → reported as changed → re-fetched. It's one-time and self-correcting (the new fingerprint gets stored), and it's the safe direction to fail in, but a full cache revalidation right after an upgrade looks a lot like a bug from the outside. Cheap to pre-empt with a line in the release notes.
Not blocking — the fix is correct and the regression test pins the right property. Just flagging so nobody has to debug the revalidation wave.
(Disclosure: I use an AI coding assistant. Both snippets above are ones I ran against main myself.)
Summary
compute_head_fingerprintlowercases the entire<head>before extracting signals:So the captured values are lowercased, not just the tag/attribute matching. Two heads that differ only in the case of a title or meta value hash to the same fingerprint:
CacheValidatortreats an equal fingerprint as "unchanged", so a genuinely updated page is reported FRESH and stale cached content is served.Fix
Match tags/attributes case-insensitively with
re.IGNORECASEagainst the original head, so extracted values keep their case. Tag/attribute case-insensitivity is preserved (e.g.<TITLE>/META NAME=...CONTENT=still parse), and identical content still hashes identically.Testing
Adds two regression tests:
test_value_case_change_changes_fingerprint— a case-only value change now changes the fingerprint (fails on the current code).test_tag_and_attribute_case_does_not_change_fingerprint— markup-only case differences still yield the same fingerprint.