Conversation
30*60 was passed where a time.Duration was expected, so the TTL was 1800 nanoseconds and cached article bodies expired immediately. Pass 30*time.Minute instead. Cached parsed feeds get their own, smaller limit (20): each cached feed carries every item of the feed and is much larger per entry than a single article body.
Each chromedp parse starts its own temporary headless Chrome process, so limit how many may run at once (2) to bound peak memory during refreshes. Queued parses wait on the caller's context, so they still respect the per-feed timeout.
Add loading="lazy" and decoding="async" to images in the rendered article body and in translated paragraphs, so offscreen images are only fetched and decoded as they approach the viewport. The already sanitized HTML is only annotated, never restructured.
The page cache is private to each connection and the pool allowed 25 of them, so the cache alone could grow to 800 MiB. Use an 8 MiB per-connection cache (200 MiB worst case across the pool) and keep only two idle connections warm, so page cache is released shortly after a burst of queries. Pool size stays at 25 to keep concurrent feed refreshes from serializing. The VACUUM migration still restricts itself to a single connection; it now restores both pool limits from the shared constants.
Serving a cached image read the file into memory, and a cache miss buffered the whole download as well, so rendering a page with several large images spiked the process heap. Stream from disk, download into a temporary file that is renamed into place only when complete, and serve through http.ServeContent so range and conditional requests work. An interrupted download no longer leaves a truncated cache entry behind.
Every selection re-fetched and re-parsed the same body, so moving between articles repeated the same work. Keep the last eight bodies in an LRU with in-flight de-duplication, skip empty bodies so a failed load can be retried, and invalidate an entry whenever the stored body changes (reload, full-text fetch, bulk cleanup).
The list kept every loaded article mounted and each row costs hundreds of KB in the web view, so a long scroll left thousands of rows resident. Keep only the rows around the viewport mounted, using measured row heights with spacers for the rest, and release unmounted rows from the translation observer.
Note the content-cache TTL fix, the bounded feed cache, the capped browser parses, lazy article images, and the reader memory changes in the Unreleased section.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the recent memory work. Eight individually revertable commits: three
fixes that were sitting uncommitted in the working tree, four memory/throughput
changes, and a CHANGELOG entry.
Commits
Fixes (pre-existing work in the tree):
fix(cache):30*60was passed where atime.Durationwas expected, so thecontent-cache TTL was 1800 nanoseconds and cached bodies expired immediately.
Cached parsed feeds now have their own, smaller bound (20), since each cached
feed carries every item of the feed.
perf(feeds): cap concurrent chromedp parses at 2 -- each parse starts its owntemporary headless Chrome, so this bounds peak memory during refreshes.
Queued parses still respect the per-feed timeout.
perf(reader): nativeloading="lazy"/decoding="async"hints for images inthe rendered article body and in translated paragraphs.
Memory and throughput:
perf(db): SQLite's page cache is per connection and the pool allowed 25, sothe cache alone could reach 800 MiB. Use an 8 MiB per-connection cache and keep
only two connections idle (worst case 200 MiB). The pool size is unchanged so
concurrent feed refreshes do not serialize.
perf(media): serving a cached image read the whole file into memory, and acache miss buffered the whole download too. Stream from disk, download into a
temporary file that is renamed into place only when complete, and serve through
http.ServeContent(which also adds range and conditional request support).perf(reader): keep the last eight article bodies in an LRU with in-flightde-duplication. Empty bodies are not cached, so a failed load can be retried;
entries are dropped on reload, full-text fetch and bulk cleanup.
perf(reader): render only the rows near the viewport, using measured rowheights with spacers for the rest, and release unmounted rows from the
translation observer.
Measurements
Measured on the Linux desktop build against isolated copies of a real database.
(-17%), whole process tree -47 MB. Cost: -4.7% throughput under a saturated
12-way synthetic API load (about 330 req/s, far above interactive use).
+376 MB (-37%) in a same-session comparison.
(the same binary varied by 30%+ between sessions). They are kept for resource
behaviour -- no whole-image buffering, no repeated fetch/parse -- not for a
measured RSS win.
Caveats worth knowing while reviewing:
covered by unit tests (measured heights, list shrink, anchor drift, multi-column
rows) and exercised in the desktop build, but visual smoothness is best checked
by scrolling it once.
ArticleList.vue's card modal only; the file isshared by two changes and was split by hunk.
Checks
go test -timeout=5m ./internal/...-- 660 passed, 58 packagesgo vet ./...,gofmt-- cleannpm run test:unit-- 202 passedeslinton the touched files,npm run build-- clean