feature(monitor): post-capture filters + JSON/CSV export#174
Open
jamby77 wants to merge 2 commits into
Open
Conversation
3 tasks
0f03629 to
e8e7644
Compare
0502b31 to
c2e8cb8
Compare
e8e7644 to
b90faee
Compare
c2e8cb8 to
b90faee
Compare
2d42d85 to
997fbc8
Compare
Closes Phase 2 frontend MVP. The session-detail page gains a Filters &
export panel that filters the captured stream by command, client,
key glob, and time window — then downloads the filtered slice as JSON
or CSV via a new server endpoint.
Backend:
- monitor-line.parser.ts: pure module
- parseMonitorLine(text) → {ts, tsRaw, db, addr, cmd, args, key, raw}
Handles backslash-escaped quotes, IPv6 bracket addresses, keyless
commands (PING etc.).
- matchesFilters(line, {command, client, key, afterTs, beforeTs}):
case-insensitive command exact match, addr substring match, glob
match on the first arg / key (* and ?), inclusive timestamp window.
- lineToCsvRow + CSV_HEADER: RFC 4180 escaping for commas, quotes,
newlines.
- GET /monitor/sessions/:id/export?format=json|csv (+ filter query
params) streams persisted chunks, parses each line, applies filters,
and emits either {sessionId, count, lines[]} JSON or a CSV with a
header row. Content-Disposition: attachment; filename=
monitor-session-<id>.<fmt>. 404 on unknown session. Default format
is json when an unknown value is supplied.
Frontend:
- New FiltersAndExport component on the session-detail page (below
the live tail).
- 5 inputs: Command, Client, Key glob, After (datetime-local),
Before (datetime-local).
- In-page preview count derived from the live-tail buffer (parser
inlined to avoid a server round-trip on every keystroke). Export
buttons hit the server endpoint which sees the full session.
- Two <a download> buttons styled as outline Buttons — clean
browser-native download with the URL carrying current filters.
- useMonitorTail lifted from TailView up to MonitorSession so the
Filters panel and the TailView share one buffer.
Tests:
- monitor-line.parser.spec.ts: 22 cases covering all parser edge cases
(escaped quotes, backslashes, IPv6, keyless commands, malformed
input), all filter axes including glob wildcards and AND semantics,
and CSV escaping.
- monitor.controller.spec.ts extended with 5 export cases (404,
unfiltered JSON, command-filtered, CSV header, format fallback).
Total backend suite: 184 tests across 14 suites, all green (1 caught
during testing: IPv6 bracket address parsing — the initial header
regex was too greedy and stopped at the inner `]`; fixed with a
non-greedy capture anchored on `\]\s+"`).
Verification (Playwright, live):
- Started a 5s session, sent 30 SETs against foo + 30 GETs against
user:* → session captured 70 lines (mix of AUTH/INFO/PING + SETs
+ GETs).
- Direct curl to /export?format=json → count: 70, cmds:
['AUTH', 'GET', 'INFO', 'PING', 'SET'].
- Filter ?command=GET → count: 30.
- Filter ?key=user:* → count: 30, all GET user:N.
- CSV export starts with header `ts,ts_raw,db,addr,cmd,args,key`.
- UI: typed `GET` and `user:*` into the form → preview text
"Buffer match: 30 of 70 lines. Export uses the full session,
server-side." matched the API. Export-link URLs updated with the
filter query string in real time.
Screenshot at docs/assets/pr11-filters-export.png.
Part of PR 11 of 25 in
docs/plans/specs/monitor-command/plan-implementation.md (closes
Phase 2 frontend MVP).
Self-review fixes: - Filter inputs (command/client/key) now go through trimmedOrUndefined so the buffer-preview client behaviour (which trims) and the export endpoint behaviour stay in sync. Without this, 'GET ' could match in the preview but return 0 rows in the export. - Key glob filters are now bounded to 128 characters via cappedKeyFilter to defuse catastrophic backtracking via patterns like '*a*a*a*a*a*' — globToRegex compiles to a non-anchored .* chain that is ReDoS-prone against equally long captured keys.
d307ec0 to
606edcb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 11 of 25 in
docs/plans/specs/monitor-command/plan-implementation.md. Stacked on top of #173 (PR 10). Closes Phase 2 frontend MVP. The session-detail page gains a Filters & export panel that filters the captured stream by command, client, key glob, and time window — then downloads the filtered slice as JSON or CSV via a new server endpoint.Backend
monitor-line.parser.ts— pure moduleparseMonitorLine(text)→{ts, tsRaw, db, addr, cmd, args, key, raw}. Handles backslash-escaped quotes, IPv6 bracket addresses, keyless commands (PING etc.).matchesFilters(line, {command, client, key, afterTs, beforeTs}): case-insensitive command exact match, addr substring match, glob match on the first arg / key (*and?), inclusive timestamp window.lineToCsvRow+CSV_HEADER: RFC 4180 escaping for commas, quotes, newlines.GET /monitor/sessions/:id/export?format=json|csv(+ filter query params) streams persisted chunks, parses each line, applies filters, emits either{sessionId, count, lines[]}JSON or a CSV with a header row.Content-Disposition: attachment; filename=monitor-session-<id>.<fmt>. 404 on unknown session. Default format isjsonwhen an unknown value is supplied.Frontend
FiltersAndExportcomponent on the session-detail page (below the live tail).<a download>buttons styled as outline Buttons — clean browser-native download with the URL carrying the current filters.useMonitorTaillifted fromTailViewup toMonitorSessionso the Filters panel and the TailView share one buffer.Test plan
SKIP_DOCKER_SETUP=true pnpm --filter api test -- --testPathPatterns "monitor|capture-sessions|health-gate|provider-detector|acl-checker|preflight|capture-writer|demo-mode|tail|monitor-line"→ 184 tests across 14 suites, all passpnpm --filter web exec tsc --noEmit→ exit 0MONITOR_DEV_PREVIEW=true pnpm dev:apiandVITE_MONITOR_DEV_PREVIEW=true pnpm dev:webcurl '/monitor/sessions/:id/export?format=json'→{count, lines[]}with parsed structurecurl '/monitor/sessions/:id/export?format=json&command=GET'→ filtered subsetcurl '/monitor/sessions/:id/export?format=json&key=user:*'→ globbed subsetcurl '/monitor/sessions/:id/export?format=csv'→ starts withts,ts_raw,db,addr,cmd,args,keyGETin Command field → preview "Buffer match: N of M lines" updatesuser:*in Key glob → preview narrows further; Export-link URL carriescommand=GET&key=user%3A*monitor-session-<id>.json; opens in browser and matches the preview countmonitor-session-<id>.csv; first line is the headerNotes for reviewers
[^\]]+that stopped at the inner]of[::1]:port. Fixed with a non-greedy(.+?)\]\s+"anchor. Covered by a unit test.filters-and-export.tsxto avoid round-tripping on every keystroke. The parser is intentionally simpler than the backend's (noargsarray, no escape decoding) since it only feeds filter predicates. If we ever surface the parsed fields in the UI, the right move is a shared@betterdb/sharedparser — flagged.(redacted)), not something this PR adds. We get it for free from the underlying protocol.docs/assets/pr11-filters-export.png.Stacked PR
Base branch is
feature/monitor-web-live-tail(#173), so the diff shown is ONLY PR 11 changes.Phase 2 MVP done
This closes Phase 2 (frontend MVP). Next is Phase 3: cross-reference engine + UI panel — the actual differentiator the spec calls out.