Skip to content

perf(store): measure snapshot persistence and real SQLite facade contention #2333

Description

@DeliciousBuding

Summary

Measure the remaining FileStore (E-P2-6) and SQLite (E-P3-8) candidates in #2256 before proposing production changes. Timeout measurement is merged in #2332. This is an isolated, synthetic workload baseline, not a claim about production frequency or an instruction to alter a live service.

Scope

  • FileStore: resident fixtures at 10 / 100 / 1,000 runs; 10 items, two artifact metadata entries and one single-file checkpoint per run; ten runs per thread. Measure snapshot copying, identical indented JSON encoding into a counting discard sink, frozen-snapshot save (including file sync/rename), and complete explicit Flush. Report serialized bytes and allocations separately from filesystem/durability costs. Explicit Flush measurements do not measure the 50 ms debounce frequency.
  • SQLite: use the real SQLiteStore GetRun/ListRuns facade with 0 / 1 / 4 busy-loop reader goroutines and one synchronous durable writer at the same fixed resident scales. Reads are in-memory Store calls, not invented SQL readers. Record write throughput, bounded reader/writer latency histograms, DB pool wait deltas, errors and recovery correctness. Report percentile buckets as upper bounds, not exact percentiles.
  • Seed through validated in-memory Store methods, persist before timing, and keep entity counts fixed during the workload. Restore through the actual backend load path after timing and compare full snapshots, including content and order. Final benchmark fixtures and measurements must not silently bypass errors.
  • Keep SQL MaxOpenConns and per-connection PRAGMAs unchanged. Newly created connections would not automatically receive the initializer's PRAGMAs, so increasing the pool is not a valid control here.

Files / File Ownership

One local writer owns:

  • edge-server/internal/store/persistence_bench_test.go (fixture and FileStore benchmarks)
  • edge-server/internal/store/sqlite_contention_bench_test.go (SQLite workload and bounded measurement helpers)
  • docs/architecture/02-edge-server.md only if a concise clarification of the real memory-read/persistence boundary is necessary.

No other write lanes. No production source, API, dependencies, CI jobs, configuration or runtime changes.

Interfaces / Invariants

Use existing Store/FileStore/SQLiteStore APIs; snapshot/save internals are called only as explicitly labeled cost-isolation controls. Preserve all durability, error, Close, locking, cleanup and checkpoint semantics. Only disposable local temp stores; artifact/checkpoint content is generated fixture data, never read from a user workspace.

Acceptance

  • Review actual constructor/CRUD/snapshot/Flush paths and existing durability tests before implementing.
  • Package tests and all new benchmark cases pass under Linux race smoke; Windows benchmark smoke/CI must exercise actual FileStore and pure-Go SQLite behavior.
  • Non-race repeated measurements with environment, filesystem, GOMAXPROCS, workload, command, raw samples and limits saved on this issue. Correctness checks remain outside timed sections except operation success/count checks.
  • go vet, staticcheck, diff check and the existing required PR gates pass. If the owner document changes, run the existing doc SSOT verifier.
  • Conclude whether measured costs justify a follow-up; do not manufacture an optimization or timing gate. Reconcile 后端性能候选收口:限定测量与 ctx / checkpoint 修复完成 #2256 and clean only the merged task's worktree/branch.

Negative Constraints

No live stores, credentials, production requests, deployment, connection-pool retuning, SQLite SQL-reader substitution, lock rewrites, background timer changes, timing thresholds or claims about RSS/peak heap/real user latency. No local progress/STATE document; status and measurement evidence live on GitHub.

Dependencies

Current origin/master; #2332 merged. EventLog #2304 is an independent persistence component, not a prerequisite or shared lock.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions