Skip to content

Convert stress-test scripts to mocha describe/it format#307

Draft
Copilot wants to merge 1 commit into
mainfrom
copilot/update-distress-test-syntax
Draft

Convert stress-test scripts to mocha describe/it format#307
Copilot wants to merge 1 commit into
mainfrom
copilot/update-distress-test-syntax

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 9, 2026

The stress-test scripts were standalone Node.js executables using process.exit() for pass/fail signaling and lacked "type": "module" in their package.json, causing a MODULE_TYPELESS_PACKAGE_JSON warning at runtime.

Changes

  • stress-test/memory.spec.js (replaces memory.js) — Rewrites the three-phase memory test as a mocha spec:

    • Each phase is an it() block with a descriptive name
    • Assertions throw Error instead of setting process.exitCode
    • Phase runners return Promises; async/await used throughout
    • Temp data directory lifecycle managed via before/after hooks
  • stress-test/stress.spec.js (new) — Wraps the crash-safety orchestration script (run.sh) in a single it() that spawns it as a child process and fails the test on non-zero exit

  • stress-test/package.json — Adds "type": "module" (eliminates the Node.js warning) and a "test" script pointing to both spec files

describe('Memory stress test', function () {
    it('Phase 1 – write: memory growth is not super-linear', async function () { ... });
    it('Phase 2 – read: all events are readable', async function () { ... });
    it('Phase 3 – Index open/close: no memory leak', function () { ... });
    // ...
});

describe('Crash-safety stress test', function () {
    it('recovers from a hard crash with bounded data loss', function (done) { ... });
});

Run via the root's mocha:

npx mocha stress-test/memory.spec.js stress-test/stress.spec.js

@albe
Copy link
Copy Markdown
Owner

albe commented May 13, 2026

@copilot why is running the memory and stress tests through mocha a good approach? Doesn't mocha itself add some overhead and fragility to the test?

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 13, 2026

@copilot why is running the memory and stress tests through mocha a good approach? Doesn't mocha itself add some overhead and fragility to the test?

The concern is valid and worth addressing for each test separately:

Crash-safety (stress.spec.js): Mocha is essentially a no-op here — the it() block just spawns run.sh as a child process and checks the exit code. All the actual work (writing, killing, recovering) happens in isolated subprocesses, so mocha overhead is completely irrelevant.

Memory (memory.spec.js): Mocha does add baseline heap usage, but it doesn't affect the correctness of the measurements because all assertions operate on deltas, not absolute values:

  • Phase 1/3 use a baseline sample taken after mocha has fully loaded, then measure growth from that point. Mocha's footprint is already baked into both the before/after samples and cancels out.
  • The super-linear growth check compares (growthHigh / growthLow) — both figures are relative to the same write-ready baseline, so any constant mocha overhead contributes identically to both.
  • The leak checks measure heapGrowth / LEAK_CYCLES across open/close cycles, again delta-only.

On fragility: the spec files use only describe/it/before/after — nothing mocha-specific that could introduce subtle timing or isolation issues. The suite was previously a standalone script that called process.exit(1) on failure; the mocha conversion replaces that with thrown Errors and returns a proper exit code through the test runner, which is actually more composable and less fragile than process-exit-based signalling.

Copilot AI requested a review from albe May 13, 2026 18:47
Copilot finished work on behalf of albe May 13, 2026 18:47
@albe
Copy link
Copy Markdown
Owner

albe commented May 13, 2026

@copilot should the same then be applied to all benchmark test scripts?

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 13, 2026

@copilot should the same then be applied to all benchmark test scripts?

No — the bench/ scripts are categorically different and shouldn't be converted.

The stress tests make correctness assertions (memory growth is sub-linear, crash recovery succeeds), which map naturally to mocha's pass/fail model.

The benchmark scripts use the benchmark npm library to produce statistical throughput measurements (ops/sec, ms/op with warm-up cycles, variance, etc.). Wrapping them in mocha would:

  • Break the benchmark.js statistical sampling model — mocha has its own async orchestration and timeouts that conflict with Benchmark.Suite.run()
  • Eliminate the useful throughput output (beautify-benchmark tables) in favour of a pass/fail result that carries no performance information
  • Serve no new purpose since bench/package.json already has "type": "module" and the scripts don't use process.exit() for signalling

The benchmarks are already well-structured: they're run via npm run bench inside the bench/ directory and output comparative tables. No changes needed there.

Copilot finished work on behalf of albe May 13, 2026 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants