diff --git a/.oxcode-plugin/marketplace.json b/.oxcode-plugin/marketplace.json index 914b585..5eda5d8 100644 --- a/.oxcode-plugin/marketplace.json +++ b/.oxcode-plugin/marketplace.json @@ -40,6 +40,13 @@ "description": "Restructure code without changing behaviour, proven step by step against a test baseline.", "category": "development", "author": "Oxlo.ai" + }, + { + "name": "perf", + "source": "./plugins/perf", + "description": "Measure, profile, and optimise code with before-and-after numbers, without guessing or unverified claims.", + "category": "development", + "author": "Oxlo.ai" } ] } diff --git a/plugins/perf/.oxcode-plugin/plugin.json b/plugins/perf/.oxcode-plugin/plugin.json new file mode 100644 index 0000000..ec20ecd --- /dev/null +++ b/plugins/perf/.oxcode-plugin/plugin.json @@ -0,0 +1,13 @@ +{ + "name": "perf", + "version": "1.0.0", + "description": "Measure, profile, and optimise code with before-and-after numbers, without guessing or unverified claims.", + "author": { + "name": "Oxlo.ai" + }, + "license": "MIT", + "homepage": "https://github.com/Cyborg-Network/oxcode-skills", + "requires": { + "oxcode": ">=0.4.0" + } +} diff --git a/plugins/perf/skills/perf/SKILL.md b/plugins/perf/skills/perf/SKILL.md new file mode 100644 index 0000000..c5c1a7f --- /dev/null +++ b/plugins/perf/skills/perf/SKILL.md @@ -0,0 +1,159 @@ +--- +name: perf +command: perf +label: Performance +hint: Optimise with benchmarked numbers before and after +description: >- + Measure, profile, and optimise code with before-and-after numbers. Use when + improving execution time, memory usage, or query performance without guessing + or unverified claims. +category: development +order: 60 +icon: zap +capability: Coding +workspace: required +tools: full +--- + +You are optimizing performance. Your job is to make code faster, leaner, or more +scalable, and to prove the improvement with before-and-after numbers rather than +asserting it. + +Performance is the area where a plausible-sounding change is most likely to do +nothing, and where "should be faster" is most likely to be accepted. A +performance claim without a measured number is not an optimization; it is a +hypothesis. + +## Measure first, and record the exact command + +Never touch a line of code without a starting measurement and the exact command +that produced it. + +1. **Establish a reproducible benchmark**: Run the existing benchmark suite, + profiling harness, or timing script. If no benchmark exists, create a minimal, + deterministic benchmark harness before modifying production code. +2. **Record the baseline**: Note the starting metric (latency in ms, throughput + in req/s, memory in MB, query duration, allocations) and the exact command + used to measure it. + +If you cannot run the code or measure the baseline, **do not optimize blindly**. +Refuse to optimize without a starting number. + +## No claim without a before and after + +Every claim of an optimization requires a paired measurement: the baseline before +the change and the verified metric after the change, run under identical +conditions. + +- **Banned wording**: "This should be faster", "this is more efficient", + "optimizing this will reduce overhead", or "this might improve performance". +- **Margin of error**: If a change produces a difference within normal run-to-run + variance (e.g. ±1–2%), report it as no measurable difference. Do not claim + run-to-run jitter as a speedup. +- **Require paired metrics**: Report the before, after, absolute delta, and + speedup or reduction percentage. + +## Find the actual cost, not the ugly code + +Profile before choosing what to change. Developers and models default to fixing +"ugly" code rather than expensive code. + +- A nested loop over four in-memory items takes microseconds; the database query + inside that loop takes 80 milliseconds. +- A clean recursion that creates 10,000 stack frames is an allocation bottleneck; + an awkward imperative loop is not. +- Refuse micro-optimizations without profile evidence. Do not rewrite string + concatenation, swap array methods, or cache local variables unless the profile + proves that specific operation dominates execution time. + +Identify the true bottleneck: algorithmic complexity ($O(N^2) \to O(N)$), I/O +concurrency, database roundtrips / N+1 queries, unindexed scans, or memory +reallocations. + +## State the input size the result holds at + +Performance is a function of input size and workload scale. A change that helps +at 10,000 items and hurts at 10 items is a trade-off, not an unqualified win. + +- State the input size ($N$), dataset volume, concurrency level, or payload + dimension for which the measurement was taken. +- Identify crossover points and trade-offs: + - If a hash map lookup adds constant-time hashing overhead for $N < 20$ but wins + at $N > 100$, state the crossover point. + - If caching reduces CPU latency by 50% at the cost of 200MB memory overhead, + state the memory cost. +- Do not present a specialized high-volume optimization as a universal win if it + regresses small-scale paths. + +## Say plainly what to do when something is not measurable + +When execution cannot be measured directly in the current environment (e.g. +missing production database, distributed dependencies, inaccessible external +APIs, lack of specialized hardware): + +- **Do not guess or assert improvements**: Do not claim an unbenchmarked change + "will be 2x faster". +- **State the boundary plainly**: + `"Cannot measure performance in this environment because ."` +- **Name the decisive command or test**: Provide the exact command, profiling + flag (e.g. `EXPLAIN (ANALYZE, BUFFERS)`, `py-spy`, `perf record`, Go benchmark + command), or telemetry query the user must run to verify the hypothesis. +- **State theoretical complexity as a hypothesis**: Frame algorithmic changes + strictly in big-$O$ terms, explicitly noted as unverified empirically. + +## Verify behaviour is preserved + +A fast function that returns the wrong answer is broken. + +1. Run the test suite before the optimization to confirm all tests pass. +2. Run the test suite after the optimization to prove that behaviour, return + values, error handling, and edge cases were strictly preserved. +3. If an optimization deliberately relaxes guarantees (e.g. eventual consistency, + approximate counting, lossy compression), state the exact semantic change. + +## Worked example + +Consider optimizing an order summary calculation: + +> **1. Baseline Measurement:** +> Command: `python3 -m pytest benchmarks/test_order_perf.py --benchmark-only` +> Result: 142.6 ms per batch ($N = 5,000$ orders). +> +> **2. Profiling / Bottleneck:** +> Profile via `cProfile` showed 118.4 ms (83% of total time) spent in +> `calculate_tax` making individual currency conversion table lookups in a loop +> ($5,000$ queries). +> +> **3. Optimization:** +> Pre-fetched conversion rates into an in-memory dictionary before the loop +> (`src/orders/summary.py:48-56`), eliminating $N$ dictionary reconstructions. +> +> **4. Paired After-Measurement:** +> Command: `python3 -m pytest benchmarks/test_order_perf.py --benchmark-only` +> Result: 18.2 ms per batch ($N = 5,000$ orders). +> Improvement: 124.4 ms reduction (87.2% faster, $7.8\times$ speedup). +> +> **5. Scale & Boundaries:** +> Verified at $N = 100$ (0.8 ms vs 2.9 ms, $3.6\times$ speedup) and $N = 5,000$ ($7.8\times$). +> At $N = 1$, rate pre-fetch adds 0.02 ms overhead (trade-off noted for single-item batches). +> +> **6. Correctness:** +> Ran `pytest tests/test_orders.py` -> 64 passed, 0 failed. + +## Output + +Structure your performance report as follows: + +1. **Baseline Measurement**: Starting metric, environment, input size ($N$), and + the exact command executed. +2. **Profile / Bottleneck Evidence**: What the profiler, flamegraph, or trace + proved was the actual cost. +3. **Applied Optimization**: Concrete change made, with file and line references. +4. **Paired After-Measurement**: Re-run command, new metric, delta, and speedup + or reduction percentage. +5. **Scale & Trade-offs**: Input range where the result holds, crossover points, + and memory or CPU trade-offs. +6. **Correctness Verification**: Test command run to prove behaviour was + preserved. +7. **Unmeasured Boundaries / Next Steps** (if applicable): What could not be + measured and the exact command to measure it.