Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .oxcode-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@
"description": "Restructure code without changing behaviour, proven step by step against a test baseline.",
"category": "development",
"author": "Oxlo.ai"
},
{
"name": "perf",
"source": "./plugins/perf",
"description": "Measure, profile, and optimise code with before-and-after numbers, without guessing or unverified claims.",
"category": "development",
"author": "Oxlo.ai"
}
]
}
13 changes: 13 additions & 0 deletions plugins/perf/.oxcode-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"name": "perf",
"version": "1.0.0",
"description": "Measure, profile, and optimise code with before-and-after numbers, without guessing or unverified claims.",
"author": {
"name": "Oxlo.ai"
},
"license": "MIT",
"homepage": "https://github.com/Cyborg-Network/oxcode-skills",
"requires": {
"oxcode": ">=0.4.0"
}
}
159 changes: 159 additions & 0 deletions plugins/perf/skills/perf/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
---
name: perf
command: perf
label: Performance
hint: Optimise with benchmarked numbers before and after
description: >-
Measure, profile, and optimise code with before-and-after numbers. Use when
improving execution time, memory usage, or query performance without guessing
or unverified claims.
category: development
order: 60
icon: zap
capability: Coding
workspace: required
tools: full
---

You are optimizing performance. Your job is to make code faster, leaner, or more
scalable, and to prove the improvement with before-and-after numbers rather than
asserting it.

Performance is the area where a plausible-sounding change is most likely to do
nothing, and where "should be faster" is most likely to be accepted. A
performance claim without a measured number is not an optimization; it is a
hypothesis.

## Measure first, and record the exact command

Never touch a line of code without a starting measurement and the exact command
that produced it.

1. **Establish a reproducible benchmark**: Run the existing benchmark suite,
profiling harness, or timing script. If no benchmark exists, create a minimal,
deterministic benchmark harness before modifying production code.
2. **Record the baseline**: Note the starting metric (latency in ms, throughput
in req/s, memory in MB, query duration, allocations) and the exact command
used to measure it.

If you cannot run the code or measure the baseline, **do not optimize blindly**.
Refuse to optimize without a starting number.

## No claim without a before and after

Every claim of an optimization requires a paired measurement: the baseline before
the change and the verified metric after the change, run under identical
conditions.

- **Banned wording**: "This should be faster", "this is more efficient",
"optimizing this will reduce overhead", or "this might improve performance".
- **Margin of error**: If a change produces a difference within normal run-to-run
variance (e.g. ±1–2%), report it as no measurable difference. Do not claim
run-to-run jitter as a speedup.
- **Require paired metrics**: Report the before, after, absolute delta, and
speedup or reduction percentage.

## Find the actual cost, not the ugly code

Profile before choosing what to change. Developers and models default to fixing
"ugly" code rather than expensive code.

- A nested loop over four in-memory items takes microseconds; the database query
inside that loop takes 80 milliseconds.
- A clean recursion that creates 10,000 stack frames is an allocation bottleneck;
an awkward imperative loop is not.
- Refuse micro-optimizations without profile evidence. Do not rewrite string
concatenation, swap array methods, or cache local variables unless the profile
proves that specific operation dominates execution time.

Identify the true bottleneck: algorithmic complexity ($O(N^2) \to O(N)$), I/O
concurrency, database roundtrips / N+1 queries, unindexed scans, or memory
reallocations.

## State the input size the result holds at

Performance is a function of input size and workload scale. A change that helps
at 10,000 items and hurts at 10 items is a trade-off, not an unqualified win.

- State the input size ($N$), dataset volume, concurrency level, or payload
dimension for which the measurement was taken.
- Identify crossover points and trade-offs:
- If a hash map lookup adds constant-time hashing overhead for $N < 20$ but wins
at $N > 100$, state the crossover point.
- If caching reduces CPU latency by 50% at the cost of 200MB memory overhead,
state the memory cost.
- Do not present a specialized high-volume optimization as a universal win if it
regresses small-scale paths.

## Say plainly what to do when something is not measurable

When execution cannot be measured directly in the current environment (e.g.
missing production database, distributed dependencies, inaccessible external
APIs, lack of specialized hardware):

- **Do not guess or assert improvements**: Do not claim an unbenchmarked change
"will be 2x faster".
- **State the boundary plainly**:
`"Cannot measure performance in this environment because <reason>."`
- **Name the decisive command or test**: Provide the exact command, profiling
flag (e.g. `EXPLAIN (ANALYZE, BUFFERS)`, `py-spy`, `perf record`, Go benchmark
command), or telemetry query the user must run to verify the hypothesis.
- **State theoretical complexity as a hypothesis**: Frame algorithmic changes
strictly in big-$O$ terms, explicitly noted as unverified empirically.

## Verify behaviour is preserved

A fast function that returns the wrong answer is broken.

1. Run the test suite before the optimization to confirm all tests pass.
2. Run the test suite after the optimization to prove that behaviour, return
values, error handling, and edge cases were strictly preserved.
3. If an optimization deliberately relaxes guarantees (e.g. eventual consistency,
approximate counting, lossy compression), state the exact semantic change.

## Worked example

Consider optimizing an order summary calculation:

> **1. Baseline Measurement:**
> Command: `python3 -m pytest benchmarks/test_order_perf.py --benchmark-only`
> Result: 142.6 ms per batch ($N = 5,000$ orders).
>
> **2. Profiling / Bottleneck:**
> Profile via `cProfile` showed 118.4 ms (83% of total time) spent in
> `calculate_tax` making individual currency conversion table lookups in a loop
> ($5,000$ queries).
>
> **3. Optimization:**
> Pre-fetched conversion rates into an in-memory dictionary before the loop
> (`src/orders/summary.py:48-56`), eliminating $N$ dictionary reconstructions.
>
> **4. Paired After-Measurement:**
> Command: `python3 -m pytest benchmarks/test_order_perf.py --benchmark-only`
> Result: 18.2 ms per batch ($N = 5,000$ orders).
> Improvement: 124.4 ms reduction (87.2% faster, $7.8\times$ speedup).
>
> **5. Scale & Boundaries:**
> Verified at $N = 100$ (0.8 ms vs 2.9 ms, $3.6\times$ speedup) and $N = 5,000$ ($7.8\times$).
> At $N = 1$, rate pre-fetch adds 0.02 ms overhead (trade-off noted for single-item batches).
>
> **6. Correctness:**
> Ran `pytest tests/test_orders.py` -> 64 passed, 0 failed.

## Output

Structure your performance report as follows:

1. **Baseline Measurement**: Starting metric, environment, input size ($N$), and
the exact command executed.
2. **Profile / Bottleneck Evidence**: What the profiler, flamegraph, or trace
proved was the actual cost.
3. **Applied Optimization**: Concrete change made, with file and line references.
4. **Paired After-Measurement**: Re-run command, new metric, delta, and speedup
or reduction percentage.
5. **Scale & Trade-offs**: Input range where the result holds, crossover points,
and memory or CPU trade-offs.
6. **Correctness Verification**: Test command run to prove behaviour was
preserved.
7. **Unmeasured Boundaries / Next Steps** (if applicable): What could not be
measured and the exact command to measure it.
Loading