Skip to content

Build the standalone native execution path - #243

Merged
luisleo526 merged 12 commits into
mainfrom
refactor/native-market-vertical-20260912
Sep 12, 2026
Merged

luisleo526 merged 12 commits into
mainfrom
refactor/native-market-vertical-20260912

Conversation

@luisleo526

@luisleo526 luisleo526 commented Sep 12, 2026 •

Copy link
Copy Markdown
Collaborator

Add the first standalone native market-order backtest and forward-execution path. C++ strategies configure NativeRunSpec and use NativeStrategyHost to submit, replace, cancel, transact, reduce, and flatten through one physical lot/account book. Native execution has its own typed lifecycle, causal order history, input-mode checks, calendar, and observations; Pine compatibility remains on the legacy/codegen-adapter path.

Includes full supported session/timeframe/calendar behavior, DST and partial warmup handling, a real C ABI batch example, runner replay/outbox verification, authenticated C++ ABI guards, and a complete native C++ guide. Native conditional orders, owner-bound relationships, and the remaining policy migration are later roadmap work.

Validation of candidate ce995a2 / tree 770f7bd:

  • Independent exact-candidate Grok review: GREEN.
  • Fresh Linux Debug ASan/UBSan: 317/317 CTests, no failures or skips; direct WebSocket verification passed.
  • 46 structured native scenarios passed; artifact readback verified the exact commit/tree/manifest. Native host checks: 699; calendar/driver: 1,840; financial/lifecycle: 357; real C ABI example: 421.
  • Real runner recovery retained identical state/action bytes through a failed delivery, retry, and idle replay without duplicate delivery.
  • Full Cloud sweep: 72/72 cases and all 4,190 probes measured. Zero full-grade, raw-trade, substantive-verifier, or non-product-input differences versus the previous accepted refactor result.
  • All nine PR checks and all four post-merge workflows passed.

Squash-merged as 5f3299d90a02e3161659720fb8c2e82f7543ef7a; its tree is identical to the reviewed and measured candidate. The actual formal gate pineforge-pr-gate-7zlr7 remains FAIL target.not-positive only, retained under the user-authorized neutral refactor rule. Baseline promotion skipped the squash, and the campaign baseline remains unchanged.


cfg_lib = root / 'same.json'
native_config(cfg_lib)
p = invoke(base_cmd(no_export, warmup5, ledger, cfg_lib), success=False)
old_spec_symbols = compile_object(
"old_spec_symbol_control", SPEC_SYMBOL_CONTROL, old_spec_include)

old_order_layout = compile_object(
"old_order_layout",
order_layout_source(7),
old_order_include)
old_calendar_layout = compile_object(
"old_calendar_layout",
calendar_layout_source(json.loads((FIXTURE / "calendar-draft-a8e34c/manifest.json").read_text())["layout"]),
old_calendar_include)
old_driver_layout = compile_object(
"old_driver_layout",
driver_layout_source(json.loads((FIXTURE / "driver-08b5c88/manifest.json").read_text())["layout"]),
old_driver_include)
now_order_layout = compile_object("current_order_layout", current_order_layout(), include)
driver_layout_source(json.loads((FIXTURE / "driver-08b5c88/manifest.json").read_text())["layout"]),
old_driver_include)
now_order_layout = compile_object("current_order_layout", current_order_layout(), include)
now_calendar_layout = compile_object("current_calendar_layout", current_calendar_layout(), include)
old_driver_include)
now_order_layout = compile_object("current_order_layout", current_order_layout(), include)
now_calendar_layout = compile_object("current_calendar_layout", current_calendar_layout(), include)
now_driver_layout = compile_object("current_driver_layout", current_driver_layout(), include)
@luisleo526
luisleo526 marked this pull request as ready for review September 12, 2026 09:52
@luisleo526
luisleo526 merged commit 5f3299d into main Sep 12, 2026
9 checks passed
@luisleo526
luisleo526 deleted the refactor/native-market-vertical-20260912 branch September 12, 2026 09:53
luisleo526 added a commit that referenced this pull request Sep 29, 2026
…GINE) (#300)

Audit of every README and doc page of pineforge-engine against engine main
35db01c and the live measurement; each change names its source.

What was wrong, and the source of the truth:

- Parity numbers were the retired 4,190-probe round-39 era (badges, hero,
  scoreboard, lane table, contributing-llm.md, design/progress pages). Now the
  active baseline pineforge-parity-baseline-20260929-engine-35db01c8 (snapshot
  sha256 47cec517..., population cf857209...): 8,006 probes, 7,989 graded =
  7,905 excellent + 84 strong, 17 TradingView-side anomalies excluded,
  18 lanes, 2,759,431 TradingView trades (population tvTradeRows / 2). The
  "2.8M matched" count has no source in the snapshot and is gone. Public
  corpus: 311 excellent + 1 declared anomaly (scripts/check_corpus_parity.sh
  pinned headline; parity step of the XAU-CAL lane job on this tree), and
  anomaly-equity-mirror-strategy-equity-01 is that anomaly (it does not
  match trade-for-trade on the pinned corpus).
- Versions: no 1.0.0, 0.14.0 or rc tag exists (git tag; gh release list).
  CHANGELOG's "## 1.0.0" becomes "## Unreleased" and lists #292-#298, which
  it lacked (squash messages 7f807b9..35db01c); "34 more" PF_API is 39.
  README Releases: v0.13.1 entry added, v0.13.0 line corrected, the merged
  live/abi-v4 "Unreleased" line and the 1.0.0 line replaced. Pages that
  framed 1.0 as shipped now say planned.
- Front door 1 could not run: pineforge-codegen has no CLI (0.10.4 or main),
  PyPI 0.10.4 output does not compile against main (cc log), a plain
  -lpineforge link drops the runtime C ABI (run_strategy.py refuses it), and
  run_strategy.py needs --ohlcv/--no-trim-output without the corpus. The new
  block ran end to end from a clean clone (14 trades on the tutorial feed).
- Prerequisites: floating-point std::to_chars (src/str_utils.cpp:55) needs
  GCC 11 / libc++ 14; ctest --test-dir needs CMake 3.20; examples need
  -DPINEFORGE_BUILD_EXAMPLES=ON; the install also ships libpineforge_kernel.a.
- MCP section: the image, hosted server and npm package run engine 0.13.1 +
  codegen 0.10.4 (pineforge-release v0.1.25 notes); hosted server and npm
  package descriptions corrected from their live/ published sources.
- Benchmarks: every number reproduces (check_provenance 163/163); framing
  fixed (commit and method named, PineTS has had strategy.* since 0.9.17),
  recipes state their prerequisites, two wrong failing-gate cells fixed,
  personal host name scrubbed from 14 Google Benchmark JSONs with the
  manifest sha256 values updated (origin_sha256 keeps the measured file).
- Reference pages: native-engine.md, pine-to-native.md, coverage.md,
  pine_v6_coverage_detail.md, the contract pages, ci.md and the design notes
  described removed members and pre-#253/#254 behaviour, stale counts
  (src 48 -> 53 .cpp, 66 -> 70 indicator classes, 43/33/10 -> 46/35/11
  NativeRunSpec fields, floors 786/298 -> 790/300), wrong anchors, drawings
  as absent, and transpiler status of an old codegen (now "on codegen main,
  not in a release yet" where it differs from v0.10.4).
- Hygiene: internal host names, personal paths, private-repo references and
  campaign tool names marked or removed; the banner image's stale
  "V0.8.0 / 245/246 / 0 engine bugs" strip removed; a dead
  pineforge-workflow-live link replaced by the public pineforge-live repo.
- Two orphan pages, docs/cheatsheet-runtime-and-execution.md and
  docs/tv-parity-probe-spec.md, were wrong in their load-bearing claims
  (17 removed engine members, pointvalue/FX/slippage claims) and nothing
  linked them: deleted.
- Sample outputs were stale: every page and the tutorial README printed the
  tutorial MACD as 49 trades / -190.85; since 650b0cb (in v0.13.1) the report
  counts the range-end close, and every harness prints 50 / +569.97 (runs on
  a clean clone of this branch). examples-python-sweep.md's 10000 capital
  could not pay for its 1-BTC order (every combo 0 trades): 100000, output
  re-measured. examples-multi.md output format matches multi.py; the
  magnifier A/B page shows today's run. The report fields script_tf_ratio
  and needs_aggregation print their contract values again on this base
  (FIX-E1E2, #299; tutorial/run_mtf.py Table A: ratio 1/1/4/16, agg
  0/0/1/1), so the pages state them as plain facts.
- Doxygen: `file:line` in four pages autolinked to a dead "file:line" URL;
  spelled `path:line` now. Build: 0 warnings.

Review fixes (REVIEW-DOCS-ENGINE-final.md, verdict MERGE-WITH-FIXES, of the
first version of this change, 79bd84c1 + 6cb01aa0): its five P2 findings with
the reviewer's fix text, and every P3 row but the last.

P2
1. docs/pages/metrics.md: a timestamped FX curve no longer fails closed for a
   leveraged position across a rate change; that position takes the
   broker-open margin checkpoint at the step bar's open
   (tests/test_affordability_fx.cpp:152-154, :781).
2. docs/pages/report-schema.md: the #254 caveat sat inside the table, so the
   bar_magnifier_enabled row rendered as text; it now follows the table.
3. CONTRIBUTING.md, docs/pages/getting-started.md: compiler minimums are the
   ones README.md states (floating-point std::to_chars, src/str_utils.cpp:55:
   libstdc++ from GCC 11, libc++ 14, macOS 13.3; g++-9 and g++-10 fail).
4. docker/README.md: the sample is the published image's run (50 trades,
   17/33, +569.97, trade 1 entering 1777488300000).
5. docs/pages/mtf.md: expected Table A says main prints 0 in ratio and agg?
   until the #254 regression is fixed.

P3
- configuration.md, CHANGELOG.md, coverage.md: v5 scripts do not "keep" host
  defaults (both codegens refuse //@Version=5); code from an earlier codegen
  and a hand-built PineStrategyConfig do.
- streaming.md: "Every strategy_stream_* call above" (create/free return no
  status).
- examples-multi.md: a module also exports std::piecewise_construct.
- integration-cmake.md: EXACT 0.13.1 also matches a git-checkout build of
  main; the ABI promise starts "From 1.0.0".
- ffi-python.md, tutorial-macd.md: the mirror covers the report structs, not
  every struct (ffi-python.md:15-17).
- configuration.md: tracing costs one branch per traced call when disabled
  (report-schema.md:180), not zero.
- benchmarks/results/raw/README.md: "Two edits were made" (host_name was
  rewritten in every Google Benchmark file); benchmarks/README.md: the
  maintainer fetch runs with the campaign environment sourced. Wording only.
- AGENTS.md: the configure line exports compile commands, which the receipt
  check needs; AGENTS.md and docs/ci.md: promote-baseline.yml is installed
  from the campaign copy, not unchanged (31 lines differ).
- docs/ci.md: the debug profile also skips the twin-parity guard
  (TWIN_PARITY_PROFILES).
- mtf.md, timeframes.md: "reported for an engine fix" dropped (no public
  issue).
- native-feature-parity.md: the C API "landed on main", not "shipped".
- tutorial/README.md: Table B's script_tf_ratio prints 0 on main until the
  #254 regression is fixed.

Not changed here: benchmarks/results/summary.md and trade_comparison.md keep
the first version's text (hand edits of generated output; fixing the generator,
benchmarks/compare.py:361 and scripts/verify_corpus.py:1403, is an
engine-lane follow-up). No benchmark number or results file changes beyond
the raw/README.md and benchmarks/README.md wording above: BENCH-REFRESH is
re-measuring the benchmark.

Rebased onto 4dc496d (FIX-E1E2, #299), squashed to one commit. docs/ci.md
states the new floors: PR registration 773 (debug, sanitizers) and 782
(native), full-run release 792 and kernel 300 (scripts/ci_verify.py:320,
:594, :638). The one rebase conflict, a native-refactor-progress.md row,
keeps main's re-anchored lines and this change's host-name removal.

Sentences this commit deletes or rewrites past recognition, named by the
commit that introduced them (scripts/check_doc_reverts.py):
- a4ab775 (docs: source-verified runtime-overrides + order-execution cheatsheet): cheatsheet-runtime-and-execution.md (142)
- 480fd19 (Production-readiness probes + engine state-reset / finite-fill fixes (): tv-parity-probe-spec.md (61)
- 28ad088 (docs: add Part 3 (cross-cutting behaviors) to engine cheatsheet): cheatsheet-runtime-and-execution.md (60)
- a8f84b9 (docs(api): OpenCV-style C ABI reference site + CF Pages deploy): abi-stability.md (21), tutorial-macd.md (3), examples-magnifier.md (2), examples-multi.md (2), getting-started.md (2), index.md (2), report-schema.md (2), examples-rust.md (1), ffi-python.md (1), install.md (1), lifecycle.md (1)
- cc4a56e (docs: add Pine v6 runtime coverage report (docs/coverage.md)): coverage.md (37)
- 24dcc53 (docs: reconcile Pine v6 audit + coverage docs with the 2026-06-10 audi): pine_v6_coverage_detail.md (29)
- d093560 (INT26: wave H, the late picks and the corpus re-pin to b40aa8e (1.0.0-): native-refactor-progress.md (13), install.md (3), pine-to-native.md (2), CONTRIBUTING.md (1)
- 73817c1 (Lower generated Pine execution onto the native kernel and retire the l): cheatsheet-runtime-and-execution.md (8), README.md (3), tv-parity-probe-spec.md (2), reservation-expansion-candidate.md (1)
- b1fbf0b (docs: detailed Pine v6 coverage report (per-identifier audit)): pine_v6_coverage_detail.md (14)
- 622d653 (docs(README): rewrite for the round-11 engine — brand-first, OSS-first): README.md (12)
- 46b2ced (docs: ABI v4 live surface; cheatsheet corrections (trail_price is read): cheatsheet-runtime-and-execution.md (7), README.md (3), live-surface.md (2)
- c45cf5a (refactor: move retained-parent priority into Pine compatibility): README.md (7), pine-order-priority-boundary.md (4)
- c1084ff (docs: record round 39 fractional price admission parity): README.md (10)
- f268342 (feat: add historical account FX lifecycle (#125)): cheatsheet-runtime-and-execution.md (10)
- af4c688 (Document realtime streaming lifecycle): cheatsheet-runtime-and-execution.md (5), coverage.md (3), examples-python-sweep.md (1), tutorial-macd.md (1)
- 9507d8f (refactor(corpus): pull equity-mirror into corpus/parity-anomalies/): coverage.md (10)
- 76482e6 (fix(runtime): align TA coverage and corpus replay): coverage.md (9)
- cc0b22d (Refactor reservation growth and Pine market instruction ownership): fill-model.md (7), frozen-market-instruction.md (2)
- d5a5eaf (Align lifecycle and admission documentation with final v11 model): market-admission.md (9)
- a6b0b70 (docs(audit): exhaustive Pine v6 coverage audit — 38 critical, ~62 mino): pine_v6_audit_master.md (7)
- 1e360bd (INT27: wave I -- magnifier on TradingView's intrabars, recalculation f): cheatsheet-runtime-and-execution.md (5), magnifier.md (1)
- 1dd5430 (Add the native feature-parity design: gaps, kernel-vs-adapter rulings,): design/native-feature-parity.md (6)
- 74491fa (INT28: wave-J engine integration (TVDEF-DROPS, W3/W3B/W4/W5/W6/W6B/W8,): cheatsheet-runtime-and-execution.md (4), README.md (1)
- 7ad5d38 (docs: refresh parity scoreboard through round 23): README.md (5)
- 7f807b9 (Another symbol's bars reach a run as a feed: kernel instrument feeds +): coverage.md (4), mtf.md (1)
- 3b13183 (Refactor order admission and trigger lifecycle into generic submodels): exit-leg-lifecycle-reflection.md (5)
- e244d65 (docs(coverage): update pine_v6_coverage_detail.md for post-sprint stat): pine_v6_coverage_detail.md (5)
- f864be5 (refactor: isolate Pine cap policy from native order state): cheatsheet-runtime-and-execution.md (2), pine-intraday-cap-boundary.md (2)
- e7cdf05 (Add native execution terms: host-resolved sizing, exact reversal, prec): abi-stability.md (2), native-engine.md (2)
- b4dd9a3 (docs(mtf): add multi-timeframe tutorial + Doxygen page): mtf.md (4)
- 5c2be9d (docs: record round 38 whole-lot budget parity results): README.md (3)
- 6bfe983 (docs: align round 39 lane table and diagnostic evidence): README.md (3)
- 471a113 (Docs: the README carries the 200-strategy refresh -- tiers per engine ): README.md (3)
- 8df08f2 (fix(engine): make dead strategy.close semantically inert (#143)): cheatsheet-runtime-and-execution.md (3)
- d3cca9e (Docs: correct migration semantics and harden publication guards): cheatsheet-runtime-and-execution.md (2), pine-to-native.md (1)
- 52f4dde (Preserve complete mirror context and earlier standalone ABI controls): fill-model.md (3)
- 4fb9db3 (Docs: CONTRIBUTING.md rewritten for someone who has never seen this re): CONTRIBUTING.md (2)
- 2ab2bfb (docs(README): round-11 closed-test results and a full factual refresh): README.md (2)
- 0501468 (docs: task-13 fix round 1 — v4 report mirrors (schema, Rust), probe-su): README.md (1), cheatsheet-runtime-and-execution.md (1)
- cfbe677 (Docs: the session flags end a session where TradingView ends it (R5 la): cheatsheet-runtime-and-execution.md (2)
- 6aa1d13 (runtime: wire syminfo.pointvalue through all money paths; hold ScopedT): coverage.md (2)
- 8b7965a (Docs: onboarding written for the contributor who is a language model (): contributing-llm.md (2)
- 16f3652 (Docs: the two site front doors re-derived against HEAD (R5 lane L14-B,): index.md (2)
- 04b0588 (Add native resolved execution settlement and preserve paid entry costs): metrics.md (2)
- 149f77c (Replace seven pending-order flags with quantity and event facts): quantity-intent.md (2)
- 962960b (Session flags match TradingView on multi-window and extended-hours cha): pine_v6_coverage_detail.md (2)
- 5215291 (Merge branch 'round5/bar-indexed-extremes-ring' into round5/candidate): pine_v6_coverage_detail.md (2)
- b1b27f0 (Initial commit: PineForge runtime v0.1.0): CONTRIBUTING.md (1)
- bfdbe96 (docs: refresh parity scoreboard through round 24): README.md (1)
- cda51d5 (Record the round 35 parity scoreboard and raw trade gains): README.md (1)
- 0d76a09 (Wave G: no ordinary quantity request stops a run, cancelled exits stay): adr/0001-kernel-adapter-boundary.md (1)
- 0983770 (Unify remaining fill paths under native settlement (#247)): ci.md (1)
- 50339d0 (Docs: the §1 inventory carries its closure markers, and a gate keeps t): design/native-feature-parity.md (1)
- 8b3b481 (docs+fix: org rename to pineforge-4pass; AI capability-gap note in REA): examples-multi.md (1)
- c28c2db (Derive placement state from original admission observations): fill-model.md (1)
- 208f00b (docs: refresh ctest count + run_strategy usage example): install.md (1)
- 0c174d9 (Docs: the arm options, the adoption table and the grid re-lowering wai): native-engine.md (1)
- 4b05b62 (Adapter re-lowering: the TradingView margin call runs through the kern): native-engine.md (1)
- 8e558eb (Say what the per-bar broker hash is for, where its API is documented (): native-engine.md (1)
- 5f3299d (Build the standalone native execution path (#243)): native-engine.md (1)
- b18f75e (Native sizing bases and fractional reduces (R5 lane L3)): native-engine.md (1)
- 0ef00da (The pages say a working row tells an absent arm from an arm at 0.0 (R5): native-engine.md (1)
- 7328f4d (Docs: the migration page is a complete map, and a gate holds it comple): pine-to-native.md (1)
- ce64f06 (docs(audit): update Pine v6 master audit — mark 20 resolved critical i): pine_v6_audit_master.md (1)
- a19f5a8 (docs: repoint codegen references to pineforge-codegen-oss; purge old f): pine_v6_audit_master.md (1)
- 2db58d9 (runtime: reconcile per-trade MFE/MAE with TV excursion semantics; gate): pine_v6_audit_master.md (1)

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant