Skip to content

feat(evaluation): add full-A strategy and ML OOS ablation - #121

Merged
Atlas2005 merged 1 commit into
mainfrom
pr121-full-a-dynamic-strategy-ai-ablation
Jul 12, 2026
Merged

feat(evaluation): add full-A strategy and ML OOS ablation#121
Atlas2005 merged 1 commit into
mainfrom
pr121-full-a-dynamic-strategy-ai-ablation

Conversation

@Atlas2005

Copy link
Copy Markdown
Owner

Summary

Adds the PR #121 Full-A strategy and ML out-of-sample ablation workflow for the offline A-share snapshot.

This PR:

  • compares fixed strategy candidates, LightGBM ranking, and a validation-only dynamic selector
  • uses common chronological train/validation/test folds
  • reuses the existing account-aware execution, fill, lot-size, capital, and transaction-cost paths
  • validates cost sensitivity at 1.0x, 1.5x, and 2.0x
  • preserves purge/embargo based on the target horizon
  • records compact ML ranking authenticity evidence
  • remains manual-only and offline after snapshot creation
  • performs no broker, DeepSeek, credential, or live-trading access

Full-A validation

Snapshot:

  • provider: all_a_share_snapshot
  • snapshot digest: ca0448aefb17cd2990fd2a095835d6dc44e25046c259748545051516bf303615
  • requested symbols: 5,481
  • valid symbols: 5,481
  • skipped symbols: 0
  • common folds: 5
  • candidates: 11
  • cost-sensitivity rows: 33
  • external calls: false

Final V3 run:

  • process return code: 0
  • timed out: false
  • run status: completed
  • successful ML folds: 5
  • authentic ML folds: 5

Main result

Recommended candidate:

equal_weight_baseline

Metrics:

  • compounded return: 36.4493%
  • benchmark return: 31.8605%
  • compounded excess return: 2.8476%
  • positive-return fold ratio: 80%
  • positive-excess fold ratio: 60%
  • maximum drawdown: -8.9672%
  • filled trades: 50
  • rejected trades: 0

ML result

The corrected LightGBM path genuinely controlled order selection in all five folds.

Evidence includes:

  • nonconstant prediction distributions
  • (test_start, symbol) prediction association
  • ML-selected symbols passed into the existing executor
  • actual filled symbols matching ML-selected symbols
  • no silent baseline fallback
  • common test ranges and benchmark values matching fixed-rule evaluation

Final LightGBM metrics:

  • compounded return: 8.2139%
  • benchmark return: 31.8605%
  • compounded excess return: -19.5798%
  • positive-return fold ratio: 40%
  • positive-excess fold ratio: 20%
  • maximum drawdown: -7.8151%

This means the ML execution path is valid, but the current feature, label, model, and portfolio mapping do not demonstrate incremental value.

Dynamic selector result

  • compounded return: 16.8098%
  • compounded excess return: -13.9608%
  • uses test outcomes: false
  • decisions are based only on prior validation evidence

The selector did not outperform the equal-weight baseline.

Integrity fixes included

  • optimized buy-and-hold benchmark construction and added run-scoped caching
  • aligned ML and fixed-rule test boundaries and benchmark basis
  • propagated actual ML filled/rejected trades, turnover, costs, and execution windows
  • fixed ML prediction key and rebalance-date handoff
  • rejected missing, nonfinite, constant, or tied ML ranking inputs instead of silently falling back to canonical ordering
  • added compact ranking evidence and progress reporting
  • corrected disabled dynamic-selector CLI output

Validation

Safety and scope

  • no commit generated by automation
  • no external API calls during the Full-A validation
  • no DeepSeek call
  • no broker access
  • no credentials accessed
  • no live or simulated broker orders
  • no branch switch

@Atlas2005
Atlas2005 merged commit b908efa into main Jul 12, 2026
1 check passed
@Atlas2005
Atlas2005 deleted the pr121-full-a-dynamic-strategy-ai-ablation branch July 12, 2026 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant