From ea0e0623ec9862d1595879a10e8822a0993a70c4 Mon Sep 17 00:00:00 2001 From: Yaniv Bernhard Date: Tue, 8 Sep 2026 23:00:30 +0300 Subject: [PATCH] feat: F&G column, the stock's fear-and-greed reading on every report row The composite (RSI 14, MACD-histogram percentile within the trailing year, Bollinger %B, equal weights, 0-100) moves from the backtest into scan.py as fear_greed() so the report and the replay share one implementation. scan_symbol computes one reading per symbol at the last close and puts it on each of the symbol's signals; signals.json gains fear_greed, the report gains the F&G column and a footer sentence explaining the zones. Information only; the stretch gate it suggests is #113. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 9 ++ README.md | 2 +- .../wiki/01-Architecture-and-Data-Pipeline.md | 3 +- docs/wiki/03-Configuration-and-Tuning.md | 4 +- scan.py | 93 ++++++++++++++++++- test_backtest.py | 20 ++-- test_pipeline.py | 5 +- tools/backtest.py | 78 +--------------- 8 files changed, 125 insertions(+), 89 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 99f666f..ef25b8a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,15 @@ All notable changes to this project are documented here. Format follows ## [Unreleased] +### Added (report: the F&G column) +- Every report row carries `F&G`, the stock's own fear-and-greed reading at + the last close, 0 to 100, one value per symbol; `fear_greed` in + `signals.json`. The composite (`scan.fear_greed`, RSI 14, MACD-histogram + percentile within the trailing year, Bollinger %B) moved from the backtest + into the scanner so the report and the replay share one implementation. + The footer explains the zones. Information only; the stretch gate it + suggests is #113. + ### Added (per-ticker fear and greed, 2026-09-08) - Replay rows and the outcome-by-feature table gain a per-ticker fear-and-greed reading: the equal-weight 0-100 composite of RSI 14, the MACD (12, 26, 9) diff --git a/README.md b/README.md index bde8286..8c22640 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,7 @@ Data errors: 1 | DG | Cup & Handle | WATCHLIST | FAILED | 134.13 | 116.6 | 161.68 | close 116.20 on 2026-09-03 at or below stop 116.6 | ``` -Max buy is the open above which the setup no longer qualifies: the trigger plus 5 %, or lower where a fill would already carry 1.5× the planned risk (CL above: a Wolfe stop sits 1.6 % under the entry, so the chase allowance is 0.8 %, not 5 %). R:R is the reward per unit of planned risk, `(target − entry) / (entry − stop)`; it shrinks with every session the entry drifts above the trigger, which is why HAL, four sessions past its breakout, shows 1.09. The last table explains every row of the previous report that is gone today (the two rows above are illustrative). `output/signals.json` carries the same rows as records plus a `meta` block (`last_bar`, per-symbol bar histogram, effective breakout-age limits, the market context: SPY regime, VIX and breadth, informational only) and the `closed` list. The schema is documented in the wiki. +Max buy is the open above which the setup no longer qualifies: the trigger plus 5 %, or lower where a fill would already carry 1.5× the planned risk (CL above: a Wolfe stop sits 1.6 % under the entry, so the chase allowance is 0.8 %, not 5 %). R:R is the reward per unit of planned risk, `(target − entry) / (entry − stop)`; it shrinks with every session the entry drifts above the trigger, which is why HAL, four sessions past its breakout, shows 1.09. F&G (added after this sample) is the stock's own fear-and-greed reading at the last close, 0 to 100, the average of RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B; above 80 the stock is stretched, and over ten years of replay such breakouts paid least. Information only. The last table explains every row of the previous report that is gone today (the two rows above are illustrative). `output/signals.json` carries the same rows as records plus a `meta` block (`last_bar`, per-symbol bar histogram, effective breakout-age limits, the market context: SPY regime, VIX and breadth, informational only) and the `closed` list. The schema is documented in the wiki. ## Documentation diff --git a/docs/wiki/01-Architecture-and-Data-Pipeline.md b/docs/wiki/01-Architecture-and-Data-Pipeline.md index 9ed9457..32d1e44 100644 --- a/docs/wiki/01-Architecture-and-Data-Pipeline.md +++ b/docs/wiki/01-Architecture-and-Data-Pipeline.md @@ -101,10 +101,11 @@ Yahoo publishes the newest daily bar per symbol at different times (volume first | `notes` | anchor dates and levels used by the detector (parseable, see tests); "breakout without volume (x.xx×)" when a breakout was watch-listed for lack of volume | | `max_buy` | the open above which the setup no longer qualifies: the lower of trigger × 1.05 (the runaway rule applied to the open) and `stop + MAX_BUY_RISK_MULT × (entry − stop)`, the fill at which the risk reaches 1.5× the planned risk. The second cap binds for tight structural stops (Wolfe point 5, shallow handles); the first for wide ones (H&S shoulders) | | `reward_risk` | `(target − entry) / (entry − stop)` at the reported entry, 2 decimals; `null` without a target. Rows below `MIN_REWARD_RISK` (when set) are not reported. It falls as the entry drifts above the trigger, so a late confirmed row can show a poor R:R on an otherwise clean pattern | +| `fear_greed` | the stock's own fear-and-greed reading at `last_bar`, 0 to 100 (`fear_greed`): the equal-weight average of RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B, one reading per symbol shared by its rows. Above 80 the stock is stretched, below 20 washed out. Informational: no rule reads it; the ten-year evidence is on the tuning page | `meta.market` is the market context at `last_bar`: the index ETF's close, its distance from its SMA200 and its SMA50's distance from the SMA200 in percent, the regime those name, the VIX close, and the breadth over `breadth_symbols`; each `null` where unavailable. It is informational and reproduced on every backtest row. -Signals are sorted `CONFIRMED` first, then by score descending. `output/report.md` renders the same rows as two Markdown tables (Ticker, Pattern, Entry, Max buy, Stop, Risk %, Target, R:R, Score, Age, Vol×, Trend, Details) with a header stating the scanned bar, effective age limits, skipped/lagging counts and data errors. **Age** is `bars_since_break / limit`, e.g. `1/3` for a cup that broke out yesterday and will be dropped after two more sessions; `-` for watchlist rows. +Signals are sorted `CONFIRMED` first, then by score descending. `output/report.md` renders the same rows as two Markdown tables (Ticker, Pattern, Entry, Max buy, Stop, Risk %, Target, R:R, Score, Age, Vol×, F&G, Trend, Details) with a header stating the scanned bar, effective age limits, skipped/lagging counts and data errors. **Age** is `bars_since_break / limit`, e.g. `1/3` for a cup that broke out yesterday and will be dropped after two more sessions; `-` for watchlist rows. **Closed since the last report.** A stateless scan only knows what qualifies today, so each run also reads the previous committed `signals.json` (the nightly job has it in the checkout) and explains every row that disappeared, using the bars since that row's `last_date`. The list is `closed` in `signals.json` (ticker, pattern, was, since, entry, stop, target, outcome, detail) and a third table in the report; `meta.previous_run` names the report it was compared with. diff --git a/docs/wiki/03-Configuration-and-Tuning.md b/docs/wiki/03-Configuration-and-Tuning.md index 76c02fe..724743e 100644 --- a/docs/wiki/03-Configuration-and-Tuning.md +++ b/docs/wiki/03-Configuration-and-Tuning.md @@ -91,7 +91,7 @@ Pooled, weighting each year by its traded signals: 1629 traded signals, +0.23 R, What the ten years overturned from the two-year window, each a lesson in what one exploratory pass over two bull years produces: fresh breakouts are not better (age 0 at first report +0.10 R against +0.42 for age 4 to 8; #98 closed), reward:risk above 4 is not bad (+0.45 R with a 16 % hit rate; #100 closed), a VIX below 15 is not bad (+0.21 R pooled, and 2017 at a VIX near 10 was the best year; #101 rewritten), the SMA band U-shape is one-sided (#99 rewritten), and the Wolfe verdict on 12 out-of-sample trades was noise (#102 closed as held). What survived: the tuned profile's expectancy, the score's role as a gate, and the reversal-after-washout context. -**Open questions:** #99 and #101 (the deep-down-trend and bear-regime contexts, information first, no rule), #108 (the drawdowns), #109 (cups), and #97 (a per-signal probability model, whose first revisit condition the ten years now meet). +**Open questions:** #99 and #101 (the deep-down-trend and bear-regime contexts, information first, no rule), #108 (the drawdowns), #109 (cups), #113 (the Bollinger stretch gate), and #97 (a per-signal probability model, whose first revisit condition the ten years now meet). ### A second review, tested (2026-09-08) @@ -126,7 +126,7 @@ A trader's suggestion: read a per-ticker fear-and-greed indicator, as the Tradin A confirmed breakout is almost never fearful by construction (one row below 20 in ten years), so at the scan day the scale runs from fear to extreme greed, and the greedier the stock at its breakout, the lower the mean R; the higher hit rate of the greediest bucket does not compensate, because its wins are smaller. Bases that formed in fear paid about twice what bases formed in greed did. Of the components, the stretch measures carry the effect and momentum does not: RSI 30-50 at the breakout ran +0.55 R on 230 signals against +0.18 for 50-70 and +0.12 above 70, the lower bucket ahead in 8 of 11 years; Bollinger %B in the lower half ran +0.53 on 185, the upper half +0.26 on 991 and a close above the upper band +0.04 on 453, the upper half ahead of the above-band bucket in 10 of 11 years (the exception, 2017, a tie at +0.44 against +0.45); the MACD percentile ran +0.18 to +0.26 across its buckets with the strongest momentum slightly best. -The one candidate rule this produces is the Bollinger stretch: a breakout bar that closes above its upper band. Leaving those 453 signals out would keep 1179 signals at +0.30 R against 1632 at +0.22, at a cost of 16 R of the ten-year total of 367. Under the protocol it remains a hypothesis: the gate has to be replayed as a rule on its own, so that its effect on the drawdown and per pattern is measured, and the reading belongs in the report as information first. +The one candidate rule this produces is the Bollinger stretch: a breakout bar that closes above its upper band. Leaving those 453 signals out would keep 1179 signals at +0.30 R against 1632 at +0.22, at a cost of 16 R of the ten-year total of 367. Under the protocol it remains a hypothesis: the gate has to be replayed as a rule on its own, so that its effect on the drawdown and per pattern is measured (#113). The reading itself is on every report row as the `F&G` column, one value per symbol at the last close, information only. ### Market context (2026-09-08) diff --git a/scan.py b/scan.py index c79ddac..2af0728 100755 --- a/scan.py +++ b/scan.py @@ -109,6 +109,14 @@ # is recorded on every report and backtest row to be tested, not gated on (issues #93, #101). MARKET_INDEX = "SPY" MARKET_VOL = "^VIX" +# Per-ticker fear and greed, informational (no rule reads it): the equal-weight 0-100 composite of +# RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B that the +# TradingView community indicators of that name share, at the last close (``fear_greed``). Ten years +# of replay (docs/wiki/03, issue #113): the greedier the stock at its breakout the lower the mean R, +# +0.45 in the neutral zone against +0.12 above 80, and a close above the upper Bollinger band ran +# +0.04 R on 453 signals. Shown on every report row so the reader sees the stretch; not gated on. +FG_RSI_LEN, FG_BB_LEN, FG_MACD = 14, 20, (12, 26, 9) +FG_LOOKBACK = 250 # bars for the MACD histogram's percentile rank # --------------------------------------------------------------------------- # # Pattern rules. The values below are the "spec" profile (docs/wiki/02 and the @@ -317,6 +325,8 @@ class Signal: :param notes: Free-text details (pattern anchor dates and levels). :param max_buy: Highest open worth filling (see :func:`max_buy_level`). :param reward_risk: ``(target - entry) / (entry - stop)`` or None (see :func:`reward_risk`). + :param fear_greed: The stock's fear-and-greed composite at the last close, 0-100 + (see :func:`fear_greed`); informational. """ ticker: str @@ -335,6 +345,7 @@ class Signal: notes: str = "" max_buy: Optional[float] = None # above this at the open, do not chase (max_buy_level) reward_risk: Optional[float] = None # reward per unit of planned risk, None without a target + fear_greed: Optional[float] = None # the stock's fear-and-greed reading at the last close, 0-100 (fear_greed) # --------------------------------------------------------------------------- # @@ -813,6 +824,73 @@ def atr(df: pd.DataFrame, n: int = ATR_LEN) -> pd.Series: return tr.rolling(n, min_periods=1).mean() +def _rsi(close: np.ndarray, n: int = FG_RSI_LEN) -> Optional[float]: + """Wilder's RSI of the last bar (an ``n``-bar simple seed, then ``(n - 1) / n`` smoothing). + + :returns: 0-100, rounded to 2 dp; 50 on a series that never moved; ``None`` with fewer than ``n + 1`` bars. + """ + if len(close) < n + 1: + return None + delta = np.diff(np.asarray(close, dtype=float)) + gains, losses = np.clip(delta, 0.0, None), np.clip(-delta, 0.0, None) + avg_gain, avg_loss = float(gains[:n].mean()), float(losses[:n].mean()) + for gain, loss in zip(gains[n:], losses[n:]): + avg_gain = (avg_gain * (n - 1) + gain) / n + avg_loss = (avg_loss * (n - 1) + loss) / n + if avg_gain == 0 and avg_loss == 0: + return 50.0 + if avg_loss == 0: + return 100.0 + return round(100 - 100 / (1 + avg_gain / avg_loss), 2) + + +def fear_greed(close: np.ndarray) -> Dict[str, Optional[float]]: + """A per-ticker fear-and-greed reading of the last bar: 0 = extreme fear, 100 = extreme greed. + + The TradingView community indicators of that name are composites of + standard oscillators computed on the symbol itself. This is a documented, + equal-weight version of the three price-based components they share: + + * ``rsi``: RSI 14 (:func:`_rsi`); + * ``macd_pct``: the MACD (12, 26, 9) histogram as its mid-rank percentile + within the trailing ``FG_LOOKBACK`` bars, so 50 means an average reading + for this stock and 99 its most bullish momentum of the year; + * ``bb_pctb``: Bollinger %B over 20 bars and 2 standard deviations, i.e. the + close's position between the bands (below 0 or above 1 = outside them), + unclipped and rounded to 3 dp; + * ``score``: the mean of RSI, the MACD percentile and %B clipped to 0-100, + over the components the history allows. + + Above 80 the TradingView scripts call it extreme greed, below 20 extreme + fear. Reported per row; no rule reads it (docs/wiki/03). + + :returns: The four keys, ``None`` where the history is too short. + + Complexity: O(bars). + """ + close = np.asarray(close, dtype=float) + out: Dict[str, Optional[float]] = {"rsi": _rsi(close), "macd_pct": None, "bb_pctb": None, "score": None} + fast, slow, signal = FG_MACD + if len(close) >= slow + signal: + s = pd.Series(close) + macd = s.ewm(span=fast, adjust=False).mean() - s.ewm(span=slow, adjust=False).mean() + hist = (macd - macd.ewm(span=signal, adjust=False).mean()).to_numpy() + window = hist[-FG_LOOKBACK:] + rank = ((window < hist[-1]).sum() + 0.5 * (window == hist[-1]).sum()) / len(window) + out["macd_pct"] = round(float(rank * 100), 1) + if len(close) >= FG_BB_LEN: + w = close[-FG_BB_LEN:] + mid, sd = float(w.mean()), float(w.std()) + if sd > 0: + out["bb_pctb"] = round((close[-1] - (mid - 2 * sd)) / (4 * sd), 3) + parts = [v for v in (out["rsi"], out["macd_pct"], + None if out["bb_pctb"] is None else min(max(out["bb_pctb"] * 100, 0.0), 100.0)) + if v is not None] + if parts: + out["score"] = round(sum(parts) / len(parts), 1) + return out + + def find_pivots(high: np.ndarray, low: np.ndarray, order: int = PIVOT_ORDER ) -> Tuple[List[int], List[int]]: """Fractal swing detection. @@ -1606,6 +1684,11 @@ def scan_symbol(sym: str, df: pd.DataFrame, detectors: Optional[Sequence[Callabl out.extend(fn(df, sym)) except Exception as exc: # one bad ticker must not abort the scan log.exception("%s failed on %s: %s", fn.__name__, sym, exc) + if out: + # One reading per symbol at the last close, shared by all of its rows; informational (FG_* constants). + fg = fear_greed(df["Close"].to_numpy(dtype=float))["score"] + for s in out: + s.fear_greed = fg return out @@ -1735,8 +1818,8 @@ def render_markdown(signals: List[Signal], meta: Mapping[str, Any], lines.append("") continue lines.append("| Ticker | Pattern | Entry | Max buy | Stop | Risk % | Target | R:R | Score | Age | Vol× | " - "Trend | Details |") - lines.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|") + "F&G | Trend | Details |") + lines.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|---|") for s in rows: # Age = bars since the breakout close / the pattern's limit, so a reader # can see whether a confirmed row is fresh (0/3) or about to expire (3/3). @@ -1745,7 +1828,8 @@ def render_markdown(signals: List[Signal], meta: Mapping[str, Any], lines.append(f"| {s.ticker} | {s.pattern} | {s.entry} | {s.max_buy if s.max_buy else '-'} | {s.stop} | " f"{s.risk_pct} | {s.target if s.target else '-'} | " f"{s.reward_risk if s.reward_risk is not None else '-'} | {s.score} | {age} | " - f"{s.volume_ratio if s.volume_ratio else '-'} | {s.trend} | {s.notes} |") + f"{s.volume_ratio if s.volume_ratio else '-'} | " + f"{s.fear_greed if s.fear_greed is not None else '-'} | {s.trend} | {s.notes} |") lines.append("") if closed is not None: since = meta.get("previous_run") or "the last report" @@ -1773,6 +1857,9 @@ def render_markdown(signals: List[Signal], meta: Mapping[str, Any], lines.append(f"_Max buy = {max_buy_rule}: if the open is above it the setup no longer qualifies. " f"R:R = (target - entry) / (entry - stop) at the reported entry; it shrinks with every " f"session the entry drifts above the trigger. " + f"F&G = the stock's own fear-and-greed reading at the last close, 0 to 100 (RSI {FG_RSI_LEN}, " + f"MACD-histogram percentile and Bollinger %B averaged): above 80 the stock is stretched and such " + f"breakouts replayed worst, below 20 it is washed out; information only, no rule uses it. " f"Age = bars since the breakout close / the limit after which the row is dropped " f"(0 = broke out on the last bar). Heuristic scan, not advice. " f"Entry = trigger level, or the breakout close when it " diff --git a/test_backtest.py b/test_backtest.py index 1a932c8..ee4926b 100644 --- a/test_backtest.py +++ b/test_backtest.py @@ -117,20 +117,20 @@ def test_row_features_match_hand_computation(mini_universe): def test_fear_greed_components_and_score(): - rising = bt.fear_greed(100 * 1.01 ** np.arange(300)) # accelerating rise: everything stretched up + rising = scan.fear_greed(100 * 1.01 ** np.arange(300)) # accelerating rise: everything stretched up assert rising["rsi"] == 100.0 and rising["macd_pct"] > 90 and rising["bb_pctb"] > 0.5 and rising["score"] > 80 - falling = bt.fear_greed(100 * 0.99 ** np.arange(300)) + falling = scan.fear_greed(100 * 0.99 ** np.arange(300)) assert falling["rsi"] == 0.0 and falling["macd_pct"] < 10 and falling["bb_pctb"] < 0.5 and falling["score"] < 20 - flat = bt.fear_greed(np.full(300, 50.0)) # never moved: neutral, and no bands + flat = scan.fear_greed(np.full(300, 50.0)) # never moved: neutral, and no bands assert flat == {"rsi": 50.0, "macd_pct": 50.0, "bb_pctb": None, "score": 50.0} - short = bt.fear_greed(np.arange(10, dtype=float)) # too short for every component + short = scan.fear_greed(np.arange(10, dtype=float)) # too short for every component assert short == {"rsi": None, "macd_pct": None, "bb_pctb": None, "score": None} # RSI by hand on a 15-bar series: gains 1 on ten bars, losses 1 on four -> avg gain 10/14, avg loss 4/14 -> RS 2.5. steps = np.array([1, 1, 1, -1, 1, 1, -1, 1, 1, 1, -1, 1, 1, -1], dtype=float) - assert bt._rsi(np.concatenate([[100.0], 100 + np.cumsum(steps)])) == round(100 - 100 / 3.5, 2) + assert scan._rsi(np.concatenate([[100.0], 100 + np.cumsum(steps)])) == round(100 - 100 / 3.5, 2) # %B is the close's position between the bands: outside them beyond 0 or 1. spike = np.concatenate([np.full(19, 100.0), [110.0]]) - fg = bt.fear_greed(np.concatenate([np.full(30, 100.0), spike])) + fg = scan.fear_greed(np.concatenate([np.full(30, 100.0), spike])) assert fg["bb_pctb"] > 1.0 and fg["score"] is not None @@ -139,14 +139,18 @@ def test_row_features_fear_greed_at_scan_day_and_base(mini_universe): (s,) = scan.detect_cup_and_handle(cup, "CUP") f = bt.row_features(cup, s, float(scan.atr(cup).iloc[-1])) close = cup["Close"].to_numpy() - fg = bt.fear_greed(close) + fg = scan.fear_greed(close) assert (f["fg_score"], f["fg_rsi"], f["fg_macd_pct"], f["fg_bb_pctb"]) == ( fg["score"], fg["rsi"], fg["macd_pct"], fg["bb_pctb"]) + assert s.fear_greed is None # a detector alone does not set it ... + (sig,) = scan.scan_symbol("CUP", cup, detectors=(scan.detect_cup_and_handle,)) + assert sig.fear_greed == fg["score"] # ... scan_symbol does, once per symbol anchor = cup.index.get_loc(pd.Timestamp(re.search(r"handle low (\S+)", s.notes)[1])) - assert f["fg_base"] == bt.fear_greed(close[:anchor + 1])["score"] + assert f["fg_base"] == scan.fear_greed(close[:anchor + 1])["score"] assert 0 <= f["fg_base"] < f["fg_score"] <= 100 # the base is fearful, the breakout greedy rows = bt.walk_forward(mini_universe, days=5, horizon=10) assert all(r["fg_score"] is not None and r["fg_base"] is not None for r in rows) + assert all(r["fear_greed"] == r["fg_score"] for r in rows) # the report's reading equals the feature md = bt.render(rows, bt.report_sections(rows, 5, 10), 5, 10) assert "| fear and greed at the scan day |" in md and "| fear and greed at the pattern's last low |" in md diff --git a/test_pipeline.py b/test_pipeline.py index 35dc2b1..651a4be 100644 --- a/test_pipeline.py +++ b/test_pipeline.py @@ -170,12 +170,14 @@ def test_end_to_end_mini_universe(tmp_path, universe_csv, fake_yfinance, mini_un rows = [ln for ln in report.splitlines() if ln.startswith("| ") and not ln.startswith("| Ticker")] assert len(rows) == len(signals) for ln in rows: - assert ln.count("|") == 14, ln # 13 columns + assert ln.count("|") == 15, ln # 14 columns by_row = {ln.split(" | ")[0].lstrip("| "): ln for ln in rows} for s in signals: # Age column: bars/limit for confirmed, '-' otherwise cells = by_row[s["ticker"]].split(" | ") assert cells[9] == (f"{s['bars_since_break']}/{scan.max_breakout_age(s['pattern'])}" if s["status"] == "CONFIRMED" else "-"), by_row[s["ticker"]] + assert 0 <= s["fear_greed"] <= 100 and cells[11] == str(s["fear_greed"]) # F&G column, one reading per symbol + assert s["fear_greed"] == scan.fear_greed(mini_universe[s["ticker"]]["Close"].to_numpy())["score"] assert float(cells[3]) == s["max_buy"] and s["entry"] < s["max_buy"] <= round(s["entry"] * 1.06, 2) assert s["max_buy"] == scan.max_buy_level(s["entry"], s["entry"], s["stop"]) or \ s["max_buy"] <= round(s["entry"] * (1 + scan.MAX_RUNAWAY), 2) # never above the runaway cap @@ -191,6 +193,7 @@ def test_end_to_end_mini_universe(tmp_path, universe_csv, fake_yfinance, mini_un for s in signals: assert f"| {s['ticker']} | {s['pattern']} | {s['entry']} | {s['max_buy']} | {s['stop']} |" in report assert f"{scan.MAX_RUNAWAY:.0%} above the trigger" in report # footer states the real rule + assert "F&G = the stock's own fear-and-greed reading" in report and "| F&G |" in report def _frame(*rows, start="2026-03-02"): diff --git a/tools/backtest.py b/tools/backtest.py index 5f93387..60334ba 100644 --- a/tools/backtest.py +++ b/tools/backtest.py @@ -46,7 +46,8 @@ against the prior 20 bars; for cups also the handle's volume against the cup's and the handle's volume slope. Nothing is added to ``scan.Signal`` or the nightly report; the features exist to be tested against outcomes. -* Per signal it records a **per-ticker fear-and-greed reading** (``fear_greed``): +* Per signal it records the **per-ticker fear-and-greed reading** + (``scan.fear_greed``, the same composite the report's F&G column shows): the equal-weight 0-100 composite of RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B that the TradingView community indicators of that name share, read at the scan day and at the pattern's @@ -134,8 +135,6 @@ N_BOOT = 2000 # month-block bootstrap resamples per summary HORIZONS = (5, 10, 20, 40, 60) # bars after the fill for the excursion table SLOPE_LOOKBACK = 40 # bars between the two SMA200 readings of the slope feature -FG_RSI_LEN, FG_BB_LEN, FG_MACD = 14, 20, (12, 26, 9) # the fear-and-greed composite's oscillators -FG_LOOKBACK = 250 # bars for the MACD histogram's percentile rank INF = float("inf") # Feature buckets for the outcome-by-feature table: key -> (label, right-inclusive edges, bucket names). # Fixed edges, so two replays (or the two windows of a split) are comparable bucket by bucket. @@ -254,73 +253,6 @@ def classify_variant(fill: float, stop: float, target: Optional[float], bars: pd return {"outcome": "open", "bars": len(w), "exit": last, "r": (last - fill) / risk if risk > 0 else None} -def _rsi(close: np.ndarray, n: int = FG_RSI_LEN) -> Optional[float]: - """Wilder's RSI of the last bar (an ``n``-bar simple seed, then ``(n - 1) / n`` smoothing). - - :returns: 0-100, rounded to 2 dp; 50 on a series that never moved; ``None`` with fewer than ``n + 1`` bars. - """ - if len(close) < n + 1: - return None - delta = np.diff(np.asarray(close, dtype=float)) - gains, losses = np.clip(delta, 0.0, None), np.clip(-delta, 0.0, None) - avg_gain, avg_loss = float(gains[:n].mean()), float(losses[:n].mean()) - for gain, loss in zip(gains[n:], losses[n:]): - avg_gain = (avg_gain * (n - 1) + gain) / n - avg_loss = (avg_loss * (n - 1) + loss) / n - if avg_gain == 0 and avg_loss == 0: - return 50.0 - if avg_loss == 0: - return 100.0 - return round(100 - 100 / (1 + avg_gain / avg_loss), 2) - - -def fear_greed(close: np.ndarray) -> Dict[str, Optional[float]]: - """A per-ticker fear-and-greed reading of the last bar: 0 = extreme fear, 100 = extreme greed. - - The TradingView community indicators of that name are composites of - standard oscillators computed on the symbol itself. This is a documented, - equal-weight version of the three price-based components they share: - - * ``rsi``: RSI 14 (:func:`_rsi`); - * ``macd_pct``: the MACD (12, 26, 9) histogram as its mid-rank percentile - within the trailing ``FG_LOOKBACK`` bars, so 50 means an average reading - for this stock and 99 its most bullish momentum of the year; - * ``bb_pctb``: Bollinger %B over 20 bars and 2 standard deviations, i.e. the - close's position between the bands (below 0 or above 1 = outside them), - unclipped and rounded to 3 dp; - * ``score``: the mean of RSI, the MACD percentile and %B clipped to 0-100, - over the components the history allows. - - Above 80 the TradingView scripts call it extreme greed, below 20 extreme - fear. Nothing gates on it; it exists to be tested (docs/wiki/03). - - :returns: The four keys, ``None`` where the history is too short. - - Complexity: O(bars). - """ - close = np.asarray(close, dtype=float) - out: Dict[str, Optional[float]] = {"rsi": _rsi(close), "macd_pct": None, "bb_pctb": None, "score": None} - fast, slow, signal = FG_MACD - if len(close) >= slow + signal: - s = pd.Series(close) - macd = s.ewm(span=fast, adjust=False).mean() - s.ewm(span=slow, adjust=False).mean() - hist = (macd - macd.ewm(span=signal, adjust=False).mean()).to_numpy() - window = hist[-FG_LOOKBACK:] - rank = ((window < hist[-1]).sum() + 0.5 * (window == hist[-1]).sum()) / len(window) - out["macd_pct"] = round(float(rank * 100), 1) - if len(close) >= FG_BB_LEN: - w = close[-FG_BB_LEN:] - mid, sd = float(w.mean()), float(w.std()) - if sd > 0: - out["bb_pctb"] = round((close[-1] - (mid - 2 * sd)) / (4 * sd), 3) - parts = [v for v in (out["rsi"], out["macd_pct"], - None if out["bb_pctb"] is None else min(max(out["bb_pctb"] * 100, 0.0), 100.0)) - if v is not None] - if parts: - out["score"] = round(sum(parts) / len(parts), 1) - return out - - def row_features(hist: pd.DataFrame, s: scan.Signal, atr_last: float) -> Dict[str, Optional[float]]: """Features of one signal at its scan day, from the history the scan saw. @@ -345,7 +277,7 @@ def row_features(hist: pd.DataFrame, s: scan.Signal, atr_last: float) -> Dict[st a line through the handle's volume as a fraction of its mean per bar (negative = drying up). * ``fg_score``, ``fg_rsi``, ``fg_macd_pct``, ``fg_bb_pctb``: the - :func:`fear_greed` reading and its components at the scan day; + ``scan.fear_greed`` reading and its components at the scan day; ``fg_base``: the composite at the pattern's last anchor bar (handle low, right shoulder, point 5), how fearful the stock was when the base formed. @@ -374,7 +306,7 @@ def in_atr(x: Optional[float]) -> Optional[float]: "volume_z": None, "handle_volume_ratio": None, "handle_volume_slope": None, "fg_score": None, "fg_rsi": None, "fg_macd_pct": None, "fg_bb_pctb": None, "fg_base": None} notes = s.notes or "" - fg = fear_greed(close) + fg = scan.fear_greed(close) out.update({"fg_score": fg["score"], "fg_rsi": fg["rsi"], "fg_macd_pct": fg["macd_pct"], "fg_bb_pctb": fg["bb_pctb"]}) @@ -387,7 +319,7 @@ def loc(day: str) -> int: anchor = loc(m[2]) if anchor >= 0: out["wait_bars"] = b - anchor - out["fg_base"] = fear_greed(close[:anchor + 1])["score"] + out["fg_base"] = scan.fear_greed(close[:anchor + 1])["score"] # Pattern depth from the anchors in the notes. depth = None if s.pattern == "Cup & Handle":