feat: F&G column, the stock's fear-and-greed reading on every report row

CHANGELOG.md

@@ -5,6 +5,15 @@ All notable changes to this project are documented here. Format follows
 
 ## [Unreleased]
 
+### Added (report: the F&G column)
+- Every report row carries `F&G`, the stock's own fear-and-greed reading at
+  the last close, 0 to 100, one value per symbol; `fear_greed` in
+  `signals.json`. The composite (`scan.fear_greed`, RSI 14, MACD-histogram
+  percentile within the trailing year, Bollinger %B) moved from the backtest
+  into the scanner so the report and the replay share one implementation.
+  The footer explains the zones. Information only; the stretch gate it
+  suggests is #113.
+
 ### Added (per-ticker fear and greed, 2026-09-08)
 - Replay rows and the outcome-by-feature table gain a per-ticker fear-and-greed
   reading: the equal-weight 0-100 composite of RSI 14, the MACD (12, 26, 9)

README.md

@@ -109,7 +109,7 @@ Data errors: 1
 | DG     | Cup & Handle | WATCHLIST | FAILED         | 134.13 | 116.6  | 161.68 | close 116.20 on 2026-09-03 at or below stop 116.6 |
 ```
 
-Max buy is the open above which the setup no longer qualifies: the trigger plus 5 %, or lower where a fill would already carry 1.5× the planned risk (CL above: a Wolfe stop sits 1.6 % under the entry, so the chase allowance is 0.8 %, not 5 %). R:R is the reward per unit of planned risk, `(target − entry) / (entry − stop)`; it shrinks with every session the entry drifts above the trigger, which is why HAL, four sessions past its breakout, shows 1.09. The last table explains every row of the previous report that is gone today (the two rows above are illustrative). `output/signals.json` carries the same rows as records plus a `meta` block (`last_bar`, per-symbol bar histogram, effective breakout-age limits, the market context: SPY regime, VIX and breadth, informational only) and the `closed` list. The schema is documented in the wiki.
+Max buy is the open above which the setup no longer qualifies: the trigger plus 5 %, or lower where a fill would already carry 1.5× the planned risk (CL above: a Wolfe stop sits 1.6 % under the entry, so the chase allowance is 0.8 %, not 5 %). R:R is the reward per unit of planned risk, `(target − entry) / (entry − stop)`; it shrinks with every session the entry drifts above the trigger, which is why HAL, four sessions past its breakout, shows 1.09. F&G (added after this sample) is the stock's own fear-and-greed reading at the last close, 0 to 100, the average of RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B; above 80 the stock is stretched, and over ten years of replay such breakouts paid least. Information only. The last table explains every row of the previous report that is gone today (the two rows above are illustrative). `output/signals.json` carries the same rows as records plus a `meta` block (`last_bar`, per-symbol bar histogram, effective breakout-age limits, the market context: SPY regime, VIX and breadth, informational only) and the `closed` list. The schema is documented in the wiki.
 
 ## Documentation
 

docs/wiki/01-Architecture-and-Data-Pipeline.md

@@ -101,10 +101,11 @@ Yahoo publishes the newest daily bar per symbol at different times (volume first
 | `notes` | anchor dates and levels used by the detector (parseable, see tests); "breakout without volume (x.xx×)" when a breakout was watch-listed for lack of volume |
 | `max_buy` | the open above which the setup no longer qualifies: the lower of trigger × 1.05 (the runaway rule applied to the open) and `stop + MAX_BUY_RISK_MULT × (entry − stop)`, the fill at which the risk reaches 1.5× the planned risk. The second cap binds for tight structural stops (Wolfe point 5, shallow handles); the first for wide ones (H&S shoulders) |
 | `reward_risk` | `(target − entry) / (entry − stop)` at the reported entry, 2 decimals; `null` without a target. Rows below `MIN_REWARD_RISK` (when set) are not reported. It falls as the entry drifts above the trigger, so a late confirmed row can show a poor R:R on an otherwise clean pattern |
+| `fear_greed` | the stock's own fear-and-greed reading at `last_bar`, 0 to 100 (`fear_greed`): the equal-weight average of RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B, one reading per symbol shared by its rows. Above 80 the stock is stretched, below 20 washed out. Informational: no rule reads it; the ten-year evidence is on the tuning page |
 
 `meta.market` is the market context at `last_bar`: the index ETF's close, its distance from its SMA200 and its SMA50's distance from the SMA200 in percent, the regime those name, the VIX close, and the breadth over `breadth_symbols`; each `null` where unavailable. It is informational and reproduced on every backtest row.
 
-Signals are sorted `CONFIRMED` first, then by score descending. `output/report.md` renders the same rows as two Markdown tables (Ticker, Pattern, Entry, Max buy, Stop, Risk %, Target, R:R, Score, Age, Vol×, Trend, Details) with a header stating the scanned bar, effective age limits, skipped/lagging counts and data errors. **Age** is `bars_since_break / limit`, e.g. `1/3` for a cup that broke out yesterday and will be dropped after two more sessions; `-` for watchlist rows.
+Signals are sorted `CONFIRMED` first, then by score descending. `output/report.md` renders the same rows as two Markdown tables (Ticker, Pattern, Entry, Max buy, Stop, Risk %, Target, R:R, Score, Age, Vol×, F&G, Trend, Details) with a header stating the scanned bar, effective age limits, skipped/lagging counts and data errors. **Age** is `bars_since_break / limit`, e.g. `1/3` for a cup that broke out yesterday and will be dropped after two more sessions; `-` for watchlist rows.
 
 **Closed since the last report.** A stateless scan only knows what qualifies today, so each run also reads the previous committed `signals.json` (the nightly job has it in the checkout) and explains every row that disappeared, using the bars since that row's `last_date`. The list is `closed` in `signals.json` (ticker, pattern, was, since, entry, stop, target, outcome, detail) and a third table in the report; `meta.previous_run` names the report it was compared with.
 

docs/wiki/03-Configuration-and-Tuning.md

@@ -91,7 +91,7 @@ Pooled, weighting each year by its traded signals: 1629 traded signals, +0.23 R,
 
 What the ten years overturned from the two-year window, each a lesson in what one exploratory pass over two bull years produces: fresh breakouts are not better (age 0 at first report +0.10 R against +0.42 for age 4 to 8; #98 closed), reward:risk above 4 is not bad (+0.45 R with a 16 % hit rate; #100 closed), a VIX below 15 is not bad (+0.21 R pooled, and 2017 at a VIX near 10 was the best year; #101 rewritten), the SMA band U-shape is one-sided (#99 rewritten), and the Wolfe verdict on 12 out-of-sample trades was noise (#102 closed as held). What survived: the tuned profile's expectancy, the score's role as a gate, and the reversal-after-washout context.
 
-**Open questions:** #99 and #101 (the deep-down-trend and bear-regime contexts, information first, no rule), #108 (the drawdowns), #109 (cups), and #97 (a per-signal probability model, whose first revisit condition the ten years now meet).
+**Open questions:** #99 and #101 (the deep-down-trend and bear-regime contexts, information first, no rule), #108 (the drawdowns), #109 (cups), #113 (the Bollinger stretch gate), and #97 (a per-signal probability model, whose first revisit condition the ten years now meet).
 
 ### A second review, tested (2026-09-08)
 
@@ -126,7 +126,7 @@ A trader's suggestion: read a per-ticker fear-and-greed indicator, as the Tradin
 
 A confirmed breakout is almost never fearful by construction (one row below 20 in ten years), so at the scan day the scale runs from fear to extreme greed, and the greedier the stock at its breakout, the lower the mean R; the higher hit rate of the greediest bucket does not compensate, because its wins are smaller. Bases that formed in fear paid about twice what bases formed in greed did. Of the components, the stretch measures carry the effect and momentum does not: RSI 30-50 at the breakout ran +0.55 R on 230 signals against +0.18 for 50-70 and +0.12 above 70, the lower bucket ahead in 8 of 11 years; Bollinger %B in the lower half ran +0.53 on 185, the upper half +0.26 on 991 and a close above the upper band +0.04 on 453, the upper half ahead of the above-band bucket in 10 of 11 years (the exception, 2017, a tie at +0.44 against +0.45); the MACD percentile ran +0.18 to +0.26 across its buckets with the strongest momentum slightly best.
 
-The one candidate rule this produces is the Bollinger stretch: a breakout bar that closes above its upper band. Leaving those 453 signals out would keep 1179 signals at +0.30 R against 1632 at +0.22, at a cost of 16 R of the ten-year total of 367. Under the protocol it remains a hypothesis: the gate has to be replayed as a rule on its own, so that its effect on the drawdown and per pattern is measured, and the reading belongs in the report as information first.
+The one candidate rule this produces is the Bollinger stretch: a breakout bar that closes above its upper band. Leaving those 453 signals out would keep 1179 signals at +0.30 R against 1632 at +0.22, at a cost of 16 R of the ten-year total of 367. Under the protocol it remains a hypothesis: the gate has to be replayed as a rule on its own, so that its effect on the drawdown and per pattern is measured (#113). The reading itself is on every report row as the `F&G` column, one value per symbol at the last close, information only.
 
 ### Market context (2026-09-08)
 

scan.py

@@ -109,6 +109,14 @@
 # is recorded on every report and backtest row to be tested, not gated on (issues #93, #101).
 MARKET_INDEX = "SPY"
 MARKET_VOL = "^VIX"
+# Per-ticker fear and greed, informational (no rule reads it): the equal-weight 0-100 composite of
+# RSI 14, the MACD histogram's percentile within the trailing year and Bollinger %B that the
+# TradingView community indicators of that name share, at the last close (``fear_greed``).  Ten years
+# of replay (docs/wiki/03, issue #113): the greedier the stock at its breakout the lower the mean R,
+# +0.45 in the neutral zone against +0.12 above 80, and a close above the upper Bollinger band ran
+# +0.04 R on 453 signals.  Shown on every report row so the reader sees the stretch; not gated on.
+FG_RSI_LEN, FG_BB_LEN, FG_MACD = 14, 20, (12, 26, 9)
+FG_LOOKBACK = 250                            # bars for the MACD histogram's percentile rank
 
 # --------------------------------------------------------------------------- #
 # Pattern rules.  The values below are the "spec" profile (docs/wiki/02 and the
@@ -317,6 +325,8 @@ class Signal:
     :param notes: Free-text details (pattern anchor dates and levels).
     :param max_buy: Highest open worth filling (see :func:`max_buy_level`).
     :param reward_risk: ``(target - entry) / (entry - stop)`` or None (see :func:`reward_risk`).
+    :param fear_greed: The stock's fear-and-greed composite at the last close, 0-100
+        (see :func:`fear_greed`); informational.
     """
 
     ticker: str
@@ -335,6 +345,7 @@ class Signal:
     notes: str = ""
     max_buy: Optional[float] = None      # above this at the open, do not chase (max_buy_level)
     reward_risk: Optional[float] = None  # reward per unit of planned risk, None without a target
+    fear_greed: Optional[float] = None   # the stock's fear-and-greed reading at the last close, 0-100 (fear_greed)
 
 
 # --------------------------------------------------------------------------- #
@@ -813,6 +824,73 @@ def atr(df: pd.DataFrame, n: int = ATR_LEN) -> pd.Series:
     return tr.rolling(n, min_periods=1).mean()
 
 
+def _rsi(close: np.ndarray, n: int = FG_RSI_LEN) -> Optional[float]:
+    """Wilder's RSI of the last bar (an ``n``-bar simple seed, then ``(n - 1) / n`` smoothing).
+
+    :returns: 0-100, rounded to 2 dp; 50 on a series that never moved; ``None`` with fewer than ``n + 1`` bars.
+    """
+    if len(close) < n + 1:
+        return None
+    delta = np.diff(np.asarray(close, dtype=float))
+    gains, losses = np.clip(delta, 0.0, None), np.clip(-delta, 0.0, None)
+    avg_gain, avg_loss = float(gains[:n].mean()), float(losses[:n].mean())
+    for gain, loss in zip(gains[n:], losses[n:]):
+        avg_gain = (avg_gain * (n - 1) + gain) / n
+        avg_loss = (avg_loss * (n - 1) + loss) / n
+    if avg_gain == 0 and avg_loss == 0:
+        return 50.0
+    if avg_loss == 0:
+        return 100.0
+    return round(100 - 100 / (1 + avg_gain / avg_loss), 2)
+
+
+def fear_greed(close: np.ndarray) -> Dict[str, Optional[float]]:
+    """A per-ticker fear-and-greed reading of the last bar: 0 = extreme fear, 100 = extreme greed.
+
+    The TradingView community indicators of that name are composites of
+    standard oscillators computed on the symbol itself.  This is a documented,
+    equal-weight version of the three price-based components they share:
+
+    * ``rsi``: RSI 14 (:func:`_rsi`);
+    * ``macd_pct``: the MACD (12, 26, 9) histogram as its mid-rank percentile
+      within the trailing ``FG_LOOKBACK`` bars, so 50 means an average reading
+      for this stock and 99 its most bullish momentum of the year;
+    * ``bb_pctb``: Bollinger %B over 20 bars and 2 standard deviations, i.e. the
+      close's position between the bands (below 0 or above 1 = outside them),
+      unclipped and rounded to 3 dp;
+    * ``score``: the mean of RSI, the MACD percentile and %B clipped to 0-100,
+      over the components the history allows.
+
+    Above 80 the TradingView scripts call it extreme greed, below 20 extreme
+    fear.  Reported per row; no rule reads it (docs/wiki/03).
+
+    :returns: The four keys, ``None`` where the history is too short.
+
+    Complexity: O(bars).
+    """
+    close = np.asarray(close, dtype=float)
+    out: Dict[str, Optional[float]] = {"rsi": _rsi(close), "macd_pct": None, "bb_pctb": None, "score": None}
+    fast, slow, signal = FG_MACD
+    if len(close) >= slow + signal:
+        s = pd.Series(close)
+        macd = s.ewm(span=fast, adjust=False).mean() - s.ewm(span=slow, adjust=False).mean()
+        hist = (macd - macd.ewm(span=signal, adjust=False).mean()).to_numpy()
+        window = hist[-FG_LOOKBACK:]
+        rank = ((window < hist[-1]).sum() + 0.5 * (window == hist[-1]).sum()) / len(window)
+        out["macd_pct"] = round(float(rank * 100), 1)
+    if len(close) >= FG_BB_LEN:
+        w = close[-FG_BB_LEN:]
+        mid, sd = float(w.mean()), float(w.std())
+        if sd > 0:
+            out["bb_pctb"] = round((close[-1] - (mid - 2 * sd)) / (4 * sd), 3)
+    parts = [v for v in (out["rsi"], out["macd_pct"],
+                         None if out["bb_pctb"] is None else min(max(out["bb_pctb"] * 100, 0.0), 100.0))
+             if v is not None]
+    if parts:
+        out["score"] = round(sum(parts) / len(parts), 1)
+    return out
+
+
 def find_pivots(high: np.ndarray, low: np.ndarray, order: int = PIVOT_ORDER
                 ) -> Tuple[List[int], List[int]]:
     """Fractal swing detection.
@@ -1606,6 +1684,11 @@ def scan_symbol(sym: str, df: pd.DataFrame, detectors: Optional[Sequence[Callabl
             out.extend(fn(df, sym))
         except Exception as exc:  # one bad ticker must not abort the scan
             log.exception("%s failed on %s: %s", fn.__name__, sym, exc)
+    if out:
+        # One reading per symbol at the last close, shared by all of its rows; informational (FG_* constants).
+        fg = fear_greed(df["Close"].to_numpy(dtype=float))["score"]
+        for s in out:
+            s.fear_greed = fg
     return out
 
 
@@ -1735,8 +1818,8 @@ def render_markdown(signals: List[Signal], meta: Mapping[str, Any],
             lines.append("")
             continue
         lines.append("| Ticker | Pattern | Entry | Max buy | Stop | Risk % | Target | R:R | Score | Age | Vol× | "
-                     "Trend | Details |")
-        lines.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|")
+                     "F&G | Trend | Details |")
+        lines.append("|---|---|---|---|---|---|---|---|---|---|---|---|---|---|")
         for s in rows:
             # Age = bars since the breakout close / the pattern's limit, so a reader
             # can see whether a confirmed row is fresh (0/3) or about to expire (3/3).
@@ -1745,7 +1828,8 @@ def render_markdown(signals: List[Signal], meta: Mapping[str, Any],
             lines.append(f"| {s.ticker} | {s.pattern} | {s.entry} | {s.max_buy if s.max_buy else '-'} | {s.stop} | "
                          f"{s.risk_pct} | {s.target if s.target else '-'} | "
                          f"{s.reward_risk if s.reward_risk is not None else '-'} | {s.score} | {age} | "
-                         f"{s.volume_ratio if s.volume_ratio else '-'} | {s.trend} | {s.notes} |")
+                         f"{s.volume_ratio if s.volume_ratio else '-'} | "
+                         f"{s.fear_greed if s.fear_greed is not None else '-'} | {s.trend} | {s.notes} |")
         lines.append("")
     if closed is not None:
         since = meta.get("previous_run") or "the last report"
@@ -1773,6 +1857,9 @@ def render_markdown(signals: List[Signal], meta: Mapping[str, Any],
     lines.append(f"_Max buy = {max_buy_rule}: if the open is above it the setup no longer qualifies. "
                  f"R:R = (target - entry) / (entry - stop) at the reported entry; it shrinks with every "
                  f"session the entry drifts above the trigger. "
+                 f"F&G = the stock's own fear-and-greed reading at the last close, 0 to 100 (RSI {FG_RSI_LEN}, "
+                 f"MACD-histogram percentile and Bollinger %B averaged): above 80 the stock is stretched and such "
+                 f"breakouts replayed worst, below 20 it is washed out; information only, no rule uses it. "
                  f"Age = bars since the breakout close / the limit after which the row is dropped "
                  f"(0 = broke out on the last bar). Heuristic scan, not advice. "
                  f"Entry = trigger level, or the breakout close when it "