OrderFillsDuringExtendedMarketHoursAnalysis reports Sample/Count from all order events instead of the flagged fills

[DerekMelchin]

Expected Behavior

When OrderFillsDuringExtendedMarketHoursAnalysis flags a backtest, the Analysis it returns should describe the flagged fills:

• Sample should be one of the order events that actually filled during extended market hours (the issue the analysis detected).
• Count should be the number of extended-hours fills found.

This matches the documented meaning of those fields on Analysis: Sample is "a representative sample value of the issue detected" and Count is "the total number of matching occurrences found by the analysis."

Actual Behavior

OrderFillsDuringExtendedMarketHoursAnalysis.Run accumulates the flagged fills into a local result list, but then builds the response from the full orderEvents list instead of result:

var potentialSolutions = result.Count > 0 ? Solutions(language) : [];
return SingleResponse(orderEvents.Count > 0 ? orderEvents[0] : null, orderEvents.Count > 1 ? orderEvents.Count : null, potentialSolutions);

So when the analysis fires:

• Sample is orderEvents[0] — the first order event of the entire backtest, which is almost always unrelated to the issue (typically a Submitted event, not even a fill, and during regular hours).
• Count is orderEvents.Count — the total number of order events, not the number of extended-hours fills.

The detection itself is correct (Solutions is gated on result.Count > 0, so the analysis only fires when there genuinely is an extended-hours fill), but the reported Sample/Count are misleading: a reader sees "filled orders during extended market hours" alongside an example order that was submitted during regular hours and a count equal to the total order-event count.

Potential Solution

Build the response from the flagged result list rather than orderEvents:

var potentialSolutions = result.Count > 0 ? Solutions(language) : [];
return SingleResponse(result.Count > 0 ? result[0] : null, result.Count > 1 ? result.Count : null, potentialSolutions);

This reports the first flagged extended-hours fill as the Sample and the number of flagged fills as the Count, consistent with the other analyses and with the Analysis field documentation.

Reproducing the Problem

This minimal algorithm places only 3 orders and trips OrderFillsDuringExtendedMarketHoursAnalysis — its single flagged fill is the engine's midnight RXDX delisting liquidation (related to #9512):

from AlgorithmImports import *


class LiquidUniverseAlgorithm(QCAlgorithm):

    def initialize(self):
        self.set_start_date(2023, 5, 25)
        self.set_end_date(2023, 6, 25)
        self.set_cash(100000)
        self.settings.seed_initial_prices = True
        self._rxdx = self.add_equity('RXDX', Resolution.DAILY)
        self._spy = self.add_equity('SPY', Resolution.DAILY)
        self.schedule.on(
            self.date_rules.month_start(self._spy),
            self.time_rules.at(8, 0),
            self._rebalance
        )

    def _rebalance(self):
        targets = [PortfolioTarget(symbol, 0.5) for symbol in [self._spy, self._rxdx]]
        self.set_holdings(targets)

There is exactly one flagged fill, but the OrderFillsDuringExtendedMarketHoursAnalysis entry in the result JSON reports:

• "count": 5 — the total number of order events, not the 1 flagged fill.
• a sample with "orderId": 1, "status": "submitted", "fillPrice": 0.0 — the first order event of the backtest (an RXDX buy submission), not the flagged extended-hours fill.

System Information

QC Cloud

Checklist

• I have completely filled out this template
• I have confirmed that this issue exists on the current master branch
• I have confirmed that this is not a duplicate issue by searching issues

• I have provided detailed steps to reproduce the issue