In semantic-conventions-conformance we drive WeaverLiveCheck from a conformance runner: one live-check per scenario, report written to disk, coverage reduced across a run. A few things we had to work around.
Raw report isn't reachable. LiveCheckReport proxies __getitem__/get/__contains__ but never exposes the underlying dict, so to persist weaver's JSON we read report._report. A public raw (or to_dict()) would fix it, and typed samples / statistics properties would cover what most callers index anyway.
extra_args instead of named parameters. --config and --advice-data have no parameter, so we pass them as raw strings positioned against weaver's CLI. config= and advice_data= would be checkable; policies_dir= could also take a sequence once weaver accepts repeated --advice-policies (weaver#1679).
Weaver's stdout/stderr are unreachable. They're written to tempfiles, read only on failure into logger.error, and deleted in close(). When a run misbehaves there's nothing to attach to a report or a test failure. Exposing them as a property, and attaching them to LiveCheckError, would make failures diagnosable.
Startup wait is noisy and fixed. _wait_for_ready polls /health through a urllib3 Retry, so every connection refused before weaver binds its port is logged as a warning — we silence the urllib3.connectionpool logger around the whole session to keep output readable. The retry ceiling (~10s) is also not configurable; a cold start against a large registry can exceed it. A quiet readiness wait plus a startup_timeout parameter would help.
end() after stop returns an empty report. It warns and returns LiveCheckReport({}). For a runner that reduces reports into committed coverage, that reads as "the scenario emitted nothing" rather than a programming error — raising would be safer.
In semantic-conventions-conformance we drive
WeaverLiveCheckfrom a conformance runner: one live-check per scenario, report written to disk, coverage reduced across a run. A few things we had to work around.Raw report isn't reachable.
LiveCheckReportproxies__getitem__/get/__contains__but never exposes the underlying dict, so to persist weaver's JSON we readreport._report. A publicraw(orto_dict()) would fix it, and typedsamples/statisticsproperties would cover what most callers index anyway.extra_argsinstead of named parameters.--configand--advice-datahave no parameter, so we pass them as raw strings positioned against weaver's CLI.config=andadvice_data=would be checkable;policies_dir=could also take a sequence once weaver accepts repeated--advice-policies(weaver#1679).Weaver's stdout/stderr are unreachable. They're written to tempfiles, read only on failure into
logger.error, and deleted inclose(). When a run misbehaves there's nothing to attach to a report or a test failure. Exposing them as a property, and attaching them toLiveCheckError, would make failures diagnosable.Startup wait is noisy and fixed.
_wait_for_readypolls/healththrough a urllib3Retry, so every connection refused before weaver binds its port is logged as a warning — we silence theurllib3.connectionpoollogger around the whole session to keep output readable. The retry ceiling (~10s) is also not configurable; a cold start against a large registry can exceed it. A quiet readiness wait plus astartup_timeoutparameter would help.end()after stop returns an empty report. It warns and returnsLiveCheckReport({}). For a runner that reduces reports into committed coverage, that reads as "the scenario emitted nothing" rather than a programming error — raising would be safer.