[observability] Observability Coverage Report - 2026-05-04

[github-actions[bot]]

Executive Summary

Analyzed 75 workflow run summaries downloaded for the last 7 days. Coverage metrics below use the 72 completed runs as the denominator; 3 in-progress runs were captured before final artifacts were uploaded and are listed as partial data.

Completed runs have strong MCP telemetry coverage through the canonical rpc-messages.jsonl fallback. Firewall coverage is nearly complete, with one failed completed run missing the Squid access.log artifact needed for network debugging.

Key Alerts and Anomalies

🔴 Critical Issues:

• Missing firewall access.log: Q §25289990921

⚠️ Warnings:

• MCP telemetry quality warning: Dev §25292503542 (0 entries, 0 parse errors, 0 missing-field entries)
• MCP telemetry quality warning: Dev §25291577575 (0 entries, 0 parse errors, 0 missing-field entries)
• MCP telemetry quality warning: Dev §25291552505 (0 entries, 0 parse errors, 0 missing-field entries)
• MCP telemetry quality warning: Contribution Check §25290783713 (0 entries, 0 parse errors, 0 missing-field entries)
• MCP telemetry quality warning: Dev §25289853621 (0 entries, 0 parse errors, 0 missing-field entries)
• MCP telemetry quality warning: Dev §25288689592 (0 entries, 0 parse errors, 0 missing-field entries)
• 3 runs were still in progress when logs were captured and are excluded from coverage denominators: Daily Observability Report for AWF Firewall and MCP Gateway §25294543771, Semantic Function Refactoring §25294535334, Daily Documentation Healer §25294520924

Coverage Summary

Component	Runs Analyzed	Logs Present	Coverage	Status
AWF Firewall (access.log)	72 (firewall_enabled_workflows)	71	98.6%	🔴
MCP Gateway (gateway.jsonl or rpc-messages.jsonl)	71 (mcp_enabled_workflows)	71	100.0%	⚠️
Overall required observability	143 log requirements	142	99.3% (observability_coverage_percentage)	🔴

📋 Detailed Run Analysis

Firewall-Enabled Runs

Workflow	Run ID	access.log	Entries	Allowed	Blocked	Status
Daily Model Inventory Checker	§25294350506	✅	11	11	0	✅ healthy
Smoke CI	§25294241277	✅	10	10	0	✅ healthy
Test Quality Sentinel	§25294236729	✅	34	34	0	✅ healthy
Design Decision Gate 🏗️	§25294236722	✅	38	38	0	✅ healthy
Daily Security Red Team Agent	§25294209948	✅	278	278	0	✅ healthy
Smoke CI	§25293921314	✅	10	10	0	✅ healthy
Smoke CI	§25293688812	✅	9	9	0	✅ healthy
Test Quality Sentinel	§25293685470	✅	103	103	0	✅ healthy
Design Decision Gate 🏗️	§25293685460	✅	124	124	0	✅ healthy
Smoke CI	§25293644486	✅	9	9	0	✅ healthy
Test Quality Sentinel	§25293323839	✅	325	325	0	✅ healthy
Design Decision Gate 🏗️	§25293323836	✅	132	132	0	✅ healthy
Smoke CI	§25293269234	✅	9	9	0	✅ healthy
Test Quality Sentinel	§25293261631	✅	135	135	0	✅ healthy
Design Decision Gate 🏗️	§25293261626	✅	34	34	0	✅ healthy
Test Quality Sentinel	§25293255563	✅	38	38	0	✅ healthy
Design Decision Gate 🏗️	§25293255559	✅	35	35	0	✅ healthy
Design Decision Gate 🏗️	§25293168791	✅	172	172	0	✅ healthy
Test Quality Sentinel	§25293168790	✅	131	131	0	✅ healthy
Design Decision Gate 🏗️	§25292991309	✅	38	38	0	✅ healthy
Test Quality Sentinel	§25292991302	✅	39	39	0	✅ healthy
Smoke CI	§25292865433	✅	10	10	0	✅ healthy
Design Decision Gate 🏗️	§25292672348	✅	37	37	0	✅ healthy
Test Quality Sentinel	§25292672333	✅	32	32	0	✅ healthy
Dev	§25292503542	✅	156	156	0	✅ healthy
Test Quality Sentinel	§25292452480	✅	145	145	0	✅ healthy
Design Decision Gate 🏗️	§25292452435	✅	105	105	0	✅ healthy
Dev	§25292144360	✅	125	125	0	✅ healthy
Daily Model Inventory Checker	§25291737132	✅	10	10	0	✅ healthy
Design Decision Gate 🏗️	§25291726865	✅	35	35	0	✅ healthy
Test Quality Sentinel	§25291726848	✅	38	38	0	✅ healthy
Dev	§25291577575	✅	159	159	0	✅ healthy
Dev	§25291552505	✅	159	159	0	✅ healthy
Daily A/B Testing Advisor	§25291428470	✅	150	150	0	✅ healthy
Daily Regulatory Report Generator	§25291265701	✅	301	301	0	✅ healthy
Agentic Workflow Audit Agent	§25291205845	✅	945	945	0	✅ healthy
Contribution Check	§25290783713	✅	119	119	0	✅ healthy
Daily Project Performance Summary Generator (Using MCP Scripts)	§25290618668	✅	712	712	0	✅ healthy
Daily Caveman Optimizer	§25290575926	✅	859	859	0	✅ healthy
Copilot PR Prompt Pattern Analysis	§25290535643	✅	121	121	0	✅ healthy
Design Decision Gate 🏗️	§25290220368	✅	36	36	0	✅ healthy
Test Quality Sentinel	§25290220364	✅	35	35	0	✅ healthy
Smoke CI	§25290168232	✅	10	10	0	✅ healthy
Test Quality Sentinel	§25290165068	✅	33	33	0	✅ healthy
Design Decision Gate 🏗️	§25290165058	✅	39	39	0	✅ healthy
Smoke CI	§25290102938	✅	10	10	0	✅ healthy
Smoke CI	§25290026937	✅	10	10	0	✅ healthy
Design Decision Gate 🏗️	§25290024945	✅	39	39	0	✅ healthy
Test Quality Sentinel	§25290024934	✅	111	111	0	✅ healthy
Q	§25289990921	❌	0	0	0	🔴 critical
Design Decision Gate 🏗️	§25289880996	✅	36	36	0	✅ healthy
Test Quality Sentinel	§25289880983	✅	54	54	0	✅ healthy
Lockfile Statistics Analysis Agent	§25289874583	✅	269	269	0	✅ healthy
Dev	§25289853621	✅	158	158	0	✅ healthy
Daily Team Evolution Insights	§25289836748	✅	175	175	0	✅ healthy
Scout	§25289739207	✅	119	119	0	✅ healthy
Smoke CI	§25289676815	✅	10	10	0	✅ healthy
Daily Model Inventory Checker	§25289424690	✅	128	128	0	✅ healthy
Design Decision Gate 🏗️	§25289411205	✅	37	37	0	✅ healthy
Test Quality Sentinel	§25289411188	✅	37	37	0	✅ healthy
Design Decision Gate 🏗️	§25289292363	✅	107	107	0	✅ healthy
Test Quality Sentinel	§25289292360	✅	212	212	0	✅ healthy
Design Decision Gate 🏗️	§25288996474	✅	35	35	0	✅ healthy
Test Quality Sentinel	§25288996461	✅	37	37	0	✅ healthy
Test Quality Sentinel	§25288952348	✅	37	37	0	✅ healthy
Design Decision Gate 🏗️	§25288952345	✅	35	35	0	✅ healthy
Test Quality Sentinel	§25288924811	✅	137	137	0	✅ healthy
Design Decision Gate 🏗️	§25288924793	✅	137	137	0	✅ healthy
Smoke CI	§25288832021	✅	9	9	0	✅ healthy
Smoke CI	§25288717329	✅	9	9	0	✅ healthy
Dev	§25288689592	✅	50	50	0	✅ healthy
Issue Monster	§25288551767	✅	32	32	0	✅ healthy

Missing Firewall Logs (access.log)

Workflow	Run ID	Date	Status
Q	§25289990921	2026-05-03	failure

MCP-Enabled Runs

Workflow	Run ID	Telemetry Source	Entries	Servers	Tool Calls	Errors	Status
Daily Model Inventory Checker	§25294350506	rpc-messages.jsonl	4	github, safeoutputs	0	0	✅ healthy
Smoke CI	§25294241277	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25294236729	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25294236722	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Daily Security Red Team Agent	§25294209948	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Smoke CI	§25293921314	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Smoke CI	§25293688812	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25293685470	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Design Decision Gate 🏗️	§25293685460	rpc-messages.jsonl	8	safeoutputs	3	0	✅ healthy
Smoke CI	§25293644486	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25293323839	rpc-messages.jsonl	12	safeoutputs	5	0	✅ healthy
Design Decision Gate 🏗️	§25293323836	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Smoke CI	§25293269234	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25293261631	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Design Decision Gate 🏗️	§25293261626	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25293255563	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25293255559	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25293168791	rpc-messages.jsonl	8	safeoutputs	3	0	✅ healthy
Test Quality Sentinel	§25293168790	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Design Decision Gate 🏗️	§25292991309	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25292991302	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Smoke CI	§25292865433	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25292672348	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25292672333	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Dev	§25292503542	rpc-messages.jsonl	0	N/A	0	0	⚠️ warning
Test Quality Sentinel	§25292452480	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Design Decision Gate 🏗️	§25292452435	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Dev	§25292144360	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Daily Model Inventory Checker	§25291737132	rpc-messages.jsonl	4	github, safeoutputs	0	0	✅ healthy
Design Decision Gate 🏗️	§25291726865	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25291726848	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Dev	§25291577575	rpc-messages.jsonl	0	N/A	0	0	⚠️ warning
Dev	§25291552505	rpc-messages.jsonl	0	N/A	0	0	⚠️ warning
Daily A/B Testing Advisor	§25291428470	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Daily Regulatory Report Generator	§25291265701	rpc-messages.jsonl	14	github, mcpscripts, safeoutputs	4	0	✅ healthy
Agentic Workflow Audit Agent	§25291205845	rpc-messages.jsonl	22	agenticworkflows, github, safeoutputs	8	0	✅ healthy
Contribution Check	§25290783713	rpc-messages.jsonl	0	N/A	0	0	⚠️ warning
Daily Project Performance Summary Generator (Using MCP Scripts)	§25290618668	rpc-messages.jsonl	22	mcpscripts, safeoutputs	9	0	✅ healthy
Daily Caveman Optimizer	§25290575926	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Copilot PR Prompt Pattern Analysis	§25290535643	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25290220368	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25290220364	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Smoke CI	§25290168232	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25290165068	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25290165058	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Smoke CI	§25290102938	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Smoke CI	§25290026937	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25290024945	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25290024934	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Design Decision Gate 🏗️	§25289880996	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25289880983	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Lockfile Statistics Analysis Agent	§25289874583	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Dev	§25289853621	rpc-messages.jsonl	0	N/A	0	0	⚠️ warning
Daily Team Evolution Insights	§25289836748	rpc-messages.jsonl	20	github, safeoutputs	8	0	✅ healthy
Scout	§25289739207	rpc-messages.jsonl	26	arxiv, markitdown, microsoftdocs, safeoutputs, tavily	8	0	✅ healthy
Smoke CI	§25289676815	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Daily Model Inventory Checker	§25289424690	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25289411205	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25289411188	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25289292363	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25289292360	rpc-messages.jsonl	8	safeoutputs	3	0	✅ healthy
Design Decision Gate 🏗️	§25288996474	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25288996461	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25288952348	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Design Decision Gate 🏗️	§25288952345	rpc-messages.jsonl	4	safeoutputs	1	0	✅ healthy
Test Quality Sentinel	§25288924811	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy
Design Decision Gate 🏗️	§25288924793	rpc-messages.jsonl	10	safeoutputs	4	0	✅ healthy
Smoke CI	§25288832021	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Smoke CI	§25288717329	rpc-messages.jsonl	6	github, safeoutputs	1	0	✅ healthy
Dev	§25288689592	rpc-messages.jsonl	0	N/A	0	0	⚠️ warning
Issue Monster	§25288551767	rpc-messages.jsonl	6	safeoutputs	2	0	✅ healthy

Missing MCP Telemetry (no gateway.jsonl or rpc-messages.jsonl)

Workflow	Run ID	Date	Status
None	N/A	N/A	N/A

Partial In-Progress Runs

Workflow	Run ID	Date	Status
Daily Observability Report for AWF Firewall and MCP Gateway	§25294543771	2026-05-03	in_progress
Semantic Function Refactoring	§25294535334	2026-05-03	in_progress
Daily Documentation Healer	§25294520924	2026-05-03	in_progress

🔍 Telemetry Quality Analysis

Firewall Log Quality

• Total access.log entries analyzed: 8115
• Domains accessed: 4 unique
• Blocked requests: 0 (0.0%)
• Allowed requests: 8115
• Most accessed domains: 0 (7187), 200 (820), 400 (79), 403 (29)

Gateway Log Quality

• Telemetry source: rpc-messages.jsonl canonical fallback
• Runs with gateway.jsonl: 0
• Runs with rpc-messages.jsonl: 71
• Total MCP telemetry entries analyzed: 418
• MCP servers used: agenticworkflows, arxiv, github, markitdown, mcpscripts, microsoftdocs, safeoutputs, tavily
• Total tool calls: 117
• Error rate: 0.0%
• Average response time: N/A (N/A; raw rpc messages did not expose pairable request IDs consistently)

Additional Artifacts

• Completed runs with agent-stdio.log: 71/72
• Completed runs with safeoutputs.jsonl: 63/72
• Completed runs with aw_info.json: 72/72

Healthy Runs Summary

• Firewall healthy: 71/72
• MCP telemetry healthy: 65/71
• Completed runs with all required observability: 65/72

Recommended Actions

1. Investigate why the failed Q run §25289990921 did not upload the Squid access.log artifact, since that prevents network-level debugging for the failure.
2. Keep monitoring in-progress scheduled runs separately from completed runs so early artifact snapshots do not create false missing-log alerts.
3. Consider adding gateway.jsonl alongside rpc-messages.jsonl where possible so response-time and status metrics are available without relying on summarized fallback data.

📊 Historical Trends

No historical trend store was available in the downloaded artifact set. This report records the current 7-day snapshot for comparison by future daily runs.

References: §25294543771, §25289990921, §25294350506

Analysis window: Last 7 days | Runs analyzed: 75 (completed coverage denominator: 72)

Warning

Firewall blocked 4 domains

The following domains were blocked by the firewall during workflow execution:

• ab.chatgpt.com
• api.github.com
• chatgpt.com
• github.com

💡 Tip: api.github.com is blocked because GitHub API access uses the built-in GitHub tools by default. Instead of adding api.github.com to network.allowed, use tools.github.mode: gh-proxy for direct pre-authenticated GitHub CLI access without requiring network access to api.github.com:

tools:
  github:
    mode: gh-proxy

See GitHub Tools for more information on gh-proxy mode.

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "api.github.com"
    - "chatgpt.com"
    - "github.com"

See Network Configuration for more information.

Generated by Daily Observability Report for AWF Firewall and MCP Gateway · ◷

• expires on May 5, 2026, 12:12 AM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Daily Observability Report for AWF Firewall and MCP Gateway.

A newer discussion is available at Discussion #30235.