Skip to content

ci: add agent guide and benchmark comparison#137

Merged
liuq19 merged 1 commit into
bytedance:masterfrom
yangzhg:agents-guide
May 11, 2026
Merged

ci: add agent guide and benchmark comparison#137
liuq19 merged 1 commit into
bytedance:masterfrom
yangzhg:agents-guide

Conversation

@yangzhg

@yangzhg yangzhg commented May 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a repository-level AGENTS.md guide for coding agents and contributors.
  • Add benchmark comparison CI that runs base/head Google Benchmark jobs, compares JSON outputs, and posts a sticky PR comment.
  • Add scripts/tools/compare-benchmark.py to generate Markdown benchmark summaries and fail on regressions above the configured threshold.

Notes

  • This PR changes no library runtime code.
  • The benchmark workflow is intended for follow-up OOM split PRs; once this lands on master, later PRs can consume it directly.

Validation

  • python3 -m py_compile scripts/tools/compare-benchmark.py
  • python3 scripts/tools/compare-benchmark.py /tmp/sonic_master_bench.json /tmp/sonic_master_bench.json --warn-threshold=3 --fail-threshold=5 --output=/tmp/sonic_benchmark_comment_test.md
  • git diff --cached --check before commit

@codecov-commenter

codecov-commenter commented May 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.67%. Comparing base (4250a05) to head (591afe4).
⚠️ Report is 25 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #137      +/-   ##
==========================================
- Coverage   74.79%   69.67%   -5.13%     
==========================================
  Files          21       27       +6     
  Lines        2436     3706    +1270     
  Branches      667     1189     +522     
==========================================
+ Hits         1822     2582     +760     
+ Misses        297      211      -86     
- Partials      317      913     +596     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yangzhg yangzhg force-pushed the agents-guide branch 7 times, most recently from 58f2acd to f95ba4a Compare May 11, 2026 09:22
@yangzhg yangzhg marked this pull request as ready for review May 11, 2026 09:22
Copilot AI review requested due to automatic review settings May 11, 2026 09:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds contributor/agent guidance and introduces CI automation to run and compare Google Benchmark results between a PR’s base and head commits, then publish the comparison as a sticky PR comment.

Changes:

  • Added a root-level AGENTS.md describing project structure, workflows, and contributor/agent guidance.
  • Added a benchmark comparison workflow that runs Bazel benchmarks on base/head, generates a Markdown summary, and uploads it as an artifact.
  • Added a follow-up workflow that downloads the benchmark artifact and posts/updates a sticky PR comment with the results.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
scripts/tools/compare-benchmark.py New Python tool to diff two Google Benchmark JSON outputs and generate a Markdown summary.
AGENTS.md New repository-level guide for agents and contributors (project shape, commands, conventions).
.github/workflows/test_coverage.yml Splits Codecov upload behavior between PR vs push events and enables OIDC for push uploads.
.github/workflows/benchmark_compare.yml New workflow to run base/head benchmarks, compare outputs, and upload artifacts.
.github/workflows/benchmark_comment.yml New workflow to post/update a sticky PR comment based on the benchmark artifact.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/tools/compare-benchmark.py
Comment thread scripts/tools/compare-benchmark.py Outdated
Comment thread .github/workflows/benchmark_compare.yml Outdated
Comment thread .github/workflows/benchmark_compare.yml
@liuq19 liuq19 merged commit 92964a8 into bytedance:master May 11, 2026
20 checks passed
@yangzhg yangzhg deleted the agents-guide branch May 12, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants