Skip to content

Commit 6182ccd

Browse files
committed
ci(conformance): bump harness to 0.2.0-alpha.5 preview (pkg.pr.new@65fcd39)
CONFORMANCE_VERSION -> CONFORMANCE_PKG (full npx spec) so the same line accepts either a registry version or a tarball URL. Pinned to the immutable commit-SHA pkg.pr.new build of conformance#357 until alpha.5 publishes; the workflow comment carries an explicit TODO + a do-not-merge guard. Reconciliation: conformance#337 gated the SEP-2350 scope-union check to 2026-07-28+, so auth/scope-step-up now passes on the 2025 client leg and is removed from expected-failures.yml (it stays in the 2026 baseline).
1 parent b7a5bff commit 6182ccd

4 files changed

Lines changed: 21 additions & 19 deletions

File tree

.github/actions/conformance/expected-failures.2026-07-28.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,10 @@
1010
# 2026 leg reads the `server:` section. Both burn down independently of the
1111
# 2025 legs.
1212
#
13-
# Baseline established against @modelcontextprotocol/conformance pinned in
14-
# .github/workflows/conformance.yml (CONFORMANCE_VERSION = 0.2.0-alpha.4).
15-
# New conformance releases are adopted by deliberately bumping that pin and
16-
# reconciling both this file and expected-failures.yml in the same change.
13+
# Baseline established against the harness pinned via CONFORMANCE_PKG in
14+
# .github/workflows/conformance.yml. New conformance releases are adopted by
15+
# deliberately bumping that pin and reconciling both this file and
16+
# expected-failures.yml in the same change.
1717
#
1818
# Entries are grouped by what unblocks them. As each gap closes the
1919
# corresponding scenarios start passing and MUST be removed from this list

.github/actions/conformance/expected-failures.yml

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,10 @@
11
# Conformance scenarios not yet passing against the Python SDK on main.
22
# CI exits 0 if only these fail, exits 1 on unexpected failures or stale entries.
33
#
4-
# Baseline established against @modelcontextprotocol/conformance pinned in
5-
# .github/workflows/conformance.yml (CONFORMANCE_VERSION = 0.2.0-alpha.4).
6-
# New conformance releases are adopted by deliberately bumping that pin and
7-
# reconciling both this file and expected-failures.2026-07-28.yml in the same
8-
# change.
4+
# Baseline established against the harness pinned via CONFORMANCE_PKG in
5+
# .github/workflows/conformance.yml. New conformance releases are adopted by
6+
# deliberately bumping that pin and reconciling both this file and
7+
# expected-failures.2026-07-28.yml in the same change.
98
#
109
# Entries are grouped by SEP. As each SEP lands in the SDK the corresponding
1110
# scenarios start passing and MUST be removed from this list (the runner fails
@@ -40,9 +39,6 @@ client:
4039
- auth/offline-access-not-supported
4140

4241
# --- Pre-existing scenarios that fail on checks added after conformance 0.1.15 ---
43-
# SEP-2350 (scope step-up): WARNING-only; the expected-failures evaluator
44-
# counts WARNINGs as failures.
45-
- auth/scope-step-up
4642
# SEP-990 (enterprise-managed authorization extension): no fixture handler /
4743
# client support for the token-exchange + JWT bearer flow.
4844
- auth/enterprise-managed-authorization

.github/actions/conformance/run-server.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,5 +47,5 @@ done
4747

4848
echo "Server ready at $SERVER_URL"
4949

50-
npx --yes @modelcontextprotocol/conformance@"${CONFORMANCE_VERSION:?set CONFORMANCE_VERSION (pinned in .github/workflows/conformance.yml)}" \
50+
npx --yes "${CONFORMANCE_PKG:?set CONFORMANCE_PKG (pinned in .github/workflows/conformance.yml)}" \
5151
server --url "$SERVER_URL" "$@"

.github/workflows/conformance.yml

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,16 @@ permissions:
1414
contents: read
1515

1616
env:
17-
# Pinned conformance harness version. Bump deliberately and reconcile
18-
# both .github/actions/conformance/expected-failures*.yml files in the
19-
# same change.
20-
CONFORMANCE_VERSION: "0.2.0-alpha.4"
17+
# Pinned conformance harness package spec (passed verbatim to `npx --yes`).
18+
# Use a published version, e.g. @modelcontextprotocol/conformance@0.2.0-alpha.5.
19+
# Bump deliberately and reconcile both
20+
# .github/actions/conformance/expected-failures*.yml files in the same change.
21+
#
22+
# TODO: replace with @modelcontextprotocol/conformance@0.2.0-alpha.5 once
23+
# https://github.com/modelcontextprotocol/conformance/pull/357 publishes.
24+
# The pkg.pr.new URL below is the preview build of that PR pinned at commit
25+
# 65fcd39 (immutable). Do not merge this branch to main with a pkg.pr.new pin.
26+
CONFORMANCE_PKG: "https://pkg.pr.new/@modelcontextprotocol/conformance@65fcd39"
2127

2228
jobs:
2329
server-conformance:
@@ -67,13 +73,13 @@ jobs:
6773
- run: uv sync --frozen --all-extras --package mcp
6874
- name: Run client conformance (all suite)
6975
run: >-
70-
npx --yes @modelcontextprotocol/conformance@"$CONFORMANCE_VERSION" client
76+
npx --yes "$CONFORMANCE_PKG" client
7177
--command 'uv run --frozen python .github/actions/conformance/client.py'
7278
--suite all
7379
--expected-failures ./.github/actions/conformance/expected-failures.yml
7480
- name: Run client conformance (2026-07-28 wire, all suite)
7581
run: >-
76-
npx --yes @modelcontextprotocol/conformance@"$CONFORMANCE_VERSION" client
82+
npx --yes "$CONFORMANCE_PKG" client
7783
--command 'uv run --frozen python .github/actions/conformance/client.py'
7884
--suite all
7985
--spec-version 2026-07-28

0 commit comments

Comments
 (0)