Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/workflows/documentation-freshness.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: Monitor Sent contract and documentation freshness

on:
workflow_dispatch:
release:
types: [published]
schedule:
- cron: "17 9 * * 1"

permissions:
contents: read

jobs:
freshness-and-routing:
runs-on: ubuntu-latest
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
SENT_ROUTING_EVAL_MODEL: ${{ vars.SENT_ROUTING_EVAL_MODEL || 'gpt-5-mini' }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: pip
cache-dependency-path: requirements-dev.txt

- name: Install validators
run: python -m pip install -r requirements-dev.txt

- name: Compare live Sent contract and documentation facts
run: >-
python scripts/check_live_contract.py
--timeout 15
--retries 2
--output artifacts/documentation-freshness.json

- name: Upload freshness diagnostics
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: documentation-freshness
path: artifacts/documentation-freshness.json
if-no-files-found: warn

- name: Run model-backed routing evaluation
if: always() && env.OPENAI_API_KEY != ''
run: python scripts/run_model_routing_eval.py

- name: Explain skipped model evaluation
if: always() && env.OPENAI_API_KEY == ''
run: echo "OPENAI_API_KEY is not configured; freshness monitoring still ran, but model routing evaluation was skipped."
32 changes: 2 additions & 30 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ on:
branches: [main]
release:
types: [published]
schedule:
- cron: "17 9 * * 1"

permissions:
contents: read
Expand Down Expand Up @@ -60,31 +58,5 @@ jobs:
- name: Test Sent API contract regressions
run: python scripts/test_contracts.py

live-contract-and-routing:
if: github.event_name == 'schedule' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
SENT_ROUTING_EVAL_MODEL: ${{ vars.SENT_ROUTING_EVAL_MODEL || 'gpt-5-mini' }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: pip
cache-dependency-path: requirements-dev.txt

- name: Install validators
run: python -m pip install -r requirements-dev.txt

- name: Compare live Sent OpenAPI with manifest
run: python scripts/check_live_contract.py

- name: Run model-backed routing evaluation
if: env.OPENAI_API_KEY != ''
run: python scripts/run_model_routing_eval.py

- name: Explain skipped model evaluation
if: env.OPENAI_API_KEY == ''
run: echo "OPENAI_API_KEY is not configured; live contract drift still ran, but model routing evaluation was skipped."
- name: Test freshness monitor simulations
run: python scripts/test_live_contract.py
6 changes: 4 additions & 2 deletions docs/PUBLIC_RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Use this checklist for every public release.
## Repository and package

- Confirm the GitHub repository is public and the release commit is on the default branch.
- Confirm the repository root contains `plugin.json`, `mcp.json`, `skills/`, and `assets/` for GitHub auto-discovery.
- Confirm the repository root contains `plugin.json`, `mcp.json`, `public-surface.json`, `skills/`, and `assets/` for GitHub auto-discovery.
- Confirm `.claude-plugin/marketplace.json` points to `./claude-plugins/sent`.
- Confirm the Claude plugin contains its manifest, README, license, skills, commands, assets, and HTTPS MCP configuration.
- Update the version consistently before release; plugin names are stable public identifiers and must not be renamed casually.
Expand All @@ -18,11 +18,13 @@ python3 scripts/generate_adapters.py --check
python3 scripts/validate.py
python3 scripts/test_validation_gates.py
python3 scripts/test_fixtures.py
python3 scripts/test_contracts.py
python3 scripts/test_live_contract.py
claude plugin validate . --strict
claude plugin validate ./claude-plugins/sent --strict
```

CI must pass on the exact commit being submitted. Resolve warnings as well as errors before submission.
CI must pass on the exact commit being submitted. Resolve warnings as well as errors before submission. The dedicated contract/documentation freshness workflow runs on schedule, manual dispatch, and release; review its uploaded JSON artifact separately from pull-request validation.

## Public-facing review

Expand Down
4 changes: 4 additions & 0 deletions packages/sent/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ python3 scripts/generate_adapters.py
python3 scripts/validate.py
python3 scripts/test_validation_gates.py
python3 scripts/test_fixtures.py
python3 scripts/test_contracts.py
python3 scripts/test_live_contract.py
```

For current product and API behavior, use the [Sent MCP documentation](https://docs.sent.dm/start/llm-docs/mcp-server), the [machine-readable documentation index](https://docs.sent.dm/llms.txt), and the [Sent API reference](https://docs.sent.dm/reference/api).

Live network checks are isolated from pull-request validation. The scheduled/manual/release freshness workflow compares normalized facts from the source catalog, classifies drift separately from source failures, and uploads a JSON diagnostic artifact.
115 changes: 107 additions & 8 deletions schemas/sent/documentation-sources.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,21 @@
"url": "https://docs.sent.dm/llms.txt",
"kind": "documentation-index",
"affected_skills": ["sent"],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "docs.index.template_definition_listed",
"extractor": "contains",
"pattern": "https://docs.sent.dm/llms/reference/api/template-definition.txt",
"expected": true
},
{
"fact": "docs.index.webhook_events_listed",
"extractor": "contains",
"pattern": "https://docs.sent.dm/llms/start/webhooks/event-types.txt",
"expected": true
}
]
},
{
"id": "openapi",
Expand All @@ -22,14 +36,29 @@
"waba-embedded-signup",
"waba-template-author"
],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": []
},
{
"id": "templates",
"url": "https://docs.sent.dm/llms/reference/api/template-definition.txt",
"kind": "documentation",
"affected_skills": ["sent-templates", "template-builder-ui", "waba-template-author"],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "template.body_max_length",
"extractor": "regex_int",
"pattern": "Body text[^\\n]*Maximum (\\d+) characters",
"expected": 1024
},
{
"fact": "template.button_total_limit",
"extractor": "regex_int",
"pattern": "maximum of (\\d+) buttons",
"expected": 10
}
]
},
{
"id": "webhooks",
Expand All @@ -40,35 +69,105 @@
"sender-profile-architect",
"waba-embedded-signup"
],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "webhook.message_delivered_event",
"extractor": "contains",
"pattern": "\"event\": \"message.delivered\"",
"expected": true
},
{
"fact": "webhook.template_event_omitted",
"extractor": "contains",
"pattern": "omitted for template events",
"expected": true
}
]
},
{
"id": "sender-profiles",
"url": "https://docs.sent.dm/llms/start/concepts/sender-profiles.txt",
"kind": "documentation",
"affected_skills": ["sender-profile-architect", "sms-10dlc-registration", "waba-embedded-signup"],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "profile.organization_scope_header",
"extractor": "contains",
"pattern": "x-profile-id",
"expected": true
},
{
"fact": "profile.rcs_agent_dedicated",
"extractor": "contains",
"pattern": "RCS Agent",
"expected": true
}
]
},
{
"id": "waba",
"url": "https://docs.sent.dm/llms/start/guides/connect-whatsapp.txt",
"kind": "documentation",
"affected_skills": ["waba-embedded-signup", "waba-template-author"],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "waba.id_field",
"extractor": "contains",
"pattern": "waba_id",
"expected": true
},
{
"fact": "waba.access_token_field",
"extractor": "contains",
"pattern": "access_token",
"expected": true
}
]
},
{
"id": "routing-rcs",
"url": "https://docs.sent.dm/llms/reference/channel-routing.txt",
"kind": "documentation",
"affected_skills": ["messaging-performance-analyzer", "rcs-agent-onboarding", "sent-messaging"],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "routing.multiple_explicit_channels",
"extractor": "contains",
"pattern": "broadcast list, not a fallback priority list",
"expected": true
},
{
"fact": "routing.rcs_supported",
"extractor": "contains",
"pattern": "Pins the message to RCS",
"expected": true
}
]
},
{
"id": "10dlc",
"url": "https://docs.sent.dm/llms/start/advanced/10dlc-registration.txt",
"kind": "documentation",
"affected_skills": ["sms-10dlc-registration"],
"last_verified": "2026-08-09"
"last_verified": "2026-08-09",
"checks": [
{
"fact": "10dlc.tcr_submission",
"extractor": "contains",
"pattern": "submit to TCR",
"expected": true
},
{
"fact": "10dlc.opt_in_consent",
"extractor": "contains",
"pattern": "consent",
"expected": true
}
]
}
]
}
Loading