Skip to content

fix(cli): SSL/gRPC/traceroute — valid assertion builders, honest reporter, rich checks get [SIM-286][SIM-287][SIM-291]#1391

Open
danielpaulus wants to merge 5 commits into
feat/cli-pertype-result-renderingfrom
daniel/sim-286-287-cli-fixes
Open

fix(cli): SSL/gRPC/traceroute — valid assertion builders, honest reporter, rich checks get [SIM-286][SIM-287][SIM-291]#1391
danielpaulus wants to merge 5 commits into
feat/cli-pertype-result-renderingfrom
daniel/sim-286-287-cli-fixes

Conversation

@danielpaulus

@danielpaulus danielpaulus commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Completes SSL/gRPC/traceroute CLI support on top of #1362: assertion constructs emit backend-valid payloads, checkly test output is honest, and checkly checks get shows rich per-type detail. Stacked on feat/cli-pertype-result-rendering (#1362).

What changed

Assertion constructs (SIM-287)

  • Traceroute responseTime() defaults property to avg + adds .avg()/.min()/.max()/.stdDev() (previously emitted an empty property the backend rejects → 400).
  • SSL baseline severity 'warn''degrade' (matches the backend enum).
  • Add greaterThanOrEqual + matches builders (for KEY_SIZE_BITS / TLS_VERSION / CIPHER_SUITE / ISSUER_CN) and SSL sources OCSP_STAPLED / HANDSHAKE_TIME_MS / SAN_CONTAINS, all round-tripped through codegen.
  • Shared degradedResponseTime ≤ maxResponseTime cross-field validation.

checkly test reporter (SIM-286)

  • No longer renders assertions as green ✔ under a request error (nothing was evaluated).
  • Humanized gRPC labels + response-time line; SSL response-time reason + security-baseline verdict.

checkly checks get --result (SIM-291)

  • Adds the missing ASSERTIONS section for SSL/gRPC/traceroute via a shared renderer used by both checkly test and checks get (can't drift).
  • Rich per-type detail: SSL CERTIFICATE (issuer/subject/validity/key/signature/SHA-256/SANs/serial/self-signed·CA/OCSP) + per-rule SECURITY BASELINE; gRPC TIMING (dns/connect/total); traceroute probe protocol + DNS timing.
  • --output json unchanged (full result body).

Testing

  • Unit: formatter (63) + reporter (36) tests; ssl-assertion-codegen, traceroute-assertion-codegen, grpc/ssl/traceroute-monitor specs; tsc clean. Reporter refactor is behavior-neutral (guarded by the unchanged reporter snapshot).
  • Live (local devenv + a prod account), all three types, pass + fail: deploy + checkly test render typed blocks and ✔/✖ assertions with Received: (no green-on-error); checks get --result shows per-type CERTIFICATE/BASELINE/TIMING/hop-table + the new ASSERTIONS section; checks list/stats/delete/trigger work for the new types; --output json emits the full body. Prod run = 3 pass / 3 fail as configured, real 11-hop trace.
  • Config coverage: 35 generated checks (14 SSL / 12 gRPC / 9 traceroute) exercising every property, config enum, and assertion source × comparator — all deploy clean.
🧪 Manually tested (live CLI runs)

Exercised end-to-end against a local devenv and a prod account, for SSL / gRPC / traceroute, pass + fail:

  • checkly test — typed result blocks + ✔/✖ assertions with Received:, no green-on-error (prod: 3 pass / 3 fail as configured; gRPC HEALTH carried; traceroute real 11-hop trace).

  • checkly checks get --result — SSL CERTIFICATE + per-rule SECURITY BASELINE, gRPC status/health/TIMING, traceroute hop table, all with the new ASSERTIONS section; also --stats-range / --group-by location.

  • checkly checks list (table, --output md, filters), checks stats, checks delete, trigger — all work for the three types.

  • checkly deploy — a 35-check exhaustive set (14 SSL / 12 gRPC / 9 traceroute) covering every property, enum, and assertion source × comparator deploys clean.

  • --output json — full result body for agents.

  • checkly import (reverse codegen) — generates valid SslMonitor / GrpcMonitor / TracerouteMonitor constructs from existing checks. This surfaced a backend gap where the import plan dropped the SSL/gRPC per-type config (both threw reading 'mode'/'hostname' before) — fixed in monorepo #2864 — then verified end-to-end on both a devenv and a prod account: the SSL/gRPC/traceroute constructs regenerate cleanly with full request.sslConfig/grpcConfig and assertions.

danielpaulus and others added 2 commits July 7, 2026 16:31
… gRPC/SSL detail [SIM-286]

Suppress the Assertions block when a result has a requestError (nothing was
evaluated); humanize gRPC assertion source labels; render gRPC response time;
add an SSL response-time reason line and a security-baseline verdict line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
…valid payloads [SIM-287]

responseTime() defaults property to 'avg' + avg/min/max/stdDev selectors;
SslBaselineSeverity 'warn' -> 'degrade'; add greaterThanOrEqual and matches
builders; add SSL sources OCSP_STAPLED/HANDSHAKE_TIME_MS/SAN_CONTAINS; add
degraded<=max validation. Codegen updated to round-trip all of the above.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
danielpaulus and others added 2 commits July 8, 2026 09:23
…et [SIM-291]

checkly checks get --result rendered the per-type RESULT block but omitted
the assertions (so a failing SSL/gRPC/traceroute check showed no reason) and
a lot of available detail. Bring it on par with checkly test and surface the
rich result body agents can use:

- Extract the assertion-line renderer into a shared formatters/assertion-line.ts
  used by BOTH the checkly test reporter and checks get, so they render
  identically and cannot drift.
- Add an ASSERTIONS section to the SSL/gRPC/traceroute detail formatters
  (terminal + markdown), including a Received value for falsy actuals (0/false/'').
- gRPC: add a TIMING breakdown (dns/connect/total).
- SSL: add a CERTIFICATE section (subject/issuer CN, validity, key, signature,
  SHA-256 fingerprint, SANs, serial, self-signed/CA, OCSP stapled) and a per-rule
  SECURITY BASELINE breakdown, keeping the one-line baseline summary.
- Traceroute: add probe protocol + DNS timing.
- Harden the shared truncate() to cap on the stringified length (objects/numbers
  no longer bypass it) and slice by code point (no split surrogate pairs); give
  the markdown request-error its own heading.

The --output json path already emits the full result body and is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
…M-291]

Adds the two branches the tri-model review flagged as untested: a scalar
securityBaseline rule (rendered as key: value) and selfSigned/isCA=true cert
flags. Test-only; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
@danielpaulus danielpaulus changed the title fix(cli): SSL/gRPC/traceroute reporter + assertion-builder fixes [SIM-286][SIM-287] fix(cli): SSL/gRPC/traceroute — valid assertion builders, honest reporter, rich checks get [SIM-286][SIM-287][SIM-291] Jul 8, 2026
…SIM-287]

GrpcMonitor capped degradedResponseTime/maxResponseTime at 30000 (JSDoc @maximum
and the validate() bounds), but gRPC calls run to the 180s timeout and the backend
allows up to 180000 (grpcResponseTimeLimitFields). Users couldn't set a gRPC
threshold above 30s. Raise the cap to 180000 to match the backend (and the
terraform provider fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant