Skip to content

fix: accept the spec's in-band error model for business-level failures#59

Open
vishkaty wants to merge 3 commits into
Universal-Commerce-Protocol:mainfrom
vishkaty:fix/error-model-55
Open

fix: accept the spec's in-band error model for business-level failures#59
vishkaty wants to merge 3 commits into
Universal-Commerce-Protocol:mainfrom
vishkaty:fix/error-model-55

Conversation

@vishkaty

Copy link
Copy Markdown

Fixes #55 (reported by @kbrackney): test_out_of_stock, test_product_not_found, and test_structured_error_messages hard-assert an HTTP 400, but the spec models business-level failures in-band — checkout-rest.md's own "Error Response" example answers an all-items-out-of-stock create with HTTP/1.1 200 OK, ucp.status: "error", and a typed messages[] entry (verified against current ucp main). A merchant implementing the spec's documented posture fails these tests today.

Follows the same pattern as #57 (accept both protocol-compatible outcomes), with each posture validated strictly rather than loosened:

  • 4xx keeps the exact assertions these tests always made against the Flower Shop reference (which uses this posture) — reference coverage is unchanged.
  • 200/201 requires messages[] to carry at least one type: "error" entry with the full message envelope (type, code, content, severity — all required by message_error.json), at least one entry using a standardized code for the scenario (checkout.md error-code table / error_code.json examples), and a ucp.status: "error" response to carry no checkout resource (checkout.md: "no resource is included in the response body").

Validation

  • Full 13-file suite against the samples Flower Shop (main, pip ucp-sdk): results identical before and after the change.
  • A minimal mock implementing the spec's canonical in-band example passes all three tests.
  • Injected defects each fail precisely: missing messages[], an error entry missing severity, a non-standard code, and a resource present on a status: "error" response.
  • ruff check + ruff format --check clean.

Fixes the posture mismatch reported in Universal-Commerce-Protocol#55: test_out_of_stock,
test_product_not_found, and test_structured_error_messages hard-asserted an
HTTP 400, but the spec models business failures in-band — checkout-rest.md's
own "Error Response" example answers an all-items-out-of-stock create with
HTTP 200, ucp.status "error", and a typed messages[] entry. A merchant
implementing the spec's documented posture failed these tests.

A shared assert_business_error helper now accepts either posture, each
validated strictly:

- 4xx keeps the exact assertions the tests always made against the Flower
  Shop reference (which uses this posture), so reference coverage is
  unchanged.
- 2xx requires messages[] to carry at least one type="error" entry with the
  full message envelope (type, code, content, severity — required by
  message_error.json), at least one entry using a standardized code for the
  scenario (checkout.md error-code table / error_code.json examples), and a
  ucp.status="error" response to carry no checkout resource (checkout.md:
  "no resource is included in the response body").

Validated: full 13-file suite results identical before and after against the
Flower Shop (main, pip ucp-sdk); a minimal mock implementing the spec's
canonical in-band example passes all three tests; injected defects each fail
precisely (missing messages[], missing severity, non-standard code, and a
resource on a status=error response).
@damaz91 damaz91 added status:needs-triage Signal that the PR is ready for human triage status:under-review and removed status:needs-triage Signal that the PR is ready for human triage labels Jul 16, 2026
Ensure that ucp.status is strictly 'error' and no resource ID is returned when a business-level failure is answered with a 2xx status code. This prevents false positives where a server might incorrectly create a resource but still return error messages.
@carolinerg1

Copy link
Copy Markdown

Hi @vishkaty, thanks for adding support for the 2xx in-band error posture!

Overall the changes look great, but it looks like test_update_inventory_validation in validation_test.py was missed. It currently still asserts self.assert_response_status(response, 400), which fails against servers returning a 2xx in-band error on checkout update requests (PUT /checkout-sessions/{id}).

Could you update test_update_inventory_validation to also use assert_business_error.

Thank you.

Converts the update-path twin the review caught: same dual-posture
assertion as the create-path tests, plus two strictness fixes to the
shared helper that surfaced while validating against an in-band mock:

- a resource-bearing in-band answer must not claim ucp.status 'error'
  (checkout.md: an error-status response includes no resource) and must
  report a valid non-'completed' checkout status;
- the update test also accepts the spec's canonical clamp-and-warn
  answer (checkout-rest.md Business Outcomes: 200 + quantity clamped +
  a 'quantity_adjusted' warning), requiring the quantity to actually be
  clamped so silent over-acceptance still fails.
@vishkaty

Copy link
Copy Markdown
Author

Thank you — you were right, and it's converted in 421008d: test_update_inventory_validation was the update-path twin of the same class.

Rather than fixing just the named site, I re-swept the file for the class. Accounting:

Test Status
test_out_of_stock, test_product_not_found, test_structured_error_messages dual-posture (this PR, unchanged)
test_update_inventory_validation converted in 421008d
test_payment_failure (hard 402) left as-is — though the spec defines no 402 anywhere (it's absent from checkout-rest.md's status-code table, and the OpenAPI declares only 200 for complete), and payment failures are modeled as payment_failed in messages[] (checkout.md Standard Errors; split-payments.md). Happy to convert here or in a follow-up — your call.
test_complete_without_fulfillment (hard 400 + exact reference prose) same question — checkout-rest.md's Complete Checkout section says field-existence expectations are set "via messages". Your call.

While validating the conversion against an in-band mock, two strictness gaps in the new helper surfaced; fixed in the same commit:

  • a resource-bearing answer claiming ucp.status: "error" now fails (checkout.md makes ucp.status the shape discriminator — an error-status response carries no resource), and the checkout status must be a valid non-completed enum value;
  • the update test also accepts the spec's other canonical answer to over-quantity — clamp-and-warn (checkout-rest.md "Business Outcomes": 200 + clamped quantity + a quantity_adjusted warning) — while requiring the quantity to actually be clamped, so silent over-acceptance still fails.

Validation: full validation_test.py green against the Flower Shop reference (posture unchanged), an in-band error-posture mock, and a clamp-posture mock; nine injected defects (the unconverted twin, silent over-acceptance, unclamped quantity_adjusted, completed/non-enum status, error-status-with-resource, missing severity, non-standard code, empty messages[]) each fail precisely.

One observation for a possible follow-up rather than this PR: checkout_lifecycle_test.py has five asserts that actively forbid the in-band posture for terminal-state operations (assertNotEqual(status_code, 200) on update/complete/cancel of canceled/completed checkouts) — same class. Happy to file separately if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: validation tests hard-assert HTTP 4xx for out-of-stock/unknown-item, rejecting the spec's in-band error_response model

4 participants