From 12643601975290cffc8a7c1862ddf421ab8b3fd9 Mon Sep 17 00:00:00 2001 From: Renn F Date: Thu, 1 Oct 2026 21:39:12 +0200 Subject: [PATCH] release(4.3.0): the safety-corpus release - the spec 12 event bus with redis-backed metrics persistence, the spec 10 geo lifecycle, the spec 04 redos safety gates, the pattern_safety/events/redis_interop conformance runners, the 100% line coverage gate, and the hex-escape indicator parity backslash; guard-core 4.3.0 parity --- CHANGELOG.md | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c1008a2..3064bca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,60 @@ # Release Notes +v4.3.0 (2026-10-01) +------------------- + +The safety-corpus release: guard-core-php 4.3.0 tracks the guard-core 4.3.0 release. The engine gains the spec 12 event bus with dynamic rules and Redis-backed metrics persistence, the spec 10 geo download and refresh lifecycle, the spec 04 ReDoS safety gates over PCRE plus the performance monitor, and the three new spec 4.1.0 conformance runners (pattern_safety, events, redis_interop) with fail-closed xfail baselines; the whole suite now sits behind a 100%-line coverage gate (every executable line covered or provably unreachable), and the hex-escape attack indicator carries the reference's parity backslash again, unbreaking PHP 8.5's stricter PCRE2. + +Event bus, metrics collector, and dynamic rules (spec 12) +---------------------------------------------------------- + +### Added + +- **The spec 12 parity port: EventBus, EventFilter, MetricsCollector, SecurityMetric, DynamicRuleManager and DynamicRules land engine-side, with the SecurityConfig agent gating (`agent_enable_events` / `agent_enable_metrics`, both default true) and the new `dynamic_rules_cache_path` field.** Ported from the reference event-bus surface at the spec 12 parity commit (PR #43, cherry-pick of 136f35b); both new suites are registered in the CI test steps and the coverage collect list. Coverage lives in the new `bin/test_event_bus.php` honesty runner (136 checks at the port). +- **Metrics persist across requests via Redis, closing the last metrics gap: the spec 12 event bus is queueable, but MetricsCollector was in-memory only, so under the shared-nothing runtime model metrics emitted with no agent handler died with the request (PR #48).** The collector's queueing ports onto the reference key semantics (spec 08 namespaced grammar, full_key = prefix + namespace + ':' + key): with no agent handler and a wired RedisHandler the pending backlog lives under `{prefix}metrics:pending` as a JSON list of SecurityMetric payloads (timestamp ATOM, metric_type, value, tags). `setAgentHandler()` dispatches the in-memory queue, then flushes the persisted backlog in order and clears the key, and `drain()` absorbs the persisted backlog too, so FPM adapters can deliver it themselves. Failure arms stay fail-closed and quiet: a corrupted or non-list payload is discarded with a log (empty backlog, never raised), malformed entries are skipped individually with a log, and Redis write failures log and never raise; `agent_enable_metrics` and the EventFilter gate before anything persists, and a disabled RedisHandler falls back to the in-memory queue. `bin/test_event_bus.php` grows the persistence arms (150 -> 174 assertions): backlog accumulation across two sends, flush-on-attach with value/tags/timestamp round trip plus key clearing, drain absorption and empty-drain, redis write failure, corrupted/non-list/scalar pending payloads, malformed-entry skipping with tag coercion, the enable gate, and the disabled-redis fallback; every new MetricsCollector line is executed with a behavioral assertion. + +Geo download and refresh lifecycle (spec 10) +--------------------------------------------- + +### Added + +- **The spec 10 parity port: IpInfoManager owns the geo database lifecycle (PR #43, cherry-pick of 0fbbdb5).** Token download (3 attempts, exponential backoff), atomic writes, mtime freshness, the Redis-cached database copy, never-raising `get_country`, `check_country_access` verdicts with `country_blocked` events, `geo_lookup_failed` events, and the Redis-backed single-flight refresh lock. Coverage lives in the new `bin/test_geo_lifecycle.php` honesty runner (100 checks after the #44 coverage arms, 92 at the port) over the new shared `tests/MmdbFixture.php` fixture; master's `bin/test_geo_country.php` keeps its own superset inline builder, and the shared fixture serves the geo-lifecycle suite exclusively (each suite runs in its own PHP process, so no redeclare). + +Spec 04 ReDoS safety gates on PCRE, and the performance monitor +---------------------------------------------------------------- + +### Added + +- **The section 04 ReDoS safety gates port to the PHP engine's PCRE model, plus the performance monitor (spec 04) and the event constant table (PR #39, mandated by specs/impl/php.md's regex-engine section).** Structural prefilters (the three reference dangerous-construct regexes verbatim, the compile check, and the nested-unbounded / adjacent-broad structural checks over a source-only scan, outputs pinned case-by-case against the Python functions); reach-probe synthesis (a subject that reaches every quantified region, null synthesizing a fail-closed reject; the reference's subprocess probe and slot/interval machinery adapted per the spec's open-questions note to the observable contract: reject slow, fail closed, bounded time); the cost arbiter (load factor floor 0.25 / ceiling 8.0 measured against the reference scan, scaled verdict deadline, the 4000..32000 ladder, the min16/min32 verdict rule with noise floor 0.001, growth per doubling with extrapolation to the cap against the 0.05 s budget, one confirming retry, the interactive probe at 0.05 s per string and 2.0 s overall fail-closed, stride-sampled probe sets, and the accept-with-disagreement log); the scan guard (per-scan `pcre.backtrack_limit` via `ini_set` with the previous limit saved/restored, `PREG_BACKTRACK_LIMIT_ERROR`/`PREG_RECURSION_LIMIT_ERROR`/`PREG_JIT_STACKLIMIT_ERROR` trips classified as scan timeouts that fail closed while other preg failures keep the engine's fail-secure 500, an hrtime deadline, the consecutive-timeout counter and pool-replacement warning, and a canary probe - the arbiter's verdict arithmetic over doubling subject prefixes - before plain-pattern scans of large subjects); and the SusPatterns classification and timeout semantics (scan-window bounded patterns unguarded, windowed finders under the full compiler timeout, plain patterns behind the canary plus the reference's 0.9x-timeout heuristic arm, and the exact reference timeout semantics: a scan timeout emits a `pattern_timeout` threats-logged-and-miss threat with the category weight, so an engine that could not finish a scan fails closed; `detect()` now also returns `timeouts`). `PregFailure` carries the preg error code captured at throw time so classification never reads stale process-global state. Coverage lives in the new `bin/test_redos_gates.php` (175 checks: allow and block paths, gate-fire assertions, and the catastrophic fixtures `(a|a)*$`, `(.*a){20}$`, `(aa|a){20}$` over adversarial subjects tripping the engine budget and the deadline deterministically, hrtime-measured in the test) and `bin/test_performance_monitor.php`, both registered in the Makefile `RUNNERS` list, the CI test matrix and the coverage collection. + +Spec 4.1.0 conformance runners: pattern_safety, events, redis_interop +---------------------------------------------------------------------- + +### Added + +- **The three new reference corpus suite kinds are vendored and replayed with one runner per kind, mirroring the existing fail-closed xfail architecture (PR #46, fixes #45).** The vendored corpus gains `safety_gates.json`, `event_stream.json` and `redis_interop.json` plus the updated `index.json` (the new comparison blocks: events envelopes with volatile-field dropping, redis interop byte-equality rules, pattern-safety record rules). `bin/conformance_safety.php` runs every case through the real `Prefilters::validatePatternSafety` gate chain and maps verdict reasons to the corpus reason classes with the corpus generator's CLASS_RULES (first-match order), comparing `expected.safe` + `expected.reason_class` only. `bin/conformance_events.php` drives the real PHP event seams (the spec 12 pipeline bus, the monitor anomaly emitters, the geo event sink, the dynamic rules manager, the ban manager) and compares full envelopes per the events_envelopes block: expected keys only, volatile fields dropped recursively, fields the PHP envelope model cannot observe recorded as null. `bin/conformance_redis.php` performs one live-Redis operation per key family under each case's prefix (flushed before and after) with byte equality for keys, static values and TTL semantics, shape pinning for engine-generated zset members, and parsed-then-canonicalized JSON comparison. Each kind carries a fail-closed xfail baseline (`php_safety_xfail.json`, `php_events_xfail.json`, `php_redis_interop_xfail.json`): a failing case not baselined is red, a baselined case that passes is red (stale), and a driverless case not baselined is red, every baseline entry naming the PHP source of the divergence. Baselines at the merge: conformance_safety 69 passed / 25 xfail (unported parse-slot structural rules - ambiguous tail, unreachable terminator, literal absorb - PCRE2 atomic-group compile acceptance and auto-possessification, and the two timed cost-arbiter disagreements), conformance_events 6 passed / 32 xfail plus 1 advisory corpus skip (handler-level emitters not ported: sus_patterns, ban envelopes, security headers, redis manager, ip extraction, decorators, dynamic-rule violations, the check-hook metadata shape, and the blockEventType default mapping), conformance_redis 7 passed / 7 xfail (unported security-headers / custom-pattern / cloud-ranges registries, the verbatim vs latin-1 GeoIP database cache, and the ATOM vs Z snapshot timestamp). + +100% line coverage gate, and the process scaffold +-------------------------------------------------- + +### Added + +- **The suite sits behind a 100%-line coverage gate: every executable line covered or provably unreachable (PR #38 and the follow-up coverage campaign, closed together with #44's geo + events arms).** `.github/coverage-runner.php` collects pcov line coverage per suite in `--collect` mode (extra suite args such as test_state's `--integration` forwarded) and a `--merge` pass aggregates every collection and gates. Coverage climbed 80.56% -> 89.19% -> 98.86% -> 99.08% across the campaign (~1650, then ~1400, then ~100 test LOC): the new `bin/test_support.php` (Rx/CMatch regex primitives, LdapIpv4 legacy-IP decoding and injection windows, the Pickle opcode VM, XmlXxe matchers, Truncation regions/caps, Base64 candidates, Preprocessor decode edges, Semantic length guards, CanonicalIp edges), the full CloudFetchers parse/dispatch surface, the Azure HTML extraction chain, download retry/redirect/deadline flows, CloudManager store modes and the redis-handler refresh paths plus CurlHttpClient against a local `php -S` server in `bin/test_m4.php`, response-pattern edges and the redis failure matrix in `bin/test_behavior.php`, SecurityConfig validation and LogRedactor url edges in `bin/test_m3c.php`, and pipeline/body-form/geo-country arms for the last reachable lines. Two minimal src fixes make documented behavior reachable without changing semantics: the Preprocessor decode-budget flag is written through the caller's by-ref `array{bool}` (so `decode_budget_exhausted` can actually fire and PHP 8 stops raising warnings on every exhausted scan), and `BodyFormScan::filename()` catches the PHP 8 `ValueError` an unknown charset raises in `mb_convert_encoding` (the ported LookupError fallback was dead and an unknown charset crashed the scan). `.github/coverage-unreachable.php` is the machine-checked provably-unreachable inventory (each carried line with its proof inline as a comment) that `--merge` subtracts from the uncovered set, so the gate fails listing any uncovered line that has no proof; the inventory was re-audited for the spec 04 port (the suspatterns surrogate-escape waiver moves 122-123 to 147-148 with a corrected proof: the 8.3 coverage runner's mb_substr substitutes every ill-formed sequence, encoded surrogates included, while 8.2's mb_substr preserves those bytes and the branch is live there). `bin/integration_state.php` also lost a latent key leak between runs and its single-call SCAN assumption (the SCAN contract requires looping to cursor 0). +- **Process scaffold and the CI repairs (PR #37, plus the follow-up ci fix).** `SECURITY.md` (supported versions, advisory reporting), `CONTRIBUTING.md`, `CODE_OF_CONDUCT.md`, `.github/FUNDING.yml`, the PR template and the issue templates, modeled on the Go family and the fastapi-guard baseline; `.github/workflows/semgrep.yml`, the PHP-capable substitute after CodeQL proved it no longer supports PHP (`Did not recognize the following languages: php` at init), running the pinned semgrep/semgrep:1.177.0 container with the p/security-audit and p/secrets rulesets over src/ (path-filtered push/PR triggers plus a weekly Monday cron, metrics off); the ecosystem-gate engine dev stamp aligned 4.1.99 -> 4.2.99 with every adapter's ci / upstream-drift / scheduled-lint workflows; and the repair of the #39 merge's mangled composer-audit job boundary (duplicate name/runs-on keys inside the coverage job mapping made GitHub reject ci.yml at parse time with zero jobs on every run since the merge) with `bin/test_redos_gates.php` and `bin/test_performance_monitor.php` added to the CI test matrix and the coverage collection loop, so the new spec 04 src lines count toward the 100% line gate. + +Hex-escape attack indicator parity +----------------------------------- + +### Fixed + +- **One backslash restored in `Preprocessor::ATTACK_INDICATORS` (PR #50, fixes #49): the hex-escape indicator now carries the escaped-backslash sequence the Python reference uses (two backslash characters plus x in the runtime string), so it matches literal hex-escape text instead of feeding PCRE2 a bare `\x` escape.** The single backslash made PCRE2 see the `\x` escape, accidentally matching NUL-plus-hex on old PCRE2, nearly the opposite of the intent; PHP 8.5's bundled PCRE2 rejects the bare `\x` outright, so `extractAttackRegions` threw `PregFailure` on any such payload and `bin/test_binary_noise_gate.php` fataled (exit 2) on php 8.5.11 while the CI matrix (8.2-8.4) stayed green. On php 8.5.11 the gate runner is back to 80/80 (was a fatal), and `extractAttackRegions` regions literal hex-escape text plus adjacent injections, matching the reference's region behavior. + +### Verification + +- Full suite green on PHP 8.5.11 (local host php + composer, host Redis on 6379), mirroring the CI workflow step for step: `make lint` exit 0 (the `php -l` sweep with LINT_OK plus `composer validate --strict`), `make test` exit 0 over every runner in the Makefile `RUNNERS`/`REDIS_RUNNERS` lists, including the new `bin/conformance_safety.php`, `bin/conformance_events.php`, `bin/conformance_redis.php`, `bin/test_redos_gates.php` 175/175, `bin/test_performance_monitor.php`, `bin/test_geo_lifecycle.php` 100/100, and `bin/test_event_bus.php` 174/174. + +___ + v4.2.0 (2026-09-27) -------------------