[lucas-koontz]

User story

As an on-call engineer
I want quiet, correlated diagnostics
So that I can diagnose failures without exposing customer data

Why this matters

Operators can receive raw database and provider values through independent console, file and telemetry handlers. These diagnostic destinations now use a safe JSON envelope at WARNING. Final HTTP failures retain request correlation across the WSGI bridge.

Visual review

A deployed before/after capture remains a rollout gate for this runtime. Local regression and container evidence below establish behavior, not a production volume reduction.

Acceptance criteria

• Successful GET/HEAD /api/util/ping and the Docker /api/status health probe stay quiet even with access logging enabled; failed probes and non-probe methods remain visible.
• Shared requirements constrain SQLAlchemy to the 2.0 series required by the pinned Snowflake dialect.
• Owned diagnostics default to WARNING with a consistent JSON envelope and safe failure fields.
• Exception values, credentials, provider bodies and arbitrary extras are excluded from diagnostic output.
• SQL results, audit and HTTP behavior remain intact; a denied authentication candidate that later succeeds produces no final refusal.
• Override/reset and rollback behavior are documented in docs/structured-logging.md.
• Deployed versions, collector mappings, alerts, retained operational evidence and matched rollout windows are verified.

How to test

1. From an initialized checkout, Set MINDSDB_STORAGE_DIR to a temporary test directory, then run tests/unit/utilities/test_diagnostics.py and the existing sanitizer/configuration tests.
2. Exercise an ordinary success, a final refusal and a dependency or streaming failure. Check safe JSON and correlation, and confirm protocol/output contracts remain unchanged.
3. Enable the documented temporary verbosity/access override, repeat the checks, then reset it. Confirm failed health checks or operational failures remain visible at the quiet default.

Notes for the reviewer

• Keep deployment-specific level overrides. Specific MINDSDB logging settings retain precedence over the shared LOG_LEVEL fallback.
• Sanitize before telemetry rendering. Exception values and arbitrary extras are removed before the optional exporter renders them.
• The Snowflake import failure was dependency incompatibility. snowflake-sqlalchemy==1.9.0 imports an ORM interface removed in SQLAlchemy 2.1. The bound lives in the shared application requirements so installation and the requirements checker agree. Hosted full-suite validation and live collector export remain separate checks.
• Rollback uses the prior release and settings together. Do not disable audit, metric or protocol streams to quiet diagnostics. Live collector and alert validation remains a release gate.

Current review validation

Check	Observed result
Snowflake import regression	Reproduced the hosted ORMSelectCompileState failure with SQLAlchemy 2.1.5; the shared <2.1 bound resolves 2.0.55 and restores the real dialect import.
Final affected suites	54 passed across Snowflake handler, SQL rendering and diagnostics tests.
Health-probe regression	Failed before the fix because /api/status emitted access records; both GET/HEAD probe paths are quiet after the fix, while 503 and POST records remain visible.
Requirements / style	Full requirements duplicate/import/relative-import check passed in a clean source archive; configured Ruff check/format and diff checks passed.
Final image verification	The standard Dockerfile built successfully for arm64 from this commit with SQLAlchemy 2.0.55. Disposable container smoke passed the baked health command, quiet /api/status and /api/util/ping under access=all, visible WARNING/404, request-ID echo and private-query exclusion.
Hosted regression gate	The full hosted workflow passed static checks, package installation and unit tests on Linux, Windows and macOS for 075c4e888, clearing the previous Snowflake import failure. Deployment jobs were skipped.

Earlier validation

These results precede the dependency/probe follow-up:

Check	Observed result
Focused tests	10 passed: diagnostics, sanitizer and configuration checks.
Regression sensitivity	All three new diagnostic test functions failed when their protected behavior was removed; restored checks passed.
Historical hosted failure	The Linux CI slow handler suite failed importing snowflake.sqlalchemy with SQLAlchemy 2.1. The dependency fix and local verification are described above; live OTLP collector export remains unverified.
Pre-PR checks	Diff integrity and the repository sweep passed; regression mutations were restored.

Ships with

mindsdb/anton#621, mindsdb/cowork-server#741, mindsdb/cowork#1263, mindsdb/mindshub#12504.

Merge/release order: mindsdb/anton#621, then mindsdb/cowork-server#741, then mindsdb/cowork#1263. #5 can proceed independently. Update reviewed release pins and rebuild mindsdb/mindshub#12504 last.