Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
### 2.9.2 (Monday, August 10, 2026)
### Features/Bug Fixes
* fix(llm): retry malformed structured responses
---
### 2.9.1 (Monday, August 10, 2026)
### Features/Bug Fixes
* fix(llm): add bounded connection retries
Expand Down
48 changes: 48 additions & 0 deletions docs/release/skillspector-2.9.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# SkillSpector v2.9.2

Released: 2026-08-10

## Summary

This patch release makes structured LLM response handling more resilient to transient malformed payloads. It retries validation and structured-output parse failures with bounded backoff while retaining fail-closed batch isolation after the retry budget is exhausted.

## Highlights

- Structured LLM response failures now receive bounded retries before a batch is isolated.

## Added

- None.

## Changed

- Applied the structured-response retry policy centrally to semantic analyzers, the meta-analyzer, TP4, and gap-fill paths.

## Fixed

- Prevented transient malformed structured responses from immediately exhausting required LLM batches.

## Security

- None.

## Breaking Changes and Migration

- None.

## Deprecations

- None.

## Validation

- `git diff --check release/2.9.1..origin/main` — passed.
- Automated CI — lint, unit, integration, and Docker smoke checks passed; Sonar analysis succeeded.

## Known Limitations

- Requests still fail closed after the bounded retry budget is exhausted.

## References

- `CHANGELOG.md`
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "skillspector"
version = "2.9.1"
version = "2.9.2"
description = "SkillSpector: Security scanner for AI agent skills (Claude Code, Cursor, and similar). Scans skills for vulnerabilities, malicious patterns, and security risks before installation. Supports Git repos, URLs, zips, and local directories; runs static pattern checks and optional LLM semantic analysis; outputs terminal, JSON, and Markdown reports with risk scoring."
readme = "README.md"
license = "Apache-2.0"
Expand Down
2 changes: 1 addition & 1 deletion src/skillspector/inspection_ledger.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ class LedgerReason(StrEnum):
LedgerReason.SYNTAX_ERROR: "Python source could not be parsed.",
LedgerReason.LLM_BATCH_FAILED: "LLM analysis failed for this file range.",
LedgerReason.LLM_STRUCTURED_RESPONSE_INVALID: (
"LLM returned a malformed structured response after retry."
"LLM returned a malformed structured response after bounded retries."
),
LedgerReason.LLM_CONNECTION_RETRIES_EXHAUSTED: ("LLM connection failed after bounded retries."),
LedgerReason.ANALYZER_RUNTIME_ERROR: ("Analyzer failed after beginning applicable work."),
Expand Down
23 changes: 18 additions & 5 deletions src/skillspector/llm_analyzer_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,11 @@
logger = get_logger(__name__)

DEFAULT_MAX_LLM_CONCURRENCY = 10
STRUCTURED_RESPONSE_MAX_ATTEMPTS = 2
API_CONNECTION_MAX_RETRIES = 3
API_CONNECTION_RETRY_DELAYS_SECONDS = (0.5, 1.0, 2.0)
STRUCTURED_RESPONSE_MAX_RETRIES = 3
STRUCTURED_RESPONSE_MAX_ATTEMPTS = STRUCTURED_RESPONSE_MAX_RETRIES + 1
STRUCTURED_RESPONSE_RETRY_DELAYS_SECONDS = API_CONNECTION_RETRY_DELAYS_SECONDS
LLM_BATCH_MAX_ATTEMPTS = STRUCTURED_RESPONSE_MAX_ATTEMPTS + API_CONNECTION_MAX_RETRIES


Expand Down Expand Up @@ -624,11 +626,16 @@ def _invoke_batch_with_retries(self, batch: Batch, prompt: str) -> tuple[Batch,
or attempt == LLM_BATCH_MAX_ATTEMPTS
):
raise
delay = STRUCTURED_RESPONSE_RETRY_DELAYS_SECONDS[structured_retries]
structured_retries += 1
logger.warning(
"LLM structured response validation failed for %s; retrying once",
"LLM structured response validation failed for %s; retrying in %.2fs (%d/%d)",
batch.file_label,
delay,
structured_retries,
STRUCTURED_RESPONSE_MAX_RETRIES,
)
time.sleep(delay)
except Exception as exc:
if (
not _is_retryable_api_connection_error(exc)
Expand Down Expand Up @@ -685,11 +692,16 @@ async def _ainvoke_batch_with_retries(self, batch: Batch, prompt: str) -> tuple[
or attempt == LLM_BATCH_MAX_ATTEMPTS
):
raise
delay = STRUCTURED_RESPONSE_RETRY_DELAYS_SECONDS[structured_retries]
structured_retries += 1
logger.warning(
"LLM structured response validation failed for %s; retrying once",
"LLM structured response validation failed for %s; retrying in %.2fs (%d/%d)",
batch.file_label,
delay,
structured_retries,
STRUCTURED_RESPONSE_MAX_RETRIES,
)
await asyncio.sleep(delay)
except Exception as exc:
if (
not _is_retryable_api_connection_error(exc)
Expand Down Expand Up @@ -793,8 +805,9 @@ async def arun_batches(
native retry timing remains provider-managed. Unrecovered errors cost
only their own batch and are omitted from the result.
Malformed structured responses (Pydantic ``ValidationError`` or CLI
JSON parse failures) are retried once and then isolated to their batch.
A batch makes at most five outer chat-model invocations even when both
JSON parse failures) receive three bounded exponential-backoff retries
and are then isolated to their batch. A batch makes at most seven outer
chat-model invocations even when both
retry policies apply; native provider retries can make additional HTTP
requests within one invocation.
Callers can detect partial results by comparing the returned batches
Expand Down
Loading
Loading