[SECURITY] SC8 presence check does not fix incomplete skip denylist (CWE-184 / CWE-693) (same class as __pycache__ still applies elsewhere)

[nedlir]

#356 reported that skills can ship clean decoy .py sources plus malicious __pycache__/*.pyc (PEP 552 UNCHECKED_HASH) and still score SAFE, because discovery skips __pycache__ and analysis never inspects bytecode.

#357 closed that by adding SC8 (Shipped Python Bytecode): detect __pycache__ / .pyc / .pyo presence and floor the risk score to block install.

That is a useful mitigation for one known hiding place. It is not a fix for the root cause.

The root cause is an incomplete denylist of paths/artifacts that the scanner refuses to analyze, while the agent runtime can still load or execute content from those locations (or from similarly skipped locations).

CWE mapping:

• CWE-184: Incomplete List of Disallowed Items - the scanner maintains an explicit skip/denylist (_SKIP_DIRS, binary extensions, SC8’s own walk skips, etc.). SC8 only adds one more special case (__pycache__ / .pyc) to that list’s alarm path. Other denylisted items remain silently unscanned and unflagged.
• CWE-693: Protection Mechanism Failure - SkillSpector’s multi-layer analysis (static + optional LLM) is the protection mechanism. Anything discovery excludes never reaches those layers. Presence-alerting for one excluded class does not restore the protection for the rest.

SC8’s own walker still skips .git, node_modules, .venv, venv, .tox, .pytest_cache - so even the “fail closed on bytecode” path can miss bytecode nested under those trees. The same architectural pattern that made __pycache__ invisible still applies to other skipped names.

Why SC8 is incomplete

Before #357:

discover → skip __pycache__ → analyze decoy .py → SAFE
runtime  → load malicious .pyc → payload executes

After #357:

discover → still skip __pycache__ (contents never analyzed)
SC8      → see __pycache__ / .pyc exist → score floor → DO_NOT_INSTALL

What changed: an alarm on a specific denylist entry.
What did not change:

1. Bytecode contents are still not analyzed (no disassembly / no .py↔.pyc integrity check).
2. Other _SKIP_DIRS entries remain silent skips with no equivalent floor.
3. Hidden files / other out-of-scope artifacts remain a parallel evasion surface (same “not analyzed ⇒ not scored” property).
4. The fix is name-specific, not property-based (“runtime-reachable + unanalyzed ⇒ block”).

The next bypass does not need __pycache__. It only needs any other skipped or unscanned location the runtime (or the skill’s own loader) can still reach - the same class of failure as before SC8, with a different label on the folder.

Incomplete denylist (non-exhaustive)

These are illustrative of the same class, not a claim that each is an identical .pyc import smuggle today:

Skipped / unscanned surface	Why it matters
__pycache__ / .pyc	Addressed by SC8 presence only - contents still blind
.git	Still in skip lists; can be a file or tree; never sent to analyzers/LLM
node_modules	Still skipped by discovery and by SC8’s own walk
.venv / venv	Same
.tox / .pytest_cache	Same
Dotfiles / other OUT_OF_SCOPE paths	Never enter components; no SC8-style floor
Other binary / non-text artifacts the agent may execute or load	Hashed or skipped; not semantically analyzed

Hard-coding SC9 for node_modules, SC10 for .git, … repeats the denylist mistake. Attackers only need the next name that is skipped but still reachable.

Expected Behavior

The protection mechanism should be based on a durable invariant, for example:

1. Do not silently exclude runtime-reachable artifacts from analysis - or
2. Fail closed on any unanalyzed, potentially executable/loadable artifact (agnostic to __pycache__ vs node_modules vs .git), or
3. For Python specifically: analyze bytecode (disassemble / verify against companion .py), not merely detect that bytecode files exist.

Presence detection for one known cache directory is a stopgap, not that invariant.

Actual Behavior

• __pycache__ / .pyc → SC8 HIGH + score floor → install blocked (good for this PoC only).
• Same architectural skip for other denylist entries → still no analysis, still no equivalent block.
• Closing __pycache__ skipped entirely during skill scans (v2.8.2) (malicious skill bypasses this scanner) #356 as fixed overstates the remediation: the specific PoC is mitigated; the denylist failure class is not.

Suggested direction

Prefer property-based handling over growing SC* special cases:

IF artifact is excluded from content analysis
AND artifact is under the skill install/load path
AND artifact type is executable / importable / loader-reachable
THEN emit a blocking finding (or analyze it)

Alternatively, remove security-sensitive paths from silent skip lists and treat them as first-class scan inputs (with size/ledger guards as needed).

SC8 can remain as a Python-specific signal, but it should not be treated as closing the incomplete-denylist issue.

References

• Original report: #356
• Mitigation merged: #357
• PoC technique: https://github.com/nedlir/skills-scanner-bypass
• CWE-184 - Incomplete List of Disallowed Items
• CWE-693 - Protection Mechanism Failure

Environment

• Scanner: NVIDIA SkillSpector (post-fix(analyzers): HIGH SC8 when skill ships __pycache__ or .pyc #357 / SC8)
• Related versions: originally demonstrated on v2.8.2; SC8 does not remove the class of bug on current main

Checklist

• I have searched existing issues to avoid duplicates (follow-up to __pycache__ skipped entirely during skill scans (v2.8.2) (malicious skill bypasses this scanner) #356 / fix(analyzers): HIGH SC8 when skill ships __pycache__ or .pyc #357)
• I have included all requested information above
• This is about incomplete remediation / residual denylist failure, not a rehash of the raw __pycache__ PoC alone