Skip to content

Code files that yield no symbols (data-only .js) enter the graph as isolated nodes, silently #3946

Description

@jnrod03-rgb

Routing is by extension: .js is code, code goes to the AST pass only, and
semantic extraction never sees it (Step B0 selects only
document/paper/image). For a pure data file, a literal array or object
with no functions, classes or imports, the AST finds nothing to extract. The
file enters the graph as a bare file node and its content contributes nothing.
Nothing in the flow reports it.

Related: #2214 (structured YAML/JSON data forced through the LLM, the opposite
routing problem), #728 (isolated dead-weight nodes from bundled/synthetic
files), #1692 / #3521 (unclassified files now surfaced; these files are
classified, so they are not covered), #3155 (file-level provenance for data
JSON).

Measured on a real corpus

After a full rebuild, 88 degree-0 nodes represented 3.10 MB and ~369,500
words, 39.5% of the corpus, with not one edge
. They were large domain data
files: question banks (~45,000 lines in total) and reference catalogues,
including a ~13,000-row code table. A three-paragraph README contributed more to
the graph than all of them, purely because of the file extension. The health
check passed, since a degree-0 node is not an integrity error.

Minimal reproduction (v0.9.65; identical on 0.9.61 and 0.9.55)

Seven one-line files, each holding the same two-element array of objects:

const BANK = [...];                // v_const.js
var BANK = [...];                  // v_var.js
window.BANK = [...];               // v_window.js
module.exports = [...];            // v_cjs.js
export default [...];              // v_esm_default.js
export const BANK = [...];         // v_esm_named.js
window.APP.bank = [...];           // v_ns.js

extract() followed by build_from_json():

graphifyy 0.9.65 graph nodes 9 edges 2
v_cjs.js           nodes=1 v_cjs.js(deg 0)
v_const.js         nodes=2 v_const.js(deg 1), BANK(deg 1)
v_esm_default.js   nodes=1 v_esm_default.js(deg 0)
v_esm_named.js     nodes=2 v_esm_named.js(deg 1), BANK(deg 1)
v_ns.js            nodes=1 v_ns.js(deg 0)
v_var.js           nodes=1 v_var.js(deg 0)
v_window.js        nodes=1 v_window.js(deg 0)

Five of the seven common ways to write a data file produce one isolated node.
The other two produce one node for the binding name. None carries anything from
the data itself.

Expected

A code file that yields no symbols should be reported, because "this file has
no symbols" and "this file is data the AST cannot read" look identical in the
output.

Proposal

  1. After Part A (SKILL.md), flag code files whose extraction produced only
    a file node (or a file node plus bindings with no functions/classes), and
    print their count and total size next to the AST summary:
    N code files yielded no symbols (X MB); they may be data rather than code.
  2. Offer to route those files through semantic extraction as documents
    (add them to the Part B file list), the same way video is re-routed to
    documents after transcription.
  3. In GRAPH_REPORT.md, give isolated file nodes their size. The report's
    "isolated nodes" section (reworked through GRAPH_REPORT.md counts more communities as "shown" than it renders #3548, GRAPH_REPORT.md: community Nodes listing includes rationale/concept nodes, so docstring fragments print as node names #3794 and GRAPH_REPORT.md offers 'undocumented components' as a possible cause on graphs that structurally cannot have one #3801 in
    v0.9.72) deliberately excludes file nodes, so a 400 KB data file with zero
    edges is invisible there. A large isolated file node is the signature of
    this failure; a small one usually is not.

Environment: observed on graphifyy 0.9.55, graphify install --platform claude,
Claude Code on Windows 11. Reproduction above re-run on 0.9.65; report.py
and the skill checked at v0.9.72.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions