Add paths-ignore support to CDS extractor and update dependencies

[data-douser]

What This PR Contributes

Resolves #326

Summary of Changes

This pull request adds support for respecting CodeQL paths-ignore patterns throughout the CDS extractor tooling. It introduces a new paths-ignore utility module, updates the dependency graph and file discovery logic to filter out ignored files and projects, and ensures that these patterns are also applied to the JavaScript extractor via environment variables. Additionally, the environment and test setup are updated to support and verify this new behavior.

Outline of Changes

Key changes include:

Support for CodeQL paths-ignore patterns:

• Added a new module paths-ignore.ts that locates, parses, and caches paths-ignore glob patterns from a CodeQL config file, and provides utilities to filter or match paths against these patterns.
• Updated the CDS file and project discovery logic in parser/functions.ts and parser/graph.ts to filter out files and projects matching paths-ignore patterns, with logging to indicate when files or projects are skipped. [1] [2] [3] [4]

Integration with JavaScript extractor environment:

• Added applyPathsIgnoreToLgtmFilters in environment.ts to append exclude: rules for each paths-ignore pattern to the LGTM_INDEX_FILTERS environment variable, so the JavaScript extractor also honors these exclusions. [1] [2]
• Updated codeql.ts to call this new function before running extraction. [1] [2]

Dependency and test updates:

• Updated dependencies in package.json to include js-yaml and minimatch for YAML parsing and glob matching, and added their type definitions.
• Added comprehensive unit tests for the new filtering logic and environment variable handling in environment.test.ts. [1] [2] [3]

These changes ensure that the extractor and associated tools consistently obey user-specified ignore patterns, improving accuracy and alignment with CodeQL configuration.

Future Works

N/A

[Copilot]

Pull request overview

This PR adds CodeQL paths-ignore support to the CDS extractor so that file/project discovery and downstream JavaScript extraction consistently skip user-ignored paths, aligning extractor behavior with CodeQL configuration.

Changes:

• Introduces a paths-ignore utility to locate and parse CodeQL config, cache patterns, and match/filter paths.
• Applies paths-ignore filtering during CDS project/file discovery and propagates excludes to the JavaScript extractor via LGTM_INDEX_FILTERS.
• Updates dependencies and adds/updates Jest tests to validate filtering and environment behavior.

Reviewed changes

Copilot reviewed 9 out of 14 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
extractors/cds/tools/src/paths-ignore.ts	New module to find CodeQL config, read/cache paths-ignore, and filter/match paths.
extractors/cds/tools/src/cds/parser/functions.ts	Filters discovered .cds files using paths-ignore patterns.
extractors/cds/tools/src/cds/parser/graph.ts	Skips entire CDS projects that match paths-ignore and skips projects emptied by filtering.
extractors/cds/tools/src/environment.ts	Adds helper to append paths-ignore patterns as exclude: rules in LGTM_INDEX_FILTERS.
extractors/cds/tools/src/codeql.ts	Calls the new environment helper before running JavaScript extraction.
extractors/cds/tools/test/src/paths-ignore.test.ts	New unit tests for config discovery, parsing, caching, and glob matching/filtering behavior.
extractors/cds/tools/test/src/environment.test.ts	Adds tests for applyPathsIgnoreToLgtmFilters and mocks paths-ignore.
extractors/cds/tools/test/src/codeql.test.ts	Updates mocks to include the new environment helper.
extractors/cds/tools/package.json	Adds js-yaml/minimatch and updates dev tooling versions.
extractors/cds/tools/package-lock.json	Lockfile updates for added/updated dependencies.

Files not reviewed (1)

• extractors/cds/tools/package-lock.json: Language not supported

---

You can also share your feedback on Copilot code review. Take the survey.