Skip to content

refactor: semantic function clustering — dedup, shared helpers, and generics consolidation#38776

Merged
pelikhan merged 5 commits into
mainfrom
copilot/refactor-semantic-function-clustering
Jun 12, 2026
Merged

refactor: semantic function clustering — dedup, shared helpers, and generics consolidation#38776
pelikhan merged 5 commits into
mainfrom
copilot/refactor-semantic-function-clustering

Conversation

Copilot AI commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Implements all P1 and P2 recommendations from the semantic function clustering analysis (#38748): eliminates duplicate logic, consolidates scattered helpers, and introduces generics where extraction loops are structurally identical.

P1 — inline dedup

  • stringutil.Truncate reuse (completions.go, logs_format_compact.go, render.go): replaced 4 inline truncation snippets (if len(s) > N { s[:N-3] + "..." }) with the shared utility
  • parseEventsJSONLFileparseEventsJSONLMetrics (copilot_events_jsonl.go + callers): clarifies intent; avoids collision with the existing parseEventsJSONL (returns raw entries, not metrics)
  • FormatTokens in pkg/console (render.go, health_metrics.go, audit_cross_run_render.go): consolidated two near-identical abbreviateNumber implementations into one exported helper; removed private formatTokens
  • sliceutil.Deduplicate (cache_integrity.go): replaced manual seen-map + loop with the shared utility

P2 — shared helpers and generics

  • validateCapabilitySupport (agent_validation.go): extracted shared helper from validateMaxTurnsSupport / validateMaxContinuationsSupport, which shared identical "is feature set → is capability supported → return error" logic:
    func validateCapabilitySupport(featureName string, featureSet, capabilitySupported bool, engineID string) error
  • extractInlineSections[T] (inline_section_helpers.go): generic helper eliminates the duplicate collection loop in both ExtractInlineSubAgents and ExtractInlineSkills:
    func extractInlineSections[T any](markdown string, allStarts [][]int, makeItem func(name, content string) T) (mainMarkdown string, items []T)
  • GitHub URL extractor consolidation (repo.go): fallback git-remote parsing now delegates to parseGitHubRepoSlugFromURL (GHE-aware via getGitHubHost()) instead of duplicating inline SSH/HTTPS parsing with hardcoded github.com

Bonus

  • Fixed pre-existing perfsprint lint warning in action_resolver.go (fmt.Sprintf("/repos/github/gh-aw/commits/%s", ref) → string concatenation)

Copilot AI and others added 2 commits June 12, 2026 04:11
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…nlineSections generic, GitHub URL extractor consolidation, perfsprint fix

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Refactor semantic function clustering for pkg/ refactor: semantic function clustering — dedup, shared helpers, and generics consolidation Jun 12, 2026
Copilot AI requested a review from pelikhan June 12, 2026 04:37
@pelikhan pelikhan marked this pull request as ready for review June 12, 2026 13:54
Copilot AI review requested due to automatic review settings June 12, 2026 13:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors several duplicate / near-duplicate helpers across pkg/ by consolidating shared logic (string truncation, token formatting, inline section extraction, capability validation) and tightening some workflow/runtime configuration paths to be $HOME-relative.

Changes:

  • Replaces multiple ad-hoc truncation snippets with stringutil.Truncate, and centralizes token-count formatting as console.FormatTokens.
  • Extracts shared helpers: validateCapabilitySupport(...) for engine capability checks and generic extractInlineSections[T] for inline skill/sub-agent extraction loops.
  • Renames the Copilot events.jsonl metrics parser to parseEventsJSONLMetrics and updates callers/tests; updates a compiled workflow to use $HOME instead of hardcoded /home/runner.
Show a summary per file
File Description
pkg/workflow/cache_integrity.go Switches manual dedup logic to sliceutil.Deduplicate for canonicalization.
pkg/workflow/agent_validation.go Adds validateCapabilitySupport and refactors max-turn/max-continuations validators.
pkg/workflow/action_resolver.go Removes a fmt.Sprintf in favor of string concatenation for the API path.
pkg/parser/sub_agent_extractor.go Uses shared generic extractInlineSections for sub-agent extraction.
pkg/parser/inline_skill_extractor.go Uses shared generic extractInlineSections for skill extraction.
pkg/parser/inline_section_helpers.go Introduces extractInlineSections[T] helper.
pkg/console/render.go Uses stringutil.Truncate for max-len rendering and adds console.FormatTokens.
pkg/cli/repo.go Delegates git-remote slug parsing to parseGitHubRepoSlugFromURL.
pkg/cli/logs_metrics.go Updates to renamed parseEventsJSONLMetrics.
pkg/cli/logs_format_compact.go Uses stringutil.Truncate for compact log rendering.
pkg/cli/health_metrics.go Uses console.FormatTokens instead of local formatting helper.
pkg/cli/health_metrics_test.go Updates tests to call console.FormatTokens.
pkg/cli/copilot_events_jsonl.go Renames metrics parser to parseEventsJSONLMetrics.
pkg/cli/copilot_events_jsonl_test.go Updates tests to call parseEventsJSONLMetrics.
pkg/cli/completions.go Reuses stringutil.Truncate for completion descriptions.
pkg/cli/audit_expanded.go Updates to renamed parseEventsJSONLMetrics.
pkg/cli/audit_cross_run_render.go Switches token formatting to console.FormatTokens.
.github/workflows/smoke-copilot-aoai-entra.lock.yml Uses $HOME for Copilot config paths and adjusts env/export behavior.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

pkg/cli/copilot_events_jsonl_test.go:86

  • The test name and doc comment still say "TestParseEventsJSONLFile" even though the implementation under test was renamed to parseEventsJSONLMetrics. Renaming the test improves clarity and keeps grep/search results consistent with the new function name.
// TestParseEventsJSONLFile verifies that parseEventsJSONLMetrics correctly extracts
// turns, tool calls, tool sequences, and token counts from events.jsonl using
// the real Copilot CLI format (nested data object, tool.execution_start events).
func TestParseEventsJSONLFile(t *testing.T) {
	tests := []struct {
  • Files reviewed: 18/18 changed files
  • Comments generated: 3

Comment on lines 133 to +136
hasMaxTurns := engineConfig != nil && engineConfig.MaxTurns != ""

if !hasMaxTurns {
// No max-turns specified, no validation needed
return nil
}

agentValidationLog.Printf("Validating max-turns support: engine=%s, maxTurns=%s", engine.GetID(), engineConfig.MaxTurns)

// max-turns is specified, check if the engine supports it
if !engine.GetCapabilities().MaxTurns {
agentValidationLog.Printf("Engine %s does not support max-turns feature", engine.GetID())
return fmt.Errorf("max-turns not supported: engine '%s' does not support the max-turns feature", engine.GetID())
}

// Engine supports max-turns - additional validation could be added here if needed
// For now, we rely on JSON schema validation for format checking

return nil
agentValidationLog.Printf("Validating max-turns support: engine=%s", engine.GetID())
return validateCapabilitySupport("max-turns", hasMaxTurns, engine.GetCapabilities().MaxTurns, engine.GetID())
Comment on lines +144 to +147
hasMaxContinuations := engineConfig != nil && engineConfig.MaxContinuations != 0

// max-continuations is specified, check if the engine supports it
if !engine.GetCapabilities().MaxContinuations {
agentValidationLog.Printf("Engine %s does not support max-continuations feature", engine.GetID())
return fmt.Errorf("max-continuations not supported: engine '%s' does not support the max-continuations feature", engine.GetID())
}

return nil
agentValidationLog.Printf("Validating max-continuations support: engine=%s", engine.GetID())
return validateCapabilitySupport("max-continuations", hasMaxContinuations, engine.GetCapabilities().MaxContinuations, engine.GetID())
Comment thread pkg/cli/repo.go
Comment on lines +50 to 55
// Delegate to the shared helper which supports both HTTPS and SSH formats,
// including GitHub Enterprise hosts configured via getGitHubHost().
repoPath := parseGitHubRepoSlugFromURL(remoteURL)
if repoPath == "" {
return "", fmt.Errorf("remote URL does not appear to be a GitHub repository: %s", remoteURL)
}
@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel completed test quality analysis.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (106 new lines across pkg/cli, pkg/console, pkg/parser, and pkg/workflow) but does not have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/38776-consolidate-duplicated-logic-into-shared-utilities.md — review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next
  1. Review the draft ADR committed to your branch — it was generated from the PR diff and the linked clustering analysis ([refactor] Semantic function clustering: duplicate, outlier & scattered-helper consolidation across pkg/ #38748).
  2. Complete the missing sections — confirm the decision framing, refine the rationale, and verify the alternatives reflect what you actually weighed.
  3. Commit the finalized ADR to docs/adr/ on your branch.
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-38776: Consolidate Duplicated Logic into Shared Utilities and Generics

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

❓ Why ADRs Matter

Even consolidation/refactoring work encodes a decision: which abstraction becomes the single source of truth, and what coupling that introduces. ADRs create a searchable, permanent record of why the codebase looks the way it does, so future contributors understand the reasoning behind the shared helpers and generics introduced here.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 38776-consolidate-duplicated-logic-into-shared-utilities.md).

🔒 This gate is advisory: enforcement was triggered by code volume in business-logic directories (>100 new lines), not an implementation label.

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · 84 AIC · ⌖ 9.79 AIC · ⊞ 13.8K ·

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot run pr-finisher skill

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 83/100 — Excellent

Analyzed 4 modified test function(s): 4 design (behavioral contracts), 0 implementation, 0 guideline violations. All changes are purely mechanical refactoring — renaming the callee from parseEventsJSONLFileparseEventsJSONLMetrics and formatTokensconsole.FormatTokens.

📊 Metrics & Test Classification (4 tests analyzed)
Metric Value
New/modified tests analyzed 4
✅ Design tests (behavioral contracts) 4 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 3 (75%)
Duplicate test clusters 0
Test inflation detected ⚠️ Technical flag (refactoring artifact — see note)
🚨 Coding-guideline violations 0

Test Classification Details

Test File Classification Issues Detected
TestParseEventsJSONLFile pkg/cli/copilot_events_jsonl_test.go:85 ✅ Design None — 6-case table driven, 2 explicit error rows
TestParseEventsJSONLFile_RealArtifact pkg/cli/copilot_events_jsonl_test.go:189 ✅ Design No error case (real-artifact validation; acceptable)
TestParseEventsJSONLFile_TBT pkg/cli/copilot_events_jsonl_test.go:291 ✅ Design None — covers single-turn and zero-interval edge cases
TestFormatTokens pkg/cli/health_metrics_test.go:225 ✅ Design None — covers zero/small/K/M boundary cases

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 4 tests — unit (//go:build !integration)
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Test Inflation Note

copilot_events_jsonl_test.go has 6 raw line additions vs 2 in the production file (ratio 3:1 > 2:1 threshold). This is a refactoring artifact: the test calls the renamed function in 6 places while the production file only needed 2 renames. There is no genuine test inflation — the net diff for both files is zero growth.

Verdict

Check passed. 0% of modified tests are implementation tests (threshold: 30%). No coding-guideline violations detected. All test changes correctly track the refactored function signatures.

📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

References: §27420151282

🧪 Test quality analysis by Test Quality Sentinel · 373.6 AIC · ⌖ 27.5 AIC · ⊞ 28.3K ·

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 83/100. Test quality is excellent — 0% of modified tests are implementation tests (threshold: 30%). All test changes correctly track the refactored function signatures (parseEventsJSONLFileparseEventsJSONLMetrics, formatTokensconsole.FormatTokens). No coding-guideline violations detected.

… parseGitHubRepoSlugFromURL

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

@copilot run pr-finisher skill

Done. Three review threads addressed in commit fix: guard debug logs on feature-set check; add ssh:// URL support to parseGitHubRepoSlugFromURL:

  1. Noisy debug log in validateMaxTurnsSupport — log is now guarded by if hasMaxTurns, matching the original early-return behaviour.
  2. Noisy debug log in validateMaxContinuationsSupport — same fix; log only fires when the feature is actually set.
  3. ssh:// URL regression in parseGitHubRepoSlugFromURL — added a third branch handling ssh://[git@]<host>/<owner>/<repo> (GHE-aware). Added test cases for ssh://git@github.com/…, without user info, and with/without .git suffix.

@pelikhan pelikhan merged commit 3426f8b into main Jun 12, 2026
29 checks passed
@pelikhan pelikhan deleted the copilot/refactor-semantic-function-clustering branch June 12, 2026 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants