Skip to content

Add Simili Bot for semantic issue triage #283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Kavirubc wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Add Simili Bot for semantic issue triage #283

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bde7fb0
feat: add simili bot configuration
Kavirubc Feb 12, 2026
1e93839
feat: update simili config and workflow per safety analysis
Kavirubc Feb 12, 2026
3681312
feat: add optional backfill script for existing issues
Kavirubc Feb 12, 2026
f98ef88
Merge branch 'main' into feature/simili-integration
Kavirubc Feb 12, 2026
eda70c6
Update scripts/simili-backfill.sh
Kavirubc Feb 12, 2026
08c1287
Update scripts/simili-backfill.sh
Kavirubc Feb 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/simili.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
qdrant:
url: "${QDRANT_URL}"
api_key: "${QDRANT_API_KEY}"
collection: "entireio-cli-issues"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Qdrant collection name is specified as "entireio-cli-issues" which doesn't align with the actual repository name "entireio/cli". While this may be intentional for the collection name, it's inconsistent with the repository naming convention used throughout the codebase. Consider using "entireio-cli-issues" or a name that clearly matches the repository for easier maintenance and debugging.

Suggested change
collection: "entireio-cli-issues"
collection: "entireio-cli"

Copilot uses AI. Check for mistakes.

embedding:
provider: "gemini"
api_key: "${GEMINI_API_KEY}"
model: "gemini-embedding-001"

defaults:
similarity_threshold: 0.75
max_similar_to_show: 5

transfer:
duplicate_confidence_threshold: 0.85

repositories:
- org: "entireio"
repo: "entireio-cli"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The repository name is specified as "entireio-cli" but based on the repository structure and other files (like scripts/install.sh which references "entireio/cli"), the correct repository name should be "cli". This mismatch will prevent the bot from correctly identifying and processing issues from this repository.

Suggested change
repo: "entireio-cli"
repo: "cli"

Copilot uses AI. Check for mistakes.
Comment on lines +19 to +20
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description repeatedly refers to the repository as "entireio-cli" and "entireio/entireio-cli", but the actual repository is "entireio/cli" (as evidenced by go.mod, README.md, scripts/install.sh, and all other documentation). This naming discrepancy appears in the configuration files and will cause the integration to fail. The description should be updated to reflect the correct repository name, and all configuration values should use "cli" as the repository name.

Copilot uses AI. Check for mistakes.
enabled: true
24 changes: 24 additions & 0 deletions .github/workflows/simili.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Simili Issue Triage

on:
issues:
types: [opened, reopened]

jobs:
triage:
runs-on: ubuntu-latest
permissions:
issues: write
contents: read
steps:
- uses: actions/checkout@v4
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow uses actions/checkout@v4 while other workflows in this repository use actions/checkout@v6. For consistency with the codebase conventions and to ensure the latest features and security fixes, this should be updated to v6.

Copilot uses AI. Check for mistakes.

- name: Run Simili Bot
uses: similigh/simili-bot@v0.1.5
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow uses a third-party GitHub Action (similigh/simili-bot@v0.1.5) with access to repository secrets (GEMINI_API_KEY, QDRANT_URL, QDRANT_API_KEY) and write permissions to issues. For security best practices, consider:

  1. Pinning to a specific commit SHA instead of a version tag to prevent supply chain attacks
  2. Reviewing the action's source code at the specified version
  3. Documenting the trust decision in the repository

This is especially important since the action has access to sensitive API keys that could be exfiltrated if the action is compromised.

Suggested change
uses: similigh/simili-bot@v0.1.5
# Pinned to a specific commit SHA for supply-chain security.
# Replace <COMMIT_SHA_FOR_V0_1_5> with the reviewed commit for v0.1.5.
uses: similigh/simili-bot@<COMMIT_SHA_FOR_V0_1_5>

Copilot uses AI. Check for mistakes.
with:
config_path: .github/simili.yaml
env:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
QDRANT_URL: ${{ secrets.QDRANT_URL }}
QDRANT_API_KEY: ${{ secrets.QDRANT_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding an automated issue triage system is a significant change that affects how contributors interact with the project, but there's no documentation update for this new workflow. Consider adding a section to CONTRIBUTING.md explaining:

  1. That issues will automatically receive triage comments from Simili Bot
  2. What the triage reports mean (similar issues, duplicate detection, suggested labels)
  3. That this is automated and not a human review

This helps set expectations for contributors who may be confused by automated comments on their issues.

Suggested change
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Explain automated triage to contributors
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const issue = context.issue;
if (!issue || !issue.number) {
core.info('No issue context available, skipping explanation comment.');
} else {
const body = [
"👋 This issue is being triaged automatically by **Simili Bot**.",
"",
"Simili Bot is an automated tool that helps maintainers by:",
"- Searching for **similar issues** that might already discuss this topic.",
"- Flagging potential **duplicates**.",
"- Suggesting **labels** based on the issue content.",
"",
"These comments and suggestions are **generated automatically** and **do not represent a human review or a maintainer decision**. A project maintainer may later follow up with additional comments, labels, or actions.",
].join("\n");
await github.rest.issues.createComment({
owner: issue.owner,
repo: issue.repo,
issue_number: issue.number,
body,
});
}

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +24
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description mentions that manual indexing is required after merge ("run simili index --repo entireio/entireio-cli"), but there's no automation or documentation for keeping the index updated. The PR also states that editing issues won't update the vector DB until manual re-indexing. This creates an operational burden where:

  1. The index will become stale over time as issues are edited
  2. Someone must remember to periodically re-run indexing
  3. There's no monitoring or alerting if indexing fails

Consider either:

  • Adding a scheduled workflow to periodically re-index (e.g., weekly)
  • Documenting the re-indexing procedure in a MAINTENANCE.md or similar file
  • Creating an issue to track the upcoming v0.2.0 feature that will handle this automatically

Without this, the integration may degrade in quality over time as the index becomes increasingly stale.

Copilot uses AI. Check for mistakes.
50 changes: 50 additions & 0 deletions scripts/simili-backfill.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/bin/bash
# Backfill script for applying Simili Bot triage to existing open issues
# Usage: ./scripts/simili-backfill.sh

set -euo pipefail

REPO="entireio/cli"
CONFIG=".github/simili.yaml"

echo "🔍 Fetching open issues from $REPO..."

# Fetch all open issues
gh issue list --repo "$REPO" --state open --json number,title,body,author,labels,createdAt --limit 1000 > /tmp/issues.json

ISSUE_COUNT=$(jq '. | length' /tmp/issues.json)
echo "📊 Found $ISSUE_COUNT open issues"

if [ "$ISSUE_COUNT" -eq 0 ]; then
echo "✅ No issues to process"
exit 0
fi

echo "🤖 Processing issues with Simili Bot..."

# Process each issue
jq -c '.[]' /tmp/issues.json | while read -r issue; do
NUMBER=$(echo "$issue" | jq -r '.number')
TITLE=$(echo "$issue" | jq -r '.title')

echo " Processing #$NUMBER: $TITLE"

# Create temporary issue file
echo "$issue" > "/tmp/issue-$NUMBER.json"

# Run simili process (requires simili CLI to be installed)
if command -v simili &> /dev/null; then
simili process --issue "/tmp/issue-$NUMBER.json" --config "$CONFIG" || echo " ⚠️ Failed to process #$NUMBER"
else
echo " ⚠️ simili CLI not found. Install with: gh extension install similigh/simili-bot"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The installation instruction suggests installing via gh extension install similigh/simili-bot, but this appears to be a GitHub CLI extension installation command for a bot that's actually a GitHub Action. The error message is misleading - users should instead follow the Simili Bot documentation for proper CLI installation. If the simili CLI tool has a different installation method, this message should reflect that.

Suggested change
echo " ⚠️ simili CLI not found. Install with: gh extension install similigh/simili-bot"
echo " ⚠️ simili CLI not found. Please install the simili CLI before running this script (see the Simili documentation for installation instructions)."

Copilot uses AI. Check for mistakes.
exit 1
fi

# Cleanup
rm "/tmp/issue-$NUMBER.json"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cleanup operation at line 44 will fail if the simili command fails on line 37 and exits due to set -e. The temporary file will not be removed in error cases, leading to leftover files in /tmp. Consider using a trap to ensure cleanup happens on exit, or move the cleanup to a finally-like construct.

Copilot uses AI. Check for mistakes.

# Rate limit: sleep 2s between issues
sleep 2
done

echo "✅ Backfill complete!"