Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file.
169 changes: 169 additions & 0 deletions plugins/github-integration/ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# GitHub Integration Plugin — Architecture

## System Overview

```
┌─────────────────┐ Events ┌──────────────────┐
│ Paperclip │ ───────────────→│ Plugin Worker │
│ (Issue Mgmt) │←────────────────│ (This Plugin) │
└─────────────────┘ Webhooks └──────────────────┘
│ HTTP
┌──────────────┐
│ GitHub API │
│(Issues/PRs) │
└──────────────┘
```

## Data Flow

### Paperclip → GitHub (Push)

```
issue.created ──→ create GitHub issue ──→ store mapping
issue.updated ──→ update GitHub issue ──→ update sync state
issue.comment.created ──→ create GitHub comment
issue.status=done ──→ create branch ──→ create PR
```

### GitHub → Paperclip (Pull)

```
Webhook: issues ──→ update Paperclip status
Webhook: issue_comment ──→ create Paperclip comment
Sync Job (6hr) ──→ batch update all statuses
```

## State Storage

```
ctx.state (instance scope)
├── issue-mappings { paperclipId → githubNumber }
├── reverse-mappings { githubNumber → paperclipId }
├── sync-state { paperclipId → { githubUpdatedAt, paperclipUpdatedAt } }
└── last-sync timestamp
```

## Conflict Resolution

```
Paperclip update → check sync-state.githubUpdatedAt
→ if GitHub newer → skip (would overwrite)
→ if Paperclip newer → proceed

GitHub webhook → check sync-state.paperclipUpdatedAt
→ if Paperclip newer → skip
→ if GitHub newer → proceed
```

## Security

```
Webhook → verify HMAC-SHA256 signature
→ reject if mismatch
→ log warning if no secret configured

Secrets → resolve via Paperclip secret service
→ company-scoped with binding check
→ no keys exposed in logs/state
```

## Rate Limiting

```
GitHub API call → check X-RateLimit-Remaining
→ if 0 → wait until X-RateLimit-Reset
→ if 5xx → retry with exponential backoff (max 3)
```

## Component Diagram

```
┌─────────────────────────────────────────┐
│ Plugin Worker │
│ ┌─────────┐ ┌─────────┐ ┌────────┐ │
│ │ setup │ │onWebhook│ │ sync │ │
│ │ hook │ │ handler │ │ job │ │
│ └────┬────┘ └────┬────┘ └───┬────┘ │
│ └─────────────┴───────────┘ │
│ │ │
│ ┌────┴────┐ │
│ │ Octokit │ │
│ │ Client │ │
│ └────┬────┘ │
│ │ │
│ ┌────┴────┐ │
│ │ GitHub │ │
│ │ API │ │
│ └─────────┘ │
└─────────────────────────────────────────┘
```

## Module Structure

```
src/
├── manifest.ts # Plugin declaration (capabilities, tools, jobs, webhooks)
└── worker.ts # All logic:
# - setup() initializes Octokit + event handlers
# - onWebhook() handles GitHub events
# - sync job runs every 6 hours
# - tools exposed to agents
```

## Key Design Decisions

1. **Module-level state** — `onWebhook` runs outside `setup` context, so shared state (octokit, config, mappings) is stored in module-level variables set during `setup()`.

2. **Timestamp-based conflict resolution** — Last-write-wins prevents sync loops when both systems are edited simultaneously.

3. **Graceful degradation** — If GitHub token is missing, plugin logs warning and skips API calls. If webhook secret is missing, webhooks are accepted but logged.

4. **Empty PR branches** — PR creation creates branch from default branch SHA. Code push is intentionally left to human/agent (security boundary).

## Troubleshooting

### "Plugin running in degraded mode"
→ `githubTokenSecretRef` not configured or secret not found. Check:
1. Secret exists in Paperclip company
2. `company_secret_bindings` table has binding row
3. `defaultCompanyId` in plugin config matches company

### "Invalid secret reference"
→ Secret name doesn't exist or UUID is wrong. Check `secrets.getByName()` returns a result.

### "Secret is not bound to plugin"
→ Missing row in `company_secret_bindings`. Insert directly via DB:
```sql
INSERT INTO company_secret_bindings
(company_id, secret_id, target_type, target_id, config_path)
VALUES ('company-uuid', 'secret-uuid', 'plugin', 'plugin-uuid', 'plugin.secrets.resolve');
```

### "Cannot execute tool — worker not running"
→ `pluginDbId` mismatch between tool registry and worker manager. Check `plugin-tool-dispatcher.ts` passes `pluginDbId` to `registerPlugin()`.

### Webhook not updating Paperclip
→ Check webhook URL is correct and GitHub webhook secret matches `githubWebhookSecretRef`.

### Rate limit errors
→ Plugin handles automatically. If persistent, check GitHub PAT has sufficient quota (5000 req/hour for free accounts).

## Testing

```bash
# Unit tests
cd plugins/github-integration && npx vitest run

# Manual test — create issue
curl -X POST http://localhost:3100/api/plugins/tools/execute \
-H "Content-Type: application/json" \
-d '{"tool":"github-integration:github_create_issue","parameters":{"title":"Test"},"runContext":{"agentId":"...","companyId":"..."}}'

# Manual test — webhook
curl -X POST http://localhost:3100/api/plugins/github-integration/webhooks/github-webhook \
-H "Content-Type: application/json" \
-H "X-GitHub-Event: issues" \
-d '{"action":"closed","issue":{"number":1,"state":"closed"}}'
```
172 changes: 172 additions & 0 deletions plugins/github-integration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# GitHub Integration Plugin

Bidirectional sync between Paperclip issues and GitHub issues/PRs.

## Quick Start

### 1. Installation

The plugin is bundled with Levi. Enable it via the Paperclip board:

```
Board → Plugins → GitHub Integration → Install
```

### 2. Configuration

Required config fields:

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `githubRepo` | string | Yes | Repository in `owner/repo` format |
| `githubTokenSecretRef` | secret-ref | Yes | Secret reference for GitHub PAT |
| `defaultCompanyId` | string | Yes | Company ID for scoped operations |

Optional config fields:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `githubApiBase` | string | `https://api.github.com` | Override for GitHub Enterprise |
| `statusMapping` | string | `{"backlog":"open","done":"closed"}` | JSON mapping of statuses |
| `enablePrOnDone` | boolean | `false` | Auto-create PR when issue marked done |
| `githubWebhookSecretRef` | secret-ref | — | Secret for webhook signature verification |

### 3. Running

After configuration, the plugin auto-starts. Verify health:

```bash
curl http://localhost:3100/api/plugins/github-integration/health
```

## Webhook Setup

To receive GitHub events:

1. Go to your GitHub repo → Settings → Webhooks
2. Add webhook URL: `https://your-paperclip-instance/api/plugins/github-integration/webhooks/github-webhook`
3. Content type: `application/json`
4. Secret: matching `githubWebhookSecretRef` value
5. Events: Issues, Issue comments

## Manual Testing

### Test 1: Create GitHub issue from Paperclip

```bash
curl -s -X POST http://localhost:3100/api/plugins/tools/execute \
-H "Content-Type: application/json" \
-d '{
"tool": "github-integration:github_create_issue",
"parameters": {
"title": "Test issue",
"body": "Testing bidirectional sync"
},
"runContext": {
"agentId": "your-agent-id",
"runId": "your-run-id",
"companyId": "your-company-id",
"projectId": "your-project-id"
}
}'
```

Expected: `Created GitHub issue #N: https://github.com/owner/repo/issues/N`

### Test 2: Sync status

```bash
curl -s -X POST http://localhost:3100/api/plugins/tools/execute \
-H "Content-Type: application/json" \
-d '{
"tool": "github-integration:github_sync_status",
"parameters": {
"paperclipIssueId": "your-issue-id"
},
"runContext": {
"agentId": "your-agent-id",
"runId": "your-run-id",
"companyId": "your-company-id",
"projectId": "your-project-id"
}
}'
```

Expected: `GitHub issue #N is open/closed`

### Test 3: Webhook delivery

```bash
curl -s -X POST http://localhost:3100/api/plugins/github-integration/webhooks/github-webhook \
-H "Content-Type: application/json" \
-H "X-GitHub-Event: issues" \
-d '{
"action": "closed",
"issue": {
"number": 1,
"state": "closed",
"updated_at": "2024-01-01T00:00:00Z"
}
}'
```

Expected: Paperclip issue status updates to `done`

## Architecture

```
Paperclip Issue → Event Bus → Plugin Worker → GitHub API
↑ ↓
└────────── Webhook / Sync Job ←───────────────┘
```

## Capabilities Used

- `issues.read` / `issues.create` / `issues.update` — issue CRUD
- `issue.comments.create` / `issue.comments.read` — comment sync
- `plugin.state.read` / `plugin.state.write` — mapping persistence
- `events.subscribe` — Paperclip event handling
- `jobs.schedule` — periodic sync (every 6 hours)
- `http.outbound` — GitHub API calls
- `secrets.read-ref` — token resolution
- `webhooks.receive` — GitHub webhook handling
- `agent.tools.register` — agent tools

## State Storage

Plugin stores mappings in `ctx.state` with `instance` scope:

- `issue-mappings`: Paperclip ID → GitHub issue number
- `reverse-mappings`: GitHub issue number → Paperclip ID
- `sync-state`: Last updated timestamps for conflict resolution
- `last-sync`: Last sync job run timestamp

## Conflict Resolution

Timestamp-based last-write-wins:

- Paperclip → GitHub: skips if GitHub version is newer
- GitHub → Paperclip: skips if Paperclip version is newer
- Sync job: skips if Paperclip version is newer

## Rate Limiting

- Tracks `X-RateLimit-Remaining` header
- Waits until reset when limit exceeded
- Exponential backoff on 5xx errors (max 3 retries)

## Security

- Webhook signatures verified via HMAC-SHA256
- Secrets resolved via Paperclip secret service
- No API keys exposed in logs or state

## Limitations

- PRs are empty branches — code push requires human/agent
- No auto-merge capability
- Comment threading models differ (flat vs nested)

## License

MIT
Loading