Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@ __pycache__/
*.pyc
*.pyo
*.csv
*.xlsx
*.log
.DS_Store
63 changes: 53 additions & 10 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,24 @@ The tool performs three main types of assessments:

```
GitHubAssessment/
├── security_assessment.py # Main: Repository security controls assessment
├── idp_assessment.py # Main: Identity & access management assessment
├── assess_copilot_repos.py # Main: GitHub Copilot best practices validation
├── list_repos_gh_cli.py # Utility: Basic repository listing
├── list_repos_gh_cli_optimized.py # Utility: Optimized repository listing with Copilot checks
├── list_and_check_repos.py # Utility: Combined listing and Copilot directory checking
├── .gitignore # Excludes .venv, __pycache__, *.csv, *.log
├── README.md # Human-readable documentation
└── AGENTS.md # This file - LLM agent documentation
├── security_assessment.py # Main: Repository security controls assessment
├── idp_assessment.py # Main: Identity & access management assessment
├── assess_copilot_repos.py # Main: GitHub Copilot best practices validation
├── validate_custom_instructions.py # Main: Custom Instructions existence & size validation
├── test_validate_custom_instructions.py # Tests: Automated tests for validate_custom_instructions.py
├── list_repos_gh_cli.py # Utility: Basic repository listing
├── list_repos_gh_cli_optimized.py # Utility: Optimized repository listing with Copilot checks
├── list_and_check_repos.py # Utility: Combined listing and Copilot directory checking
├── .gitignore # Excludes .venv, __pycache__, *.csv, *.xlsx, *.log
├── README.md # Human-readable documentation
└── AGENTS.md # This file - LLM agent documentation
```

### Generated Files (Excluded from Git)
- `github_security_assessment_YYYYMMDD_HHMMSS.csv` - Security assessment reports
- `github_idp_assessment_YYYYMMDD_HHMMSS.csv` - IDP assessment reports
- `github_copilot_assessment_YYYYMMDD_HHMMSS.csv` - Copilot assessment reports
- `custom_instructions_violations_YYYYMMDD_HHMMSS.xlsx` - Custom Instructions violations report
- `.venv/` - Python virtual environment directory

## Codebase Architecture
Expand Down Expand Up @@ -164,7 +167,47 @@ Instructions Dir, Instructions Count, Agents Dir, Agents Count, Collections Dir,
Collections Count, Scripts Dir, Scripts Count, Overall Copilot Status, Recommendations, Errors
```

#### 4. Utility Scripts
#### 4. validate_custom_instructions.py

**Purpose**: Validates that GitHub Copilot Repository Custom Instructions files exist and comply
with the 4,000-character limit defined in the GitHub Copilot documentation.

**Reference**: https://docs.github.com/en/enterprise-cloud@latest/copilot/concepts/prompting/response-customization?tool=webui#about-repository-custom-instructions

**Key Functions**:
- `check_gh_installed()` - Validates GitHub CLI
- `check_rate_limit()` - Monitors API rate limits
- `run_gh_command(command)` - Executes GitHub CLI commands returning JSON
- `fetch_repositories()` - Fetches all repositories with parallel execution
- `decode_file_content(api_response)` - Decodes base64-encoded file content from GitHub API
- `get_custom_instruction_files(repo_name)` - Retrieves all custom instruction files for a repo
- `assess_repo(repo)` - Main assessment for a single repository
- `check_all_repositories(repos)` - Parallel assessment of all repositories
- `validate_existence(results)` - Checks that at least one custom instruction file exists
- `validate_file_sizes(results)` - Checks all files are within the 4,000-character limit
- `export_violations_to_excel(violations)` - Exports violations to an Excel file
- `main()` - Entry point; halts with exit code 1 when a validation fails

**Files Checked**:
- `.github/copilot-instructions.md` - Primary repository-level custom instruction file
- `.github/instructions/*.instructions.md` - Additional instruction files

**Validation Rules**:
1. **Existence**: At least one custom instruction file must be present
2. **Size**: Each file must not exceed 4,000 characters

**Output**:
- Console: Pass/Fail per validation rule with GitHub Copilot rule references
- Excel: `custom_instructions_violations_YYYYMMDD_HHMMSS.xlsx` (only when violations found)

**Exit Codes**:
- `0` - All validations passed
- `1` - One or more validations failed

**Dependencies**:
- `openpyxl` (optional; falls back to CSV if not installed): `pip install openpyxl`

#### 5. Utility Scripts

**list_repos_gh_cli.py** (143 lines):
- Basic repository listing using GitHub CLI
Expand Down
79 changes: 63 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,50 @@ python assess_copilot_repos.py
- Overall Copilot readiness status
- Recommendations for missing components

### 2. Security Assessment
### 2. Custom Instructions Validation

Validates that GitHub Copilot Repository Custom Instructions files exist and comply with the
4,000-character limit imposed by GitHub Copilot.

> 📖 Reference: [GitHub Copilot Repository Custom Instructions](https://docs.github.com/en/enterprise-cloud@latest/copilot/concepts/prompting/response-customization?tool=webui#about-repository-custom-instructions)

**Install the required library:**
```bash
pip install openpyxl
```

**Run the validation:**
```bash
python validate_custom_instructions.py
```

**What it checks:**
- **Existence** – at least one of the following files must be present in a repository:
- `.github/copilot-instructions.md`
- `.github/instructions/*.instructions.md`
- **Size limit** – each custom instruction file must not exceed **4,000 characters**

**Validation flow:**
1. Fetches all accessible repositories
2. Checks each repository for custom instruction files
3. **Fails immediately** if no custom instruction files are found across all repositories
4. **Fails immediately** if any file exceeds the 4,000-character limit and generates an Excel report

**Output:**
- Console summary with ✅/❌ validation results per rule
- Excel file `custom_instructions_violations_YYYYMMDD_HHMMSS.xlsx` (only when violations are found)
- Lists every violating file with its repository, path, character count, and excess characters

**Exit codes:**
- `0` – All validations passed
- `1` – One or more validations failed

**Running tests:**
```bash
python test_validate_custom_instructions.py -v
```

### 3. Security Assessment

Evaluates repository-level security controls across all accessible repositories.

Expand Down Expand Up @@ -186,7 +229,7 @@ python security_assessment.py
- Overall security compliance status
- Error details (if any)

### 3. Identity & Access Management (IDP) Assessment
### 4. Identity & Access Management (IDP) Assessment

Evaluates organization-level identity, authentication, and access controls.

Expand Down Expand Up @@ -296,20 +339,23 @@ This tool supports assessment alignment with:

```
GitHubAssessment/
├── security_assessment.py # Repository security controls assessment
├── idp_assessment.py # Identity & access management assessment
├── assess_copilot_repos.py # GitHub Copilot best practices validation
├── list_repos_gh_cli.py # Repository listing utility
├── list_repos_gh_cli_optimized.py # Optimized repository listing
├── list_and_check_repos.py # Combined listing and checking
├── MCP/ # Model Context Protocol configurations
├── .venv/ # Python virtual environment
├── .gitignore # Excludes .venv, CSV files, logs
├── github_security_assessment_*.csv # Generated security reports
├── github_idp_assessment_*.csv # Generated IDP reports
├── github_copilot_assessment_*.csv # Generated Copilot reports
├── README.md # This file - Human-readable documentation
└── AGENTS.md # LLM/AI agent documentation for code assistance
├── security_assessment.py # Repository security controls assessment
├── idp_assessment.py # Identity & access management assessment
├── assess_copilot_repos.py # GitHub Copilot best practices validation
├── validate_custom_instructions.py # Custom Instructions existence & size validation
├── test_validate_custom_instructions.py # Automated tests for the above
├── list_repos_gh_cli.py # Repository listing utility
├── list_repos_gh_cli_optimized.py # Optimized repository listing
├── list_and_check_repos.py # Combined listing and checking
├── MCP/ # Model Context Protocol configurations
├── .venv/ # Python virtual environment
├── .gitignore # Excludes .venv, CSV files, Excel files, logs
├── github_security_assessment_*.csv # Generated security reports
├── github_idp_assessment_*.csv # Generated IDP reports
├── github_copilot_assessment_*.csv # Generated Copilot reports
├── custom_instructions_violations_*.xlsx # Generated violations reports
├── README.md # This file - Human-readable documentation
└── AGENTS.md # LLM/AI agent documentation for code assistance
```

## Output and Reports
Expand Down Expand Up @@ -507,6 +553,7 @@ gh config list
- [x] Security assessment with parallel execution
- [x] IDP assessment with Enterprise SSO support
- [x] Copilot best practices validation
- [x] Custom Instructions existence & size validation with Excel report
- [x] CSV export with timestamped files
- [ ] Advanced analytics dashboard
- [ ] Trend analysis across multiple assessments
Expand Down
Loading