Skip to content

📋 docs: add detailed scheduled tasks integration plan#12

Merged
AndreiDrang merged 8 commits into
mainfrom
vibe/scheduled-integration-plan-d1a46c
Jul 2, 2026
Merged

📋 docs: add detailed scheduled tasks integration plan#12
AndreiDrang merged 8 commits into
mainfrom
vibe/scheduled-integration-plan-d1a46c

Conversation

@AndreiDrang

Copy link
Copy Markdown
Owner

📋 Scheduled Tasks Integration Plan

Summary

Добавлены детальные планы интеграции фичи периодического обновления AGENTS.md файлов в zai-code-bot.

What's Added

1. SCHEDULED_TASKS_INTEGRATION_PLAN.md

Детальный технический план интеграции с:

  • Архитектурой решения
  • Детальным планом интеграции по фазам
  • Техническими деталями реализации
  • Критериями успеха
  • Вопросами и ответами
  • Полезными ресурсами

2. SCHEDULED_TASKS_QUICK_START.md

Быстрое руководство для пользователей с:

  • Быстрой настройкой за 3 шага
  • Примерами конфигурации
  • Руководством по устранению неполадок
  • Примерами cron-выражений
  • Советами и лучшими практиками

Why This PR

Этот PR адресует запрос пользователя на:

  1. Интеграция как отдельный хендлер - Документирована существующая архитектура
  2. Гибкая настройка расписания - YAML-конфигурация подробно описана
  3. Автоматическое выполнение и PR - Алгоритмы и потоки данных описаны
  4. Гибкость для будущих команд - Архитектура расширяемости документирована

Current Status

Фича уже реализована в этом форке! Эти документы:

  • Документируют существующую реализацию
  • Предоставляют план для улучшений и расширений
  • Служат справочником для пользователей
  • Помогают в будущем развитии

Next Steps

  1. Обсудить и утвердить план
  2. Реализовать улучшения из Фазы 2 (опционально)
  3. Добавить документацию в README.md
  4. Создать docs/scheduled-tasks.md

Related Files

  • src/lib/handlers/scheduled.js - Основной обработчик
  • src/lib/config/scheduled-config.js - Конфигурация
  • .zai-scheduled.yml - Конфигурация для этого репо
  • .zai-scheduled.yml.template - Шаблон для пользователей
  • .github/workflows/zai-agents-update.yml - Пример workflow

Note: Это draft PR для обсуждения. После утверждения планов можно будет приступить к реализации улучшений.

- Add SCHEDULED_TASKS_INTEGRATION_PLAN.md with comprehensive integration plan
- Add SCHEDULED_TASKS_QUICK_START.md with quick start guide
- Document existing scheduled tasks architecture
- Provide implementation phases and technical details
- Include examples and troubleshooting guide

This plan addresses the user request for:
1. Integration as separate handler
2. Flexible YAML-based scheduling configuration
3. Automatic execution and PR creation
4. Extensible architecture for future commands

Co-authored-by: AndreiDrang <AndreiDrang@users.noreply.github.com>
@codecov

codecov Bot commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.93191% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.00%. Comparing base (5a379a5) to head (fa974fe).

Files with missing lines Patch % Lines
src/lib/repository-context.js 98.64% 5 Missing ⚠️
src/lib/handlers/scheduled.js 98.24% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #12       +/-   ##
===========================================
+ Coverage   75.52%   86.00%   +10.48%     
===========================================
  Files          22       24        +2     
  Lines        6500     7204      +704     
===========================================
+ Hits         4909     6196     +1287     
+ Misses       1591     1008      -583     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown

Z.ai Code Review

🔍 Review Summary

This PR significantly enhances the "Scheduled Tasks" pipeline by implementing a "Grounded Generation" architecture to fix hallucination bugs (specifically PR #15). It introduces robust context collection (repository-context.js), validation guards (agents-validation.js), and extensive configuration scoping. The changes are well-structured, thoroughly tested, and the documentation is comprehensive.

🚨 Critical Issues & Bugs

  • src/lib/agents-validation.js: The hallucination detection logic in extractReferencedPaths ignores tokens that do not contain a slash (/) or a file extension (e.g., Dockerfile, Makefile, Procfile, LICENSE). If the LLM hallucinates one of these specific extensionless root files, the validation guard will skip checking them, allowing false positives to pass through the "hallucination check."

    • Why this is a problem: The core purpose of this PR is to prevent hallucinated files from being committed. This logic gap undermines that safety guarantee for common root-level configuration files.
    • Fix: Modify the filter logic to treat tokens that appear in file-list-like structures (e.g., after - or |) as potential file paths even if they lack extensions or slashes.
    // Inside extractReferencedPaths
    // Existing check
    if (GENERIC_TERMS.has(token)) continue;
    
    // Improved check: Allow extensionless tokens if they look like typical config filenames
    // or if they appear in a list context (simplified heuristic).
    const isExtensionless = !EXT_RE.test(token);
    const looksLikeConfigFile = /^(Makefile|Dockerfile|Procfile|Rakefile|Gemfile|Cargo\.toml|go\.mod|requirements\.txt|package\.json)$/i.test(token);
    
    if (token.includes('/') || EXT_RE.test(token) || looksLikeConfigFile) {
      found.add(token);
    }

💡 Suggestions & Best Practices

  • src/lib/repository-context.js: The globToRegExp function is a custom implementation. While it handles the basic cases in the PR (**, *), custom glob parsers can be fragile for edge cases (e.g., character classes [...], brace expansion {a,b}).
    • Suggestion: Add a comment explicitly documenting the supported glob syntax to prevent users from relying on unsupported features like src/**/*.{js,ts}. Alternatively, consider using a lightweight, well-maintained library like minimatch if the bundle size permits, to ensure RFC-compliant glob matching.
  • src/lib/handlers/scheduled.js: In handleUpdateAgentsTask, if the repository tree is truncated by the GitHub API (truncated: true), the context passed to the LLM is incomplete. However, the validation step (validateGeneratedAgentFiles) runs against this incomplete tree.
    • Suggestion: This creates a "Fail Closed" scenario where valid files in the truncated part might be flagged as hallucinations if the LLM infers them. This is generally safe (better to reject a good update than accept a bad one), but it might be noisy. Consider logging a specific warning if validation fails and the tree was truncated, so users understand the limitation.

📊 Final Assessment

  • Rating: Good
  • Reason: The PR successfully addresses the critical hallucination vulnerability with a multi-layered defense (Grounding + Validation). The code is clean, well-documented, and backed by solid unit tests. The identified issue with extensionless files is a refinement opportunity for the guardrail logic rather than a fundamental flaw.

@AndreiDrang
AndreiDrang marked this pull request as ready for review June 30, 2026 23:01
AndreiDrang and others added 7 commits July 1, 2026 03:10
Why:
* the scheduled pipeline existed but the AGENTS.md knowledge base was stale and referenced old symbol names

What:
* upgrade root AGENTS.md to document the three flows incl. schedule routing
* add scheduled module symbols and manual update-agents to nested AGENTS.md files
* flag the missing scheduled test coverage gap

Changes:
* AGENTS.md: overview, structure tree, code map symbols, fixed stale names
* src/lib/AGENTS.md: add events.js row
* src/lib/handlers/AGENTS.md: add update-agents + scheduled key symbols
* tests/AGENTS.md: add scheduled test gap

Stats:
* 4 files changed
* +46/-14 lines

Co-authored-by: opencode <noreply@opencode.ai>
Why:
* parseFileUpdatesFromResponse was internal-only, blocking unit coverage of the file-update parser

What:
* add parseFileUpdatesFromResponse to the scheduled module exports
* rebuild dist/index.js (CI enforces no dist drift)

Changes:
* src/lib/handlers/scheduled.js: export parseFileUpdatesFromResponse
* dist/index.js: ncc rebuild

Stats:
* 2 files changed
* +2 lines

Co-authored-by: opencode <noreply@opencode.ai>
Why:
* the scheduled pipeline had no test coverage (closes the plans Phase 4 gap)

What:
* cover the full config module at 100% (validation, defaults, task normalization, schedule dual-match, gist priority, loadScheduledConfig via mock octokit)
* cover scheduled handler pure functions (registry, buildExecutionContext, parseFileUpdatesFromResponse, createPR, executeScheduledTask, handler early-returns)

Changes:
* tests/scheduled-config.test.js: 44 config unit tests
* tests/handlers/scheduled.test.js: 21 handler unit tests

Stats:
* 2 files changed
* +690 lines

Co-authored-by: opencode <noreply@opencode.ai>
Why:
* the scheduled feature had no consumer-facing documentation (closes the plans Phase 3 gap)

What:
* add a full scheduled-tasks guide (quickstart, config reference, gist priority, cron syntax, troubleshooting)
* surface the feature in README: update-agents command, scheduled action inputs, and a Scheduled Tasks section

Changes:
* docs/scheduled-tasks.md: new consumer-facing guide
* README.md: add update-agents command, scheduled inputs, Scheduled Tasks section

Stats:
* 2 files changed
* +356/-1 lines

Co-authored-by: opencode <noreply@opencode.ai>
…validate output

Why:
* the update-agents task asked the model to scan the repo but passed no
  repo context, so it hallucinated a fictional project from the repo name
  (PR #15: invented a Python Telegram bot for a JavaScript GitHub Action).

What:
* collect real repository context (git tree, auto-discovered existing
  AGENTS.md files, key file contents) before calling Z.ai.
* build a grounded prompt that embeds that context and tells the model it
  has NO live repo access.
* validate generated output against the real tree before any PR: reject
  non-AGENTS paths, out-of-scope writes, and content referencing files
  that do not exist (hallucination guard).
* add optional scoping/budget config knobs (context_paths, target_paths,
  exclude_paths, max_context_chars, max_file_chars, max_files_to_fetch,
  allow_create_new, update_existing_only).

Changes:
* src/lib/repository-context.js: new module, collectRepositoryContext + renderRepositoryContext
* src/lib/agents-validation.js: new module, validateGeneratedAgentFiles + path/hallucination guards
* src/lib/config/scheduled-config.js: validateAgentsConfig for scoping/budget fields
* src/lib/handlers/scheduled.js: handleUpdateAgentsTask grounded flow + buildAgentsUpgradePrompt
* tests/repository-context.test.js: tree discovery, budgets, globs, truncation, failures (23)
* tests/agents-validation.test.js: path/target/hallucination guards incl. PR #15 regression (22)
* tests/scheduled-config.test.js: validateAgentsConfig coverage (+7)
* tests/handlers/scheduled.test.js: grounded handler flow incl. hallucination rejection (+5)

Stats:
* 8 files changed
* 693 tests passing (57 new)

Co-authored-by: Copilot <copilot@github.com>
Why:
* rebuild the ncc bundle to reflect the source changes in the previous commit.

What:
* regenerate dist/index.js from src/ so CI dist-drift gate passes.

Changes:
* dist/index.js: rebuilt bundle

Stats:
* 1 file changed

Co-authored-by: Copilot <copilot@github.com>
…knobs

Why:
* the knowledge-base and consumer template must reflect the new grounded +
  validated AGENTS.md upgrade flow and its optional scoping config.

What:
* document collectRepositoryContext, buildAgentsUpgradePrompt, and
  validateGeneratedAgentFiles in the knowledge base and handler guide.
* document the new .zai-scheduled.yml scoping/budget fields in the template.
* update the scheduled-pipeline test map (gaps resolved).

Changes:
* AGENTS.md: new modules, symbols, grounded-flow notes, anti-patterns
* src/lib/handlers/AGENTS.md: scheduled module symbols + test map
* tests/AGENTS.md: scheduled pipeline test coverage map
* .zai-scheduled.yml.template: context_paths/target_paths/exclude_paths/budgets/policy docs

Stats:
* 4 files changed

Co-authored-by: Copilot <copilot@github.com>
@AndreiDrang
AndreiDrang merged commit f116ac1 into main Jul 2, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants