-
Notifications
You must be signed in to change notification settings - Fork 0
Daemon module #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| { | ||
| "permissions": { | ||
| "allow": [ | ||
| "Bash(pip install:*)", | ||
| "Bash(python -c \":*)", | ||
| "Bash(python -m pytest tests/ -v)", | ||
| "Bash(python -m pytest tests/ --cov=memsync --cov-report=term-missing -q)", | ||
| "Bash(python -m pytest tests/ -q)", | ||
| "Bash(memsync doctor:*)", | ||
| "Bash(python -m pytest tests/test_cli.py -v)", | ||
| "Bash(python -m pytest tests/test_cli.py::TestCmdDoctor::test_all_checks_pass_returns_0 -v -s)", | ||
| "Bash(python -m pytest tests/test_cli.py -q)", | ||
| "Bash(python -m pytest -m smoke -v)", | ||
| "Bash(ruff check:*)", | ||
| "Bash(bandit -r memsync/ -c pyproject.toml)", | ||
| "Bash(echo \"=== SMOKE \\(25 tests, <2s\\) ===\")", | ||
| "Bash(python -m pytest -m smoke -q --no-cov)", | ||
| "Bash(bandit -r memsync/ -c pyproject.toml -q)", | ||
| "Bash(python -m pytest tests/ -q --no-cov)", | ||
| "Bash(python -c \"from memsync.daemon import DAEMON_VERSION; print\\(''daemon __init__ ok, version:'', DAEMON_VERSION\\)\")", | ||
| "Bash(python -m pytest tests/ -x -q)", | ||
| "Bash(python -c \"import apscheduler; import flask; print\\(''OK''\\)\")", | ||
| "Bash(python -m pytest tests/test_daemon_notify.py::TestNotifyEmail::test_uses_env_var_password_over_config -v)", | ||
| "Bash(python -m pytest tests/test_daemon_notify.py::TestNotifyEmail::test_uses_env_var_password_over_config -v -s)", | ||
| "Bash(python -m pytest tests/test_daemon_scheduler.py::TestJobNightlyRefresh::test_skips_when_no_session_log -v -s)", | ||
| "Bash(python -m pytest tests/ -q --cov-report=term-missing)", | ||
| "Bash(python -m pytest tests/test_cli.py::TestDaemonCLIGuard::test_schedule_shows_jobs -v -s)", | ||
| "Bash(python:*)", | ||
| "Bash(grep -v \"^$\")", | ||
| "Bash(memsync refresh:*)", | ||
| "Bash(memsync config:*)" | ||
| ] | ||
| } | ||
| } |
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| --- | ||
| name: Bug report | ||
| about: Something isn't working | ||
| labels: bug | ||
| --- | ||
|
|
||
| **OS:** (macOS / Windows / Linux + version) | ||
|
|
||
| **Python version:** (e.g. 3.12.1) | ||
|
|
||
| **Provider:** (onedrive / icloud / gdrive / custom) | ||
|
|
||
| **`memsync status` output:** | ||
| ``` | ||
| (paste here) | ||
| ``` | ||
|
|
||
| **Error message:** | ||
| ``` | ||
| (paste full error here) | ||
| ``` | ||
|
|
||
| **Steps to reproduce:** | ||
| 1. | ||
| 2. | ||
| 3. | ||
|
|
||
| **Expected behavior:** | ||
|
|
||
| **Actual behavior:** |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| name: Provider request | ||
| about: Request support for a new cloud storage provider | ||
| labels: provider | ||
| --- | ||
|
|
||
| **Provider name:** (e.g. Dropbox, Box, Synology Drive) | ||
|
|
||
| **OS(es) where you use it:** (macOS / Windows / Linux) | ||
|
|
||
| **Default install path(s):** | ||
| - macOS: `~/...` | ||
| - Windows: `C:/Users/.../...` | ||
| - Linux: `~/...` | ||
|
|
||
| **Are you willing to implement it?** | ||
| Yes / No / Maybe — if yes, see [docs/adding-a-provider.md](../../docs/adding-a-provider.md) for a guide. | ||
|
|
||
| **Any other notes:** |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
| branches: [main] | ||
|
|
||
| jobs: | ||
| lint: | ||
| name: Lint & security | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Install dev dependencies | ||
| run: pip install -e ".[dev]" | ||
|
|
||
| - name: ruff (lint + style) | ||
| run: ruff check memsync/ | ||
|
|
||
| - name: bandit (security scan) | ||
| run: bandit -r memsync/ -c pyproject.toml | ||
|
|
||
| test: | ||
| name: Test (${{ matrix.os }}, Python ${{ matrix.python-version }}) | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest, windows-latest] | ||
| python-version: ["3.11", "3.12"] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Install dependencies | ||
| run: pip install -e ".[dev]" | ||
|
|
||
| - name: Run tests (with coverage) | ||
| run: pytest tests/ -v | ||
|
|
||
| - name: Smoke test | ||
| run: pytest -m smoke -v | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| name: Release | ||
|
|
||
| on: | ||
| push: | ||
| tags: | ||
| - "v*" | ||
|
|
||
| jobs: | ||
| publish: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| id-token: write # for PyPI Trusted Publishing (OIDC) | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.12" | ||
|
|
||
| - name: Build | ||
| run: | | ||
| pip install build | ||
| python -m build | ||
|
|
||
| - name: Publish to PyPI | ||
| uses: pypa/gh-action-pypi-publish@release/v1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,183 @@ | ||
| # ARCHITECTURE.md | ||
|
|
||
| ## System overview | ||
|
|
||
| ``` | ||
| User runs: memsync refresh --notes "..." | ||
| │ | ||
| ▼ | ||
| memsync/cli.py ← argument parsing, routes to commands | ||
| │ | ||
| ▼ | ||
| memsync/config.py ← loads ~/.config/memsync/config.toml | ||
| │ | ||
| ▼ | ||
| memsync/providers/<x>.py ← resolves sync root path for this machine | ||
| │ | ||
| ▼ | ||
| memsync/sync.py ← calls Claude API, merges notes into memory | ||
| │ | ||
| ▼ | ||
| memsync/backups.py ← backs up before writing | ||
| │ | ||
| ▼ | ||
| memsync/claude_md.py ← syncs GLOBAL_MEMORY.md → ~/.claude/CLAUDE.md | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## Module responsibilities | ||
|
|
||
| ### `memsync/cli.py` | ||
| - Entry point. Parses args, loads config, routes to command functions. | ||
| - Does NOT contain business logic — only wiring. | ||
| - Every command function signature: `def cmd_<name>(args, config) -> int` | ||
| - Returns exit code. Print errors to stderr, output to stdout. | ||
|
|
||
| ### `memsync/config.py` | ||
| - Loads and saves `~/.config/memsync/config.toml` (Mac/Linux) | ||
| or `%APPDATA%\memsync\config.toml` (Windows). | ||
| - Exposes a `Config` dataclass — no raw dicts passed around the codebase. | ||
| - Handles missing keys with sensible defaults. | ||
| - See `CONFIG.md` for full schema. | ||
|
|
||
| ### `memsync/providers/` | ||
| - `__init__.py` — defines `BaseProvider` ABC and `get_provider(name)` registry function. | ||
| - One file per provider: `onedrive.py`, `icloud.py`, `gdrive.py`, `custom.py`. | ||
| - Each provider implements `detect() -> Path | None` and `is_available() -> bool`. | ||
| - See `PROVIDERS.md` for full spec and all three implementations. | ||
|
|
||
| ### `memsync/sync.py` | ||
| - The only module that calls the Anthropic API. | ||
| - Takes: notes (str), current memory (str), config (Config). | ||
| - Returns: updated memory (str), changed (bool). | ||
| - Does NOT write files. Caller handles I/O. | ||
| - See `PITFALLS.md` — this module has the most trust/safety concerns. | ||
|
|
||
| ### `memsync/backups.py` | ||
| - `backup(source: Path, backup_dir: Path) -> Path` — copies with timestamp. | ||
| - `prune(backup_dir: Path, keep_days: int) -> list[Path]` — removes old backups. | ||
| - `list_backups(backup_dir: Path) -> list[Path]` — sorted newest-first. | ||
| - `latest_backup(backup_dir: Path) -> Path | None` | ||
|
|
||
| ### `memsync/claude_md.py` | ||
| - `sync(memory_path: Path, target_path: Path) -> None` | ||
| - `target_path` comes from `config.claude_md_target` — never hardcoded. | ||
| - Mac/Linux: create symlink if not already correct, backup any existing file first. | ||
| - Windows: copy (symlinks require admin rights on Windows). | ||
| - `is_synced(memory_path: Path, target_path: Path) -> bool` | ||
|
|
||
| --- | ||
|
|
||
| ## Data flow: `memsync init` | ||
|
|
||
| ``` | ||
| 1. cli.py — parse args | ||
| 2. config.py — check if config already exists (warn if --force not set) | ||
| 3. providers/ — run detect() on each registered provider in priority order | ||
| 4. cli.py — if multiple detected, prompt user to choose | ||
| 5. config.py — write config with chosen provider + detected path | ||
| 6. providers/ — call get_memory_root() to get the .claude-memory path | ||
| 7. (filesystem) — create .claude-memory/, backups/, sessions/ dirs | ||
| 8. (filesystem) — write starter GLOBAL_MEMORY.md if not exists | ||
| 9. claude_md.py — sync to ~/.claude/CLAUDE.md | ||
| 10. cli.py — print summary of what was created | ||
| ``` | ||
|
|
||
| ## Data flow: `memsync refresh` | ||
|
|
||
| ``` | ||
| 1. cli.py — parse args, read notes from --notes / --file / stdin | ||
| 2. config.py — load config | ||
| 3. providers/ — resolve memory root path | ||
| 4. (filesystem) — read current GLOBAL_MEMORY.md | ||
| 5. sync.py — call Claude API with current memory + notes | ||
| 6. sync.py — enforce hard constraints (append-only diff) | ||
| 7. backups.py — backup current file before overwriting | ||
| 8. (filesystem) — write updated GLOBAL_MEMORY.md | ||
| 9. claude_md.py — sync to ~/.claude/CLAUDE.md | ||
| 10. sessions/ — append notes to dated session log | ||
| 11. cli.py — print summary (changed/unchanged, backup path) | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## File layout on disk | ||
|
|
||
| ``` | ||
| # In cloud sync folder (synced across machines): | ||
| OneDrive/.claude-memory/ ← or iCloud/.claude-memory/, etc. | ||
| GLOBAL_MEMORY.md ← source of truth | ||
| backups/ | ||
| GLOBAL_MEMORY_20260321_143022.md | ||
| GLOBAL_MEMORY_20260320_091145.md | ||
| ... | ||
| sessions/ | ||
| 2026-03-21.md ← raw notes, append-only, never deleted | ||
| 2026-03-20.md | ||
| ... | ||
|
|
||
| # On each machine (not synced): | ||
| ~/.config/memsync/config.toml ← machine-specific config | ||
| ~/.claude/CLAUDE.md ← symlink → OneDrive/.claude-memory/GLOBAL_MEMORY.md | ||
| (or copy on Windows) | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## What does NOT belong in this tool | ||
|
|
||
| - Project-specific memory (that belongs in each project's CLAUDE.md) | ||
| - Cold storage / knowledge bases (use Hipocampus or RAG for that) | ||
| - Multi-user or team memory (out of scope for v1) | ||
| - Anything that requires a server, database, or API key beyond Anthropic's | ||
|
|
||
| --- | ||
|
|
||
| ## Futureproofing decisions | ||
|
|
||
| These are low-effort now and expensive to retrofit later. All three are | ||
| already reflected in the code specs above — this section explains the *why*. | ||
|
|
||
| ### 1. Version the memory file format | ||
|
|
||
| Write a version comment at the top of every `GLOBAL_MEMORY.md` when it's | ||
| first created: | ||
|
|
||
| ```markdown | ||
| <!-- memsync v0.2 --> | ||
| # Global Memory | ||
| ... | ||
| ``` | ||
|
|
||
| If the schema ever needs to change (section names, structure, anything), | ||
| the version comment lets migration code know what it's dealing with. | ||
| Without it, you can't distinguish an old file from a new one. | ||
|
|
||
| Implementation: write this comment in `load_or_init_memory()` when creating | ||
| the starter template. Check for it in `refresh_memory_content()` and warn | ||
| (don't fail) if it's missing. | ||
|
|
||
| ### 2. Don't hardcode the CLAUDE.md target path | ||
|
|
||
| `~/.claude/CLAUDE.md` is where Claude Code reads its global config today. | ||
| That could change. The target path lives in `config.claude_md_target` and | ||
| is never hardcoded anywhere in the logic modules. `cli.py` reads it from | ||
| config and passes it to `claude_md.sync()`. This is already reflected in | ||
| the `claude_md.py` module spec above. | ||
|
|
||
| ### 3. Keep the Anthropic SDK version loose | ||
|
|
||
| `pyproject.toml` already has `anthropic>=0.40.0` — keep it that way. | ||
| Never pin to an exact version. Users should get SDK updates automatically | ||
| when they upgrade their environment. | ||
|
|
||
| --- | ||
|
|
||
| ## Key constraints | ||
|
|
||
| - Python 3.11+ only. Use match statements, `Path` everywhere, `tomllib` (stdlib). | ||
| - No dependencies beyond `anthropic`. Everything else stdlib. | ||
| - `tomllib` is read-only (stdlib in 3.11+). Use `tomli_w` for writing, or write TOML | ||
| manually for the simple schema we have. See `CONFIG.md`. | ||
| - Must work offline except for `memsync refresh` (the only command needing the API). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the
testjob, dependencies are installed withpip install -e ".[dev]", but the test suite includestests/test_daemon_*modules that importmemsync.daemon.*at import time, which requires optional daemon packages like Flask/APScheduler. With the current install line, those imports fail withModuleNotFoundError, so CI will fail before executing the daemon tests; install daemon extras in this job (for example.[dev,daemon]) or exclude daemon tests from this matrix.Useful? React with 👍 / 👎.