[davior]

What

A conversation with the assistant can now be given material to work from:

• Files: text, Markdown, subtitles (.srt/.vtt), CSV/TSV, JSON, PDF (via pypdf), Word (.docx) and HTML. Attach them with the paperclip, or drop them anywhere on the panel.
• Pictures: JPEG, PNG, WebP or GIF, as a file or by URL. The model sees them as real images: a storyboard, a style reference, a diagram to explain. The chip shows a thumbnail.
• Notes from Gecko Notes: a picker that searches your notes as you type. Notes are read with the caller's own token, so Notes' ownership checks decide what can be read. BlockNote bodies are turned into Markdown with Notes' own block_markdown.py, copied into app/notes/.
• Web pages: the page is fetched and turned into readable text. Navigation, scripts and forms are dropped, and <article>/<main> is used when it has real content. A URL that points to a PDF, plain text or a picture works too.

How

• Text material is read once, when it is attached, and stored on assistant_session.attachments, a JSON TEXT column added by a new Alembic migration. Every later turn reads the same text the user saw listed. Nothing re-reads the note or page afterwards, which is D3's rule applied to attached notes.
• The text goes in as its own system block between REFERENCE and the project context. It stays the same for the whole conversation, so prompt caching covers it across turns. It is fenced in <attachment> tags and labelled as data, not instructions, and the text can't close its own fence.
• Pictures are rotated upright, scaled to at most 1568 px, and saved as JPEG (PNG where there is transparency) under data/assistant/<session>/, not in the database. A small inline thumbnail is kept for the chip. The files are deleted when the attachment or the conversation is removed. Each turn sends the pictures as image blocks at the very start of the conversation, ahead of the history (system prompts take text only), on both the Anthropic and OpenAI-compatible clients, so they stay in the cached prefix too. The text block lists them by number, and says they aren't in the library and so can't be used as visuals.
• Text-only models: pictures go only to a provider with "This model can look at images" turned on. Attaching one while a text-only model is active is refused with that reason. If the provider is switched to a text-only one later, the pictures stay listed but are marked as not shown.
• SSRF: app/web_fetch.py is a synchronous port of Notes' fetcher. Every redirect hop is checked against private address space after DNS resolution, and literal IPs are checked without a lookup. Only text, PDF and image content types are read, and the size is capped while the body streams in.
• Limits (in config.py):
  • 60k characters per item; anything longer is cut, and the chip says "cut short".
  • 150k characters and 12 items per conversation, of which at most 6 can be pictures.
  • 20 MB per file or page.
  • Pictures with too many pixels (decompression bombs) are refused.
  • Going over a limit is refused, never silently dropped.
• New endpoints:
  • POST /api/assistant/sessions: an empty conversation, so material can be attached before the first request.
  • POST /sessions/{id}/attachments/{file|note|url} and DELETE /sessions/{id}/attachments/{aid}.
  • GET /api/assistant/notes?q=, for the note picker.

Not done

• Video and YouTube URLs: getting captions from those is the job of GAM's URL import.

Testing

• pytest -q: 662 passed. This includes tests/test_assistant_attachments.py (text extraction, picture scaling and refusals, the API, Notes token forwarding, SSRF and redirect refusals, limits, file cleanup, and what reaches the model's prompt and images), new image-body tests in test_provider_conversations.py for both protocols, and the migration chain test.
• npm test: 155 passed, including panel tests for attaching a URL, a note and a picture. npm run build and npm run lint are clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_016K8ApooCKMPBVQiLjaWjcp