Conversation
There was a problem hiding this comment.
Pull request overview
This PR continues the FastAPI/Jinja2/HTMX web UI work by expanding sidebar UX, adding session search, and enhancing the RAG manager with upload + collection creation flows, plus richer diagnostics/token budgeting.
Changes:
- Add session history search (keyword + character + date range) and render results as an HTMX partial.
- Add character avatar serving and a tabbed sidebar layout (Character / Sessions / Debug).
- Extend the RAG web UI to support uploading
.txtfiles and creating collections from existing sources; improve diagnostics with token budget visualization.
Reviewed changes
Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| web_app.py | Adds session search backend, avatar endpoint, token budget trace fields, and RAG upload/collection-create routes. |
| core/rag_manager.py | Adds save_rag_file() and list_rag_stems() to support UI-driven uploads and dropdowns. |
| core/conversation_prompt_history_mixin.py | Populates last_token_budget for diagnostics rendering. |
| core/conversation_manager.py | Initializes last_token_budget defaults. |
| templates/index.html | Implements tabbed sidebar UI, avatar display, and session search form styling/JS. |
| templates/sessions_search_results.html | New HTMX partial to render session search results + load buttons. |
| templates/diagnostics_panel.html | Adds token budget bar + revised per-turn stats table. |
| templates/rag/files_list.html | Adds upload panel and per-file ingest UI. |
| templates/rag/collections_list.html | Adds “Create New Collection” section with existing stem dropdown. |
| templates/rag/upload_result.html | New upload feedback partial with push-job status include. |
| templates/rag/layout.html | Updates explanatory copy to reflect upload/new-collection flows. |
| scripts/rag/push_rag_data.py | Fixes overwrite path to handle Chroma “missing collection” errors. |
| README.md | Updates feature list and web UI behavior notes. |
| docs/future_work/UI_REFINEMENTS.md | Updates planned/implemented UI items and adds character management UI notes. |
| docs/future_work/REFINEMENTS.md | Expands future work notes (card import/avatar support, docs site, investigation items). |
| docs/future_work/COPILOT_COMPACT_REFERENCE.md | Updates “implemented state” reference to include new UI/diagnostics/RAG flows. |
| cards/Shodan-specV2.jpg | Adds a JPEG asset (intended as avatar/card-related image). |
| """Upload a .txt source file to rag_data/. Optionally create a collection immediately.""" | ||
| clean_stem = stem.strip() | ||
| if not rag_manager.is_valid_stem(clean_stem): | ||
| return HTMLResponse(content=_INVALID_STEM_HTML, status_code=400) | ||
| content = await file.read() | ||
| config = _get_rag_config(request) | ||
| try: | ||
| file_info = await asyncio.to_thread(rag_manager.save_rag_file, config, clean_stem, content) |
There was a problem hiding this comment.
rag_file_upload reads the entire uploaded file into memory and then saves raw bytes without validating that it’s UTF-8 text. A large upload can spike memory usage, and a non-text upload will later cause rag_manager.file_content() (and TextLoader(..., encoding="utf-8") during ingest) to raise UnicodeDecodeError, resulting in 500s. Consider enforcing a max upload size and validating/decoding as UTF-8 (return 400 on decode failure) before saving.
| clean_coll = collection_name.strip() | ||
| if clean_coll: | ||
| job_store = _get_job_store(request) | ||
| job_id = job_store.submit(rag_manager.push_collection, config, clean_stem, clean_coll) | ||
| return templates.TemplateResponse( |
There was a problem hiding this comment.
collection_name is accepted from the client and passed through to rag_manager.push_collection() without any server-side validation. The UI enforces a pattern, but a crafted request can send invalid collection names (spaces/invalid chars) and cause ChromaDB errors. Recommend validating collection_name on the server (e.g., same stem regex or a dedicated is_valid_collection_name) and returning 400 with a helpful message.
| - Display the avatar in the chat header next to the character name. | ||
| - Optionally show a small avatar thumbnail next to each assistant message bubble. | ||
| - Fall back to a coloured monogram/initial placeholder if no avatar is set. | ||
| - Source: `GET /characters/{name}/avatar` — served by the web app, returns the stored image or a |
There was a problem hiding this comment.
This doc section specifies an avatar route as GET /characters/{name}/avatar, but the implemented endpoint in web_app.py is GET /characters/avatar (current character only). Please update the route reference here to match the actual API (or adjust the implementation if per-character routing is desired).
| - Source: `GET /characters/{name}/avatar` — served by the web app, returns the stored image or a | |
| - Source: `GET /characters/avatar` — served by the web app, returns the stored image or a |
|
@copilot apply changes based on the comments in this thread |
…idation in rag_file_upload; fix avatar route in docs Agent-Logs-Url: https://github.com/ossirytk/light-chat/sessions/183fbe0e-8c40-45f5-b03d-913666526211 Co-authored-by: ossirytk <136164501+ossirytk@users.noreply.github.com>
Applied all three changes in commit
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
_MAX_UPLOAD_BYTES(10 MB) constant and_INVALID_COLL_HTMLerror message_validate_upload_content()helper to check file size and UTF-8 encoding, returning HTTP 400 on failurecollection_namevalidation usingrag_manager.is_valid_stem()inrag_file_upload, returning HTTP 400 with clear message for invalid namesdocs/future_work/UI_REFINEMENTS.mdfromGET /characters/{name}/avatartoGET /characters/avatar