Problem
The Ingest jobs panel on the Manage tab shows "No ingest jobs yet" for ingests that actually ran, which is confusing. It only reflects jobs submitted through the management API (the Reindex button or "Add & index"), tracked by the in-memory JobManager in the agent process (src/tpk/api.py). Two gaps make it unreliable as a status view:
- CLI ingests are invisible.
docker compose exec tpk tpk ingest ... runs in the tpk container, entirely separate from the agent's JobManager, so those runs never appear — even though they're the primary way to (re)ingest at scale.
- In-memory and per-process. Even API-submitted jobs disappear when the agent container restarts, and there is no history.
Meanwhile the persistent record already exists: kg_ingest_log (append-only: repo/entry_key, run_id, nodes, edges, git_sha, status, time) is what the table's own "Last ingest" column reads. The jobs panel just doesn't use it.
Proposal
Make the Ingest jobs panel show persistent per-entry ingest history from kg_ingest_log, merged with the live in-flight JobManager jobs (so a running Reindex still shows live progress, and finished/CLI/historical runs show their real ok/failed/nodes/edges/timestamp).
Sketch:
- New read endpoint (e.g.
GET /api/ingest-log or extend GET /api/jobs) that returns recent kg_ingest_log rows per entry_key (latest N, or arg_max per repo like the corpus table already does), under require_admin.
- UI: the jobs panel lists recent ingests from that endpoint (repo, status, nodes/edges, sha, relative time), and overlays any live
JobManager job for an entry as "running…". This makes the panel a truthful status view regardless of whether the ingest came from the UI, the CLI, or a previous session.
Context
Surfaced while debugging why chat returned nothing (stale bare-name node keys from pre-versioning ingests; fixed by re-ingesting via CLI — which correctly did not appear in this panel, prompting the question). Repo-filter robustness was fixed separately in PR #14. This is the UI/observability follow-up.
Related: the corpus table's "Last ingest" column already reads kg_ingest_log via arg_max(...) GROUP BY repo (src/tpk/api.py list_repos), so the query pattern exists to reuse.
Problem
The Ingest jobs panel on the Manage tab shows "No ingest jobs yet" for ingests that actually ran, which is confusing. It only reflects jobs submitted through the management API (the Reindex button or "Add & index"), tracked by the in-memory
JobManagerin the agent process (src/tpk/api.py). Two gaps make it unreliable as a status view:docker compose exec tpk tpk ingest ...runs in thetpkcontainer, entirely separate from the agent'sJobManager, so those runs never appear — even though they're the primary way to (re)ingest at scale.Meanwhile the persistent record already exists:
kg_ingest_log(append-only: repo/entry_key, run_id, nodes, edges, git_sha, status, time) is what the table's own "Last ingest" column reads. The jobs panel just doesn't use it.Proposal
Make the Ingest jobs panel show persistent per-entry ingest history from
kg_ingest_log, merged with the live in-flightJobManagerjobs (so a running Reindex still shows live progress, and finished/CLI/historical runs show their real ok/failed/nodes/edges/timestamp).Sketch:
GET /api/ingest-logor extendGET /api/jobs) that returns recentkg_ingest_logrows per entry_key (latest N, or arg_max per repo like the corpus table already does), underrequire_admin.JobManagerjob for an entry as "running…". This makes the panel a truthful status view regardless of whether the ingest came from the UI, the CLI, or a previous session.Context
Surfaced while debugging why chat returned nothing (stale bare-name node keys from pre-versioning ingests; fixed by re-ingesting via CLI — which correctly did not appear in this panel, prompting the question). Repo-filter robustness was fixed separately in PR #14. This is the UI/observability follow-up.
Related: the corpus table's "Last ingest" column already reads
kg_ingest_logviaarg_max(...) GROUP BY repo(src/tpk/api.pylist_repos), so the query pattern exists to reuse.