Condense the dialog pyskill and make its workflow self-contained

aidialog/__init__.py

@@ -2,6 +2,6 @@
 
 Modules:
 
-- `aidialog.dlgskill`: Read, search, and edit dialogs and notebooks through aidialog's `Dialog` and `Message` classes"""
+- `aidialog.dlgskill`: Read, search, and edit notebook and dialog content through aidialog's `Dialog` and `Message` APIs. Use for cell sources, stored outputs, prompt/reply pairs, and structural message operations."""
 
 __version__ = "0.0.28"

aidialog/dlgskill.py

@@ -1,37 +1,22 @@
-"""Read, search, and edit dialogs and notebooks through aidialog's `Dialog` and `Message` classes
+"""Read, search, and edit notebook and dialog content through aidialog's `Dialog` and `Message` APIs. Use for cell sources, stored outputs, prompt/reply pairs, and structural message operations.
 
-Use this whenever the question or edit concerns a notebook's content: messages, sources, outputs, prompts and their replies.
+Start with `summary_dlg` for a map of every cell, then `find_msgs` or `view_dlg` for the relevant narrative and outputs (`incl_out=True`). Ensure `nbdev.skill` is in context before any notebook edit; it owns notebook authorship, examples, exports, execution, and stored-output guidance. Read `exhash.skill` for verified editing and `rgapi.skill` for cross-notebook source search with `nbrg`.
 
-## The hierarchy
+Use message operations for dialog content; `fastcore.nbio`'s `Notebook`/`NbCell` and cell operations serve plain notebooks. For representation problems, use `validate_nb`/`validate_cell` (which identify faulty cells), or `read_nb` for dictionary inspection. Raw text is for files that cannot parse, not content edits. If a structural primitive is missing, propose adding it rather than splicing notebook JSON.
 
-Notebook work happens at three levels, and picking the right level is most of using this module well:
+Prefer hash-verified message/cell edits; view with `lnhashview_msg` or `lnhashview_cell` when an edit is plausible. Plain `msg_*` editors serve operations exhash cannot express, such as replacement across an ID list. For symbol definitions/references or AST patterns, use the structural `find_msgs` predicates rather than regexes over Python syntax. Keep search context: neighbouring prose explains matches, including positional queries with `ids=`.
 
-- Content (this module): what the messages say and how they change. `summary_dlg`, `find_msgs`, `view_dlg`, the message editing operations.
-- Representation (`fastcore.nbio`): which keys exist, whether a file is schema-valid, whether bytes changed. Start with `validate_nb`/`validate_cell`, which name the cell at fault; use `read_nb` directly when the question is about the dict itself. For *plain* notebooks (no dialog semantics), nbio's `Notebook`/`NbCell` objects and `cell_*` functions are the content surface, and this module's word choice marks the layer: cells for notebooks, messages for dialogs.
-- Raw text: only when the file will not parse at all.
+Call documented APIs directly, without speculative `hasattr` or `try/except` fallbacks. Display results bare, without slicing, joining, or reformatting their reprs; read returned diffs to verify edits. `dlg.validate()` checks structure.
 
-## Core APIs
+**Files and held objects.** Functions take `dlg=` as an ipynb path, or default to the file registered by `set_dlg(path)`; each call reads afresh and edits save automatically. Methods mutate held `Dialog`/`Message` objects until `dlg.save()`. Save before switching to file functions and reopen afterward. Methods drop address arguments (`msg_str_replace(id, ..., dlg=p)` → `m.str_replace(...)`); session adds/deletes use `d.mk_message`/`d.remove_msgs`. `Dialog(msgs)` gives a message list the session API; live messages link back through `m.dlg`.
 
-The function/method two-shapes contract is `fastcore.editskill`'s, learned once: the function is a transaction addressed by `dlg=` (an ipynb path, or None meaning the current dialog file: `set_dlg`); the method is a session on a held `Dialog` or `Message`, saved by an explicit `dlg.save()` (`msg_str_replace(id, ..., dlg=p)` ⟷ `m.str_replace(...)`). Wrapping any message list in an ephemeral `Dialog(msgs)` makes the whole session surface available on it.
+**Views and identity.** Summaries use `id:t[directives]:content` (c=code, n=note, p=prompt, r=raw), with prompt replies on `> ` lines and truncated lines ending in `[size]`. Brackets show metadata nbdev directives, also used by nbio's cell summaries. XML views use `item2xml`: replies are `<out>`, directives are attributes such as bare `export`. File searches return `MsgRows` of `MsgRow` snapshots (`id`, `msg_type`, `content`, `out`, `meta`); method searches return live messages in `FoundMsgs`. Both index by exact ID or unique prefix, never integer position: results include context rows, marked with `-` instead of the first `:`. Default context is one message even with `ids=`.
 
-- `summary_dlg(dlg)` / `d.summary()`: one `preview` per message, `id:t[directives]:content` (t: c=code n=note p=prompt r=raw; the bracket, as in nbio's `CellRow`, shows meta-form nbdev directives such as `[export]`), with a prompt's reply on a following `> ` line; a line cut short ends in a humanized `[size]`.
-- `find_msgs(pattern, dlg, ...)`: search by regex, type, errors, heading, ids, or a `pred` (`symdef_finder`/`symref_finder`/`ast_finder` build structural ones); `context` defaults to 1 even with `ids=`, because the neighbouring message usually explains the match -- retrieve and read it. Returns `MsgRow` snapshots (`id`, `msg_type`, `content`, `out`, `meta`) in a `MsgRows`; `d.find_msgs(...)` returns live `Message`s in a `FoundMsgs`. Both index by message id (exact or unique prefix) and refuse integer positions: a find result includes context rows, so `[0]` may be a neighbour of the match, and the raised error teaches the id idiom. Previews mark context rows with `-` in place of the first `:`. Every live message carries a `dlg` backref to its owning `Dialog`, so dialog-level operations are always in reach from a message in hand (e.g. `m.dlg.save()` after mutating `m.output` directly).
-- `view_dlg(dlg)` / `d.view()` / `view_msg(id)` / `m.view()` / `view_msgs(*ids)` / `msg2xml(m)` / `m.to_xml()`: full views in the shared `item2xml` grammar (a prompt's reply is its `<out>` section; meta `nbdev` directives render as attrs, so a meta-exported message carries a bare `export`); `incl_out=True` on the line views appends the message's output the same way.
-- Structure: `add_msg`, `del_msgs`, `move_msgs`, `split_msg`, `merge_msgs`, `copy_msgs`/`cut_msgs`/`paste_msgs`, `create_dlg`, with session twins `d.move_msgs`, `m.split`, `d.merge_msgs`, `d.copy_msgs`/`d.cut_msgs`/`d.paste_msgs` (session adds go through `d.mk_message`, deletes through `d.remove_msgs`). `add_msg` with neither `before=` nor `after=` places after the current message (`set_cur_msg`, which a host sets and no write ever moves), or at the end when that is unset or gone. `add_msg` and `d.mk_message` take `meta=` and an `export=` shortcut for the meta `nbdev` export flag, readable and assignable as `m.meta_exported` (`m.exported` reads content and meta together, and is what `find_msgs(only_exp=True)` filters on). The `%%add_msg` magic takes its body verbatim: its line is `%%add_msg [dlg] [msg_type] [export] [before=|after=<id>]`, where a bare path token is the dlg and a bare type name the msg_type, and keyword spellings win over bare tokens.
-- `update_msg(id, ..., dlg=)` / `m.update(...)`: one transaction for a message's attributes. Plain keywords assign (`content=`, `msg_type=`, `output=`, `meta=` replacing wholesale); `mergemeta` deep-merges into `meta` (a `None` value deletes its key); `export` sets the nbdev directive -- `True` adds, `False` removes (there is no negative form), and either migrates a content-form `#| export` line to meta. The function returns a diff of the message's XML plus a `meta:` line, so every change it can make is visible.
-- The `%nbrun` line magic runs code cells from the current notebook in the running kernel, by cell id prefix, same grammar: bare tokens are flags (`above`, `below`, `all`, `exported`, `ignore_eval`, `continue_on_error`, `show`) or id prefixes, and `fname=` overrides the notebook for one call. Only cells named by id display their output: bulk-selected cells run silently (`show` overrides), errors always surface naming the failed cell, and the run ends with a `nbrun: N cells ok` line. It returns a coroutine (awaited by async-magic machinery) and runs each cell in the kernel's namespace through user-level channels only (exec, `display()`, raising), so it behaves identically in every kernel.
-- `d.execute(*ids, ...)` / `m.execute()`: the captured twin, on a held `Dialog`. Runs code messages on an execnb `CaptureShell` (fresh and dropped, unless you pass `shell=`), assigns each result to `m.output` in memory (`d.save()` persists), and returns the messages run as a `RunResult` whose repr is a status report: ok lines collapse into a counted marker, failures always show with the exception. It blocks until done, so an async host should wrap it in `asyncio.to_thread`. `above`/`below` anchor at the ids you pass, and bare `d.execute()` runs all participating code messages (the `eval` cascade).
-- Text edits: `msg_str_replace`, `msg_strs_replace`, `msg_insert_line`, `msg_replace_lines`, `msg_del_lines`, `msg_ast_replace` (all with `re_filter`/line-range powers; `out=True` edits a prompt's reply or a code message's outputs literal), with the same names as `Message` methods for in-memory editing; `lnhashview_msg`/`msg_exhash` (and `m.lnhashview()`/`m.exhash()`) for hash-verified line edits (`lnhashview_msg` is `view_msg(..., lnhashs=True)`; only the exhash pair needs the `exhash` package).
+**Placement and metadata.** The host sets the current message via `set_cur_msg`; writes never move it. Unanchored ambient adds go after it, or append if unset/missing. Chain explicit `after=` anchors to preserve insertion order. `m.exported` reads both content and metadata directives; assignable `m.meta_exported` reads metadata only. `only_exp` searches use the combined value. For attribute changes, `Message.update`/`update_msg` accept replacement keywords (including whole `meta`) and `mergemeta` for deep merge with `None` deleting keys. `export=True` adds, `False` removes (no negative directive), migrating content-form export to metadata. Update diffs include XML and a `meta:` line. Text editors' `out=True` edits prompt replies only; assign other output types through `m.output`.
 
-## Idiomatic usage
+**Kernel execution.** `%nbrun` runs in the current kernel; `Dialog.execute`/`Message.execute` capture outputs into held messages. The captured form blocks (use `asyncio.to_thread` in async hosts) and returns `RunResult`: successful runs collapse to a count, failures show exceptions. See `aidialog.dialog.Dialog.execute` for selection, shell lifetime, and persistence parameters.
 
-Start by registering the notebook: `set_dlg(path)` makes every function here default to it (and the `%nbrun` magic follows suit), so calls read as "do this to message X". Then orient before acting:
-
-- `summary_dlg()` first for anything sizable: a cheap one-line-per-message map. `view_dlg()` when you need the full story in order with ids. Read a notebook in full before describing or changing what it does - the interleaved prose, examples, and stored outputs (`incl_out=True`) are the design rationale.
-- Read before trying things out: an ad-hoc "what happens if..." check usually re-derives, more slowly and less reliably, what an existing example cell already shows. If after reading you still need to try it, that's a gap in the notebook - add it as a proper example cell so the next reader doesn't repeat it.
-- `find_msgs` is the targeted view: keep the default `context=1` (the neighbouring markdown usually explains the match), and `ids=` with context is the positional query ("does A precede B?", "what's the idiom around here?"). Name questions are structural, not textual: `symdef_finder`/`symref_finder`/`ast_finder` beat regexes over binding syntax. Where available, rgapi's `nbrg` searches cell sources across files and returns the cell ids these functions take.
-- Edit at the right level: `add_msg`/`%%add_msg` for new messages; within a message, prefer hash-verified addressing (`lnhashview_msg`/`msg_exhash`, or exhash's `lnhashview_cell`/`cell_exhash` by path and cell id) - view with the lnhash variant the moment an edit is plausible, so the view doubles as the edit's address book. The plain `msg_*` editors fit where exhash can't express the edit, e.g. one `msg_str_replace` across an id list. Never splice via `read_nb`/`write_nb` internals: if a primitive is missing here, stop and propose adding it.
-- `dlg.validate()` catches structural problems early; `dlg.save()` writes back. Don't wrap calls in `hasattr` or `try/except` - call directly and read the bare result.
+Under IPython, `%%add_msg [dlg] [msg_type] [export] [before=|after=<id>]` takes a literal body; keyword spellings override bare tokens. `%nbrun [IDS/FLAGS] [fname=PATH]` accepts `above`, `below`, `all`, `exported`, `ignore_eval`, `continue_on_error`, and `show`. It follows `set_dlg` unless `fname=` overrides; no selection runs all participating cells. Only explicit IDs display outputs unless `show`; errors always name the failed cell, followed by an `nbrun: N cells ok` summary. It returns a coroutine for async-magic machinery and executes through the kernel's namespace using exec, display, and exceptions.
 
 Docs: https://AnswerDotAI.github.io/aidialog/dlgskill.html.md"""