Skip to content

P0 usefulness-first: trust & activation, plus P1.1-P1.3 formatting sandbox - #42

Closed
Ocean82 wants to merge 9 commits into
mainfrom
feat/p0-usefulness-first
Closed

Ocean82 wants to merge 9 commits into
mainfrom
feat/p0-usefulness-first

Conversation

@Ocean82

@Ocean82 Ocean82 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Lands the remaining P0 (Useful First) items and the first three P1 (competitive polish) items from docs/strategy/2026-09-24-usefulness-first-strategy.md.

Note on scope: this is more than the 4 CI fixes from the final commit. The branch also carries the P0/P1.1-P1.3 work that had not yet reached main.

P0 — Useful First

ID Item Implementation
P0.1 Formula engine parity @ocean8219/formualizer@^0.9.3; gaps doc refreshed; real-WASM golden set extended
P0.2 Import honesty src/io/xlsx.ts warns on Excel cached values / dropped styles, surfaced via import meta
P0.3 Act-path safety apply_formula gap detection + Apply/Reject preview; confirmGaps override
P0.4 Activation UI ImportInsightsOverlay waits for audit/grace, never auto-dismisses on critical/high
P0.5 AI quality loop Thumbs + user-entered detail on thumbs-down for failover analysis

P1 — Competitive polish

  • P1.1 format_cells extended to the full CellFormat surface (underline, strikethrough, fontFamily, align, wrap, borders) — src/lib/formatCellsTool.ts
  • P1.2 Layout tools set_column_width / set_row_height / auto_fit — src/agent/toolHandlers/layoutOps.ts + ExecutionContext hooks
  • P1.3 NL parser coverage for layout + formatting phrases, shared by client and server (shared/spreadsheetPhrases.ts, shared/actTemplates.ts)

Plus deploy hardening (.env backup pruning, ENV source-of-truth).

Notable refactor

P0.3 gap detection was extracted out of columnOps.ts into src/lib/formulaGapRisk.ts so the preview path can share it, not just the execute path. columnOps.ts shrinks by ~110 lines as a result.

Fixes in the final commit

The branch had four pre-existing breakages, masked until an unresolved merge was cleared:

  • previewBuilders — missing detectFormulaRangeGapRisk import; the P0.3 preview path did not compile
  • styleRecipes.test — refToCell(row, col); cellToRef takes a cell id
  • parser — style_recipe / format_as_table added to the question-veto set, so "Should I add a total row?" no longer fires a bulk restyle
  • styleRecipes — dead totalsRow binding, which failed lint:ci (--max-warnings=0)

Verification

Gate Result
npm run lint:ci pass
npm run typecheck pass
npm run test 1822 / 1822
npm run test --prefix server 344 / 344
npm run test:realengine 12 / 12

Not covered by this PR

The P0 gate (strategy doc §3, steps 1-6) is a manual smoke test — import a real budget .xlsx, confirm totals match Excel, 5+ grounded Q&A turns, safe edit with preview/undo. CI cannot verify it. Tracked in docs/verification-backlog.md (items 1-3).

🤖 Generated with Claude Code

Summary by Sourcery

Complete the remaining usefulness-first trust and activation work while adding safer formula previews and broader spreadsheet formatting, layout, and natural-language editing capabilities.

New Features:

  • Expand spreadsheet formatting to cover typography, alignment, wrapping, and borders.
  • Add agent tools for column widths, row heights, auto-fitting, and bounded style recipes.
  • Improve thumbs-down feedback with optional user-entered explanations.

Bug Fixes:

  • Prevent aggregate formulas from silently excluding adjacent numeric cells by surfacing gap risks in previews and execution.
  • Preserve correct parser behavior for questions and existing formatting/table commands.
  • Fix formula preview, style recipe tests, and lint/typecheck regressions.

Enhancements:

  • Share layout and formatting phrase parsing across client and server paths.
  • Refactor formula gap-risk detection into a reusable module for preview and execution.
  • Surface import warnings for cached Excel values and dropped styles, and strengthen activation insights behavior.

Build:

  • Harden deployment environment handling with .env backup pruning and a single environment source of truth.

Deployment:

  • Harden deployment environment handling with .env backup pruning and a single environment source of truth.

Documentation:

  • Update the usefulness-first strategy to mark the delivered P0 and P1.1–P1.3 work as complete.

Tests:

  • Extend coverage for formula gap detection, formatting, layout tools, style recipes, parser phrases, and real-engine behavior.

…truth

The workflow ENV-sync now keeps only the 10 most recent /opt/smartsht/.env.bak-gha-* backups instead of accumulating them unbounded. Also documents that the GitHub ENV secret is authoritative (every auto-deploy overwrites the server .env from it), warns against hand-editing the live file, and notes how to verify the loaded env via /health or the boot log.
Move the keep-last-10 prune outside the [ -f /opt/smartsht/.env ] guard so stale backups are pruned on every sync, including runs where the live .env file does not exist. Pruning depends only on the .bak-gha-* files, not the live file.
Make imports honest, block gapped apply_formula without preview, keep critical insights visible, and document the useful-first strategy.
…umber-format-only imports

- applyAction: treat explicit Apply of a previewed apply_formula as gap
  confirmation (confirmGaps=true) so reviewed formulas are not rejected
- xlsx import: track numberFormatsApplied so number-format-only workbooks
  no longer trigger the misleading 'no visual styles applied' warning
…s to full CellFormat (P1.1)

P0.5: thumbs-down now opens an inline optional comment field; the rating
is recorded on click and re-recorded with the user's own words on submit.

P1.1: expose underline, strikethrough, fontFamily, textAlign, verticalAlign,
textWrap, and borders through buildFormatPatch, FormatCellsParams, and the
format_cells registry schema. Borders accept a shared string or per-side
object; align values are enum-validated. Extends formatCellsTool tests.
…fit (P1.2)

Expose the existing undoable store layout methods to the agent:
- ExecutionContext gains setColumnWidth/setRowHeight/autoFitRows hooks that
  mutate without pushing history (the handler owns the single undo point,
  matching the deleteRow convention)
- new layoutOps handlers parse column (B, B:D, B,D,F) and row (2, 2:5)
  specs; auto_fit falls back to all populated rows
- register the three tools (category: mutate) so they reach MUTATION_TOOL_NAMES
  and the LLM tool prompt
- layoutOps unit tests for parsers and handlers
…(P1.3)

Add parseLayoutPhrase (shared) and route it through both the client
agent-parser and server actTemplates so coverage stays aligned:
- width: 'set column C width to 200', 'make column B wider', 'widen columns B:D'
- height: 'set row 2 height to 40', 'make row 1 taller'
- auto-fit: 'auto-fit the rows', 'resize rows to fit content'

Patterns are conservative (require explicit column/row + keyword) so the
false-positive corpus stays green. Adds phrase + parser + actTemplates tests.
…update and implement plans from docs, p0 complete, check p0.5, started 1.0-5
…reakage

Concludes the merge of 9e68548 (Feat/p0 usefulness first, #41). HEAD already
contained all of #41's content, so the merge itself is a no-op; resolved by
keeping our side, which is a strict superset: P0.3 gap detection extracted to
src/lib/formulaGapRisk.ts and wired into the preview path, P0.5 user-entered
thumbs-down detail, and P1.4 style_recipe preview support.

Fixes four pre-existing breakages on this branch, masked until now by the
unresolved conflict markers:

- previewBuilders: add the missing detectFormulaRangeGapRisk import; the P0.3
  preview path did not compile
- styleRecipes.test: use refToCell(row, col) -- cellToRef takes a cell id
- parser: add style_recipe and format_as_table to the question-veto set, so
  "Should I add a total row?" no longer fires a bulk restyle
- styleRecipes: drop the dead totalsRow binding, which failed
  lint:ci (--max-warnings=0)

Verified green: lint:ci, typecheck, 1822 unit tests, 344 server tests,
12 realengine golden-set tests.
Copilot AI lite review requested due to automatic review settings September 27, 2026 17:38
@sourcery-ai

sourcery-ai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Implements the remaining Useful First trust, safety, activation, and AI-feedback work, while adding formatting/layout tools and shared natural-language routing with preview-aware execution; it also hardens deployment configuration and resolves CI regressions. Review the shared parser/tool contracts, preview-versus-execution parity, undo behavior, and import/audit messaging closely; the manual P0 smoke-test gate remains outside this PR.

Sequence diagram for natural-language formatting and layout execution

sequenceDiagram
    actor User
    participant Chat as ChatPanel
    participant Parser as parseMessage
    participant PhraseParser as parseLayoutPhrase
    participant Handler as ToolHandler
    participant Context as ExecutionContext
    participant Store as aiExecution

    User->>Chat: Enter layout or formatting request
    Chat->>Parser: parseMessage(message, sheetContext)
    Parser->>PhraseParser: parseLayoutPhrase(message)
    PhraseParser-->>Parser: LayoutPhrase
    Parser->>Handler: set_column_width / set_row_height / auto_fit
    Handler->>Context: pushHistory(description)
    Handler->>Context: setColumnWidth / setRowHeight / autoFitRows
    Context->>Store: Mutate sheet layout
    Store-->>User: Updated layout with one undo point
Loading

Sequence diagram for formula gap-risk preview and confirmation

sequenceDiagram
    actor User
    participant Preview as buildActionPreview
    participant Risk as detectFormulaRangeGapRisk
    participant UI as ApplyRejectPreview
    participant Executor as handleApplyFormula
    participant Context as ExecutionContext

    User->>Preview: Request apply_formula preview
    Preview->>Risk: detectFormulaRangeGapRisk(formula, sheet, getComputedValue)
    Risk-->>Preview: Warning or null
    Preview-->>UI: changes and optional warnings
    UI-->>User: Show proposed formula and gap warning
    alt User rejects or edits formula
        User->>UI: Reject preview
    else User confirms safely
        UI->>Executor: handleApplyFormula(params)
        Executor->>Risk: detectFormulaRangeGapRisk(formula, sheet, getComputedValue)
        Risk-->>Executor: Warning or null
        Executor->>Context: setCellValue(cell, null, formula)
        Context-->>User: Formula applied
    else User explicitly confirms gap
        UI->>Executor: handleApplyFormula(confirmGaps)
        Executor->>Context: setCellValue(cell, null, formula)
        Context-->>User: Formula applied with override
    end
Loading

File-Level Changes

Change Details Files
Adds trust and safety controls around imports, formula edits, activation, and feedback collection.
  • Refreshes formula-engine parity coverage and real-WASM golden cases.
  • Propagates Excel cached-value and dropped-style warnings through import metadata.
  • Shares extracted aggregate-range gap detection between execution and preview, with confirm/Apply/Reject handling.
  • Keeps critical/high import audit findings visible until explicitly addressed.
  • Captures optional user explanations for negative chat feedback.
src/io/xlsx.ts
src/lib/formulaGapRisk.ts
src/agent/toolHandlers/columnOps.ts
src/lib/previewBuilders.ts
src/components/ChatPanel.tsx
src/store/chat*
docs/strategy/2026-09-24-usefulness-first-strategy.md
Extends spreadsheet formatting and layout mutations across the tool, execution, parser, and store layers.
  • Maps underline, strikethrough, font family, alignment, wrapping, and borders into CellFormat patches and registry schemas.
  • Adds bounded style recipes with shared planning for preview and execution.
  • Adds column-width, row-height, and auto-fit handlers with range parsing, clamping, and single undo points.
  • Adds shared natural-language phrase parsing and routes client/server act paths to the new tools.
  • Adds execution-context hooks backed by column/row layout and canvas-based autofit store utilities.
src/lib/formatCellsTool.ts
src/shared/toolTypes.ts
src/shared/toolRegistry.ts
src/lib/styleRecipes.ts
src/agent/toolHandlers/formatOps.ts
src/agent/toolHandlers/layoutOps.ts
src/agent/executor.ts
src/store/aiExecution.ts
src/shared/spreadsheetPhrases.ts
src/shared/actTemplates.ts
src/agent/parser.ts
Hardens deployment configuration and updates strategy status and regression coverage.
  • Prunes environment backups and establishes the intended environment source of truth.
  • Marks delivered P0 and P1.1–P1.3 work complete in the strategy documentation.
  • Fixes preview imports, cell-reference test usage, question-veto routing, and lint failures.
  • Adds parser, layout, formatting, style-recipe, and formula-gap regression tests.
docs/strategy/2026-09-24-usefulness-first-strategy.md
.env*
shared/*.test.ts
src/**/*.test.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Advanced

Run ID: bc3cbd64-631e-48c9-a62c-fb0597feb166


Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Needs a human reviewer. The new style recipe and layout paths can persist formulas, filters, formatting, and workbook dimensions based on natural-language parsing, so a misinterpretation could write incorrect totals or alter the sheet before being noticed. The changes are bounded and generally undoable or recomputable, but incorrect persisted values or layout state may need cleanup after reverting.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

@Ocean82

Ocean82 commented Sep 27, 2026

Copy link
Copy Markdown
Owner Author

Superseded by the split into #43 (P0) and #44 (P1.1–P1.4). The CI fixes that were buried in this PR's final commit are now distributed to where the code they repair actually lives — 2 in #43, 2 in #44 — since 2 of the 4 only make sense against styleRecipes.ts, which does not exist on main.

@Ocean82 Ocean82 closed this Sep 27, 2026
@Ocean82
Ocean82 deleted the feat/p0-usefulness-first branch September 27, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants