Skip to content

Feat/p0 usefulness first - #41

Merged
Ocean82 merged 5 commits into
mainfrom
feat/p0-usefulness-first
Sep 27, 2026
Merged

Ocean82 merged 5 commits into
mainfrom
feat/p0-usefulness-first

Conversation

@Ocean82

@Ocean82 Ocean82 commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Summary by Sourcery

Complete the usefulness-first P0 by improving spreadsheet import transparency, safe formula edits, audit visibility, and quality feedback while documenting the next formatting-focused priorities.

New Features:

  • Add import honesty warnings for cached formula values and un-applied visual styles, surfacing them in chat and toast notifications.
  • Add bounded formula-application safety with range-gap detection, preview support, and explicit confirmation overrides.
  • Keep critical import audit findings visible and surface their titles in the activation overlay.
  • Collect optional local context with chat feedback to support AI quality analysis.

Bug Fixes:

  • Avoid false-positive style-loss warnings for plain or number-format-only workbooks.
  • Prevent stale audit results from controlling post-import overlay dismissal.

Enhancements:

  • Define a living usefulness-first product strategy that marks core trust, safety, activation, and feedback work complete and prioritizes formatting polish next.
  • Refresh Formualizer compatibility documentation and establish an Excel-parity golden test set around the current 0.9.3 dependency.

Documentation:

  • Document the usefulness-first roadmap, P0 completion criteria, formatting sandbox priorities, and explicit product non-goals.
  • Update Formualizer gap documentation to reflect the 0.9.3 baseline and remaining upstream risks.

Tests:

  • Add coverage for formula range-gap detection and confirmation behavior.
  • Add tests for import honesty warnings, formula previews, and representative Formualizer Excel-parity scenarios.

Summary by CodeRabbit

  • New Features

    • Formula actions now show a preview before application and flag adjacent numeric cells that may be unintentionally excluded from a range. You can confirm the change or cancel it.
    • Workbook imports surface warnings about formula values and formatting, including in a chat note and warning toast. Import insights highlight serious findings and remain open when critical or high-severity issues are present.
    • Chat feedback can include brief details about the message being rated.
  • Bug Fixes

    • Improved detection of meaningful workbook styles and number formats to reduce misleading import warnings.
  • Documentation

    • Updated the spreadsheet-engine upgrade guidance and added strategy and planning documentation.

…truth

The workflow ENV-sync now keeps only the 10 most recent /opt/smartsht/.env.bak-gha-* backups instead of accumulating them unbounded. Also documents that the GitHub ENV secret is authoritative (every auto-deploy overwrites the server .env from it), warns against hand-editing the live file, and notes how to verify the loaded env via /health or the boot log.
Move the keep-last-10 prune outside the [ -f /opt/smartsht/.env ] guard so stale backups are pruned on every sync, including runs where the live .env file does not exist. Pruning depends only on the .bak-gha-* files, not the live file.
Make imports honest, block gapped apply_formula without preview, keep critical insights visible, and document the useful-first strategy.
Copilot AI lite review requested due to automatic review settings September 26, 2026 14:07
@sourcery-ai

sourcery-ai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

This PR completes the usefulness-first P0 foundation by documenting the roadmap, making XLSX import limitations visible, adding guarded formula application with previews, improving audit activation behavior, capturing bounded feedback context, and extending real Formualizer parity tests.

Sequence diagram for honest workbook import

sequenceDiagram
    participant User
    participant Toolbar
    participant XLSX as importWorkbookFromFileWithMeta
    participant Store as importWorkbook
    participant Effects as applyWorkbookImportEffects
    participant UI as ImportInsightsOverlay

    User->>Toolbar: Select workbook
    Toolbar->>XLSX: importWorkbookFromFileWithMeta(file)
    XLSX-->>Toolbar: workbook, meta.warnings
    Toolbar->>Store: importWorkbook(workbook, fileName, warnings)
    Store->>Effects: applyWorkbookImportEffects(workbook, meta)
    Effects-->>UI: Import message and warning toast
    UI-->>User: Show insights and import limitations
Loading

Sequence diagram for guarded formula application

sequenceDiagram
    participant Agent
    participant Preview as buildActionPreview
    participant Handler as handleApplyFormula
    participant Risk as detectApplyFormulaRangeGapRisk
    participant Sheet

    Agent->>Preview: buildActionPreview(apply_formula, params)
    Preview-->>Agent: Proposed cell change
    Agent->>Handler: handleApplyFormula(params, ctx, sheet)
    Handler->>Risk: detectApplyFormulaRangeGapRisk(formula, sheet, getComputedValue, cell)
    alt Range gap risk and confirmGaps not set
        Risk-->>Handler: Warning
        Handler-->>Agent: Blocked apply_formula
    else No risk or confirmGaps set
        Risk-->>Handler: null or warning
        Handler->>Sheet: setCellValue(cell, null, formula)
        Handler-->>Agent: Successful apply result
    end
Loading

File-Level Changes

Change Details Files
Add a living usefulness-first product strategy and align supporting documentation with the current P0/P1/P2 priorities.
  • Document completed P0 capabilities and the next formatting-focused P1 roadmap.
  • Update Formualizer status, known residual risks, and upgrade validation guidance.
  • Clarify deployment environment ownership and retain only the 10 newest backups.
docs/strategy/2026-09-24-usefulness-first-strategy.md
docs/ARCHIVE.md
docs/planning/README.md
docs/formualizer-gaps.md
docs/DEPLOY.md
.github/workflows/deploy.yml
Make spreadsheet imports more honest about formula evaluation and style preservation, and surface those limitations consistently to users.
  • Track formula cached-value presence and meaningful style metadata during XLSX import.
  • Generate warnings for cached formulas, missing cached values, and unapplied visual styles.
  • Propagate import warnings into the chat import message and warning toast.
  • Add tests covering cached-formula and unstyled-workbook behavior.
src/io/xlsx.ts
src/io/xlsx.test.ts
src/store/importOrchestration.ts
src/store/__tests__/importOrchestration.test.ts
src/components/Toolbar.tsx
src/store/slices/chatSlice.ts
src/store/slices/workbookSlice.ts
src/store/storeTypes.ts
src/store/useStore.ts
Harden formula application with range-gap detection and integrate it into the preview/apply workflow.
  • Detect adjacent numeric cells excluded from aggregate ranges in row and column orientations.
  • Block risky formula writes unless the caller explicitly confirms or forces the operation.
  • Add apply_formula previews and tests for blocked and confirmed writes.
src/agent/toolHandlers/columnOps.ts
src/agent/toolHandlers/columnOps.test.ts
src/lib/previewBuilders.ts
src/lib/previewBuilders.test.ts
src/store/slices/chatSlice.ts
Improve post-import activation by keeping important audit findings visible until acknowledged.
  • Delay auto-dismiss until the audit completes or a grace period expires.
  • Keep overlays with critical/high findings open and show up to three finding titles.
  • Add critical-state styling and status text.
src/components/ImportInsightsOverlay.tsx
Establish lightweight feedback context and expand formula-engine parity coverage.
  • Persist bounded optional feedback details locally and include them in telemetry.
  • Add a small real-engine golden set for common aggregation, lookup, conditional, and error-handling formulas.
src/ai/chatFeedback.ts
src/components/ChatPanel.tsx
src/engine/formualizer.realengine.test.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Essentials

Run ID: f0b2c780-a9b5-4a7b-bf0a-91955160b823

📥 Commits

Reviewing files that changed from the base of the PR and between aa9b749 and 75d3dfb.

📒 Files selected for processing (21)
  • docs/ARCHIVE.md
  • docs/formualizer-gaps.md
  • docs/planning/README.md
  • docs/strategy/2026-09-24-usefulness-first-strategy.md
  • src/agent/toolHandlers/columnOps.test.ts
  • src/agent/toolHandlers/columnOps.ts
  • src/ai/chatFeedback.ts
  • src/components/ChatPanel.tsx
  • src/components/ImportInsightsOverlay.tsx
  • src/components/Toolbar.tsx
  • src/engine/formualizer.realengine.test.ts
  • src/io/xlsx.test.ts
  • src/io/xlsx.ts
  • src/lib/previewBuilders.test.ts
  • src/lib/previewBuilders.ts
  • src/store/__tests__/importOrchestration.test.ts
  • src/store/importOrchestration.ts
  • src/store/slices/chatSlice.ts
  • src/store/slices/workbookSlice.ts
  • src/store/storeTypes.ts
  • src/store/useStore.ts

📝 Walkthrough

Walkthrough

This PR adds formula previews and range-gap checks, improves XLSX import warnings and audit-panel behavior, adds context to chat feedback telemetry, expands Formualizer parity tests, and updates strategy and engine-gap documentation.

Changes

Formula Application

Layer / File(s) Summary
Aggregate formula gap checks
src/agent/toolHandlers/columnOps.ts, src/agent/toolHandlers/columnOps.test.ts
Aggregate formulas are checked for adjacent numeric cells outside their ranges. Unconfirmed gaps block writes; confirmed writes include a warning. Tests cover detection and confirmation.
Formula previews and action gating
src/lib/previewBuilders.ts, src/lib/previewBuilders.test.ts, src/store/slices/chatSlice.ts
Formula actions now produce cell or column previews. apply_formula requires a preview, and applying a previewed formula passes gap confirmation.

Workbook Import Feedback

Layer / File(s) Summary
XLSX warnings and import tests
src/io/xlsx.ts, src/io/xlsx.test.ts
XLSX import distinguishes meaningful styles from default style stubs and reports formula cached-value warnings. Tests cover formula warnings and unstyled workbooks.
Import warning propagation
src/store/storeTypes.ts, src/store/slices/workbookSlice.ts, src/store/useStore.ts, src/store/slices/chatSlice.ts, src/store/importOrchestration.ts, src/components/Toolbar.tsx, src/store/__tests__/importOrchestration.test.ts
Import metadata carries warnings through the store. Import orchestration appends an “Import honesty” note to chat and shows the first warning in a toast.
Import audit overlay behavior
src/components/ImportInsightsOverlay.tsx
The overlay waits for an audit result or a one-second grace period before enabling its dismissal timer. Critical and high findings prevent auto-dismiss and appear in the panel.

Chat Feedback Context

Layer / File(s) Summary
Feedback detail capture
src/ai/chatFeedback.ts, src/components/ChatPanel.tsx
Feedback entries can store trimmed detail of up to 200 characters. ChatPanel supplies the message’s tool name and up to 80 characters of content when available.

Formula Engine Compatibility

Layer / File(s) Summary
Engine golden cases and version guidance
src/engine/formualizer.realengine.test.ts, docs/formualizer-gaps.md
Real-engine tests cover aggregate functions, exact-match VLOOKUP, and IFERROR. The gap report updates the Formualizer version status, remaining gaps, and upgrade checks.

Forward Plan Documentation

Layer / File(s) Summary
Strategy scope and priorities
docs/strategy/2026-09-24-usefulness-first-strategy.md
The strategy document records the core workflow, completed P0 items, P1 and P2 priorities, and non-goals.
Formatting plan and documentation links
docs/strategy/2026-09-24-usefulness-first-strategy.md, docs/planning/README.md, docs/ARCHIVE.md
The strategy specifies formatting sandbox steps and documentation roles. The planning README and archive link to the strategy.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant XLSXImporter as XLSX importer
  participant Toolbar
  participant ImportWorkbook as importWorkbook
  participant ImportOrchestration as applyWorkbookImportEffects
  participant Chat
  participant Toast
  XLSXImporter->>Toolbar: workbook and import warnings
  Toolbar->>ImportWorkbook: workbook and metadata
  ImportWorkbook->>ImportOrchestration: workbook and metadata
  ImportOrchestration->>Chat: append import honesty note
  ImportOrchestration->>Toast: show first warning
Loading
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/agent/toolHandlers/columnOps.ts" line_range="172-174" />
<code_context>
+  return Number.isFinite(num) && computed.trim() !== '' ? num : null
+}
+
+function confirmGapsRequested(params: Record<string, unknown>): boolean {
+  return params.confirmGaps === true || params.confirmGaps === 'true' || params.force === true
+}
+
 /** Apply a formula below the last data row in a column. */
</code_context>
<issue_to_address>
**issue (bug_risk):** A gapped `apply_formula` action that has a generated preview is still rejected when the user clicks Apply, because the preview does not add `confirmGaps` or `force` to the action parameters and the executor calls `handleApplyFormula` unchanged. The only way through is for the model to pre-populate an override, so the user cannot confirm the displayed warning through the normal Apply flow.

**Triggers:** When an aggregate formula excludes an adjacent numeric cell and the action is generated without `confirmGaps: true`.

**Suggested fix:** Treat an explicit Apply after displaying the preview as confirmation, or provide a confirmation path that sets `confirmGaps: true` before executing the action.
</issue_to_address>

### Comment 2
<location path="src/io/xlsx.ts" line_range="214-216" />
<code_context>
+  if (style.fgColor || style.bgColor || style.fill || style.font || style.border || style.alignment)
+    return true
+  if (style.numFmt && typeof style.numFmt === 'object') return true
+  // patternType alone (e.g. "none") is a default stub, not real styling
+  const keys = Object.keys(style).filter((k) => k !== 'patternType')
+  return keys.length > 0
+}
+
</code_context>
<issue_to_address>
**issue (bug_risk):** A workbook containing only number-format metadata is classified as having meaningful style objects, but `visualStylesApplied` remains zero because number formats are not included in `visualKeys`. Import therefore emits the misleading warning that no visual styles could be applied even though the style metadata was valid and intentionally non-visual.

**Triggers:** When an imported workbook uses number formats without fills, fonts, borders, or other visual styles.

**Suggested fix:** Track number-format application separately or exclude number-format-only objects from the visual-style warning condition.
</issue_to_address>

Fix all in Cursor

Sourcery assessment

Needs a human reviewer. 2 findings to address first, and the deployment sync now permanently deletes all but the ten newest production .env backups, so reverting the change cannot restore backups already pruned. The feedback change also sends up to 200 characters of assistant content through telemetry, which could expose workbook-derived information and cannot be undone after transmission.

Blocking findings: src/agent/toolHandlers/columnOps.ts:174, src/io/xlsx.ts:216


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread src/agent/toolHandlers/columnOps.ts
Comment thread src/io/xlsx.ts
…umber-format-only imports

- applyAction: treat explicit Apply of a previewed apply_formula as gap
  confirmation (confirmGaps=true) so reviewed formulas are not rejected
- xlsx import: track numberFormatsApplied so number-format-only workbooks
  no longer trigger the misleading 'no visual styles applied' warning
@Ocean82
Ocean82 merged commit 9e68548 into main Sep 27, 2026
5 checks passed
@Ocean82
Ocean82 deleted the feat/p0-usefulness-first branch September 27, 2026 13:00
Ocean82 added a commit that referenced this pull request Sep 27, 2026
…reakage

Concludes the merge of 9e68548 (Feat/p0 usefulness first, #41). HEAD already
contained all of #41's content, so the merge itself is a no-op; resolved by
keeping our side, which is a strict superset: P0.3 gap detection extracted to
src/lib/formulaGapRisk.ts and wired into the preview path, P0.5 user-entered
thumbs-down detail, and P1.4 style_recipe preview support.

Fixes four pre-existing breakages on this branch, masked until now by the
unresolved conflict markers:

- previewBuilders: add the missing detectFormulaRangeGapRisk import; the P0.3
  preview path did not compile
- styleRecipes.test: use refToCell(row, col) -- cellToRef takes a cell id
- parser: add style_recipe and format_as_table to the question-veto set, so
  "Should I add a total row?" no longer fires a bulk restyle
- styleRecipes: drop the dead totalsRow binding, which failed
  lint:ci (--max-warnings=0)

Verified green: lint:ci, typecheck, 1822 unit tests, 344 server tests,
12 realengine golden-set tests.
Ocean82 added a commit that referenced this pull request Sep 27, 2026
…eedback (#43)

Completes the remaining P0 (Useful First) items. P0.1, P0.2 and P0.4 already
landed on main via #41; this covers the rest.

P0.3 — act-path safety
  The range-gap detector ("SUM skips an adjacent numeric cell") moves out of
  columnOps.ts into src/lib/formulaGapRisk.ts so the Apply/Reject *preview*
  path can share it, not just the execute path. The preview now surfaces the
  risk as a warning alongside the proposed changes, and Apply confirms those
  warnings via the existing confirmGaps override. columnOps.ts shrinks by
  ~110 lines as a result.

P0.5 — AI quality loop
  Thumbs-down now opens an optional "what went wrong?" field. The rating is
  recorded immediately on click; submitting the note re-records it with the
  user's own words instead of the auto-derived fingerprint, which is what
  makes failover analysis actionable. Cmd/Ctrl+Enter submits, Escape skips.

Parser question-veto
  format_as_table joins DESTRUCTIVE_TOOLS so "should I format this as a
  table?" no longer fires a bulk restyle. Targeted tools stay excluded — the
  polite-framing path deliberately treats "can you highlight X" as a command.

Verified: lint:ci, typecheck, 1791 unit tests, 344 server tests,
12 realengine golden-set tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants