Skip to content

[Payment due @abdulrahuman5196] Improve column auto-mapping when the file structure is different - #96654

Merged
chiragsalian merged 9 commits into
mainfrom
alberto-betterMaps
Sep 17, 2026
Merged

chiragsalian merged 9 commits into
mainfrom
alberto-betterMaps

Conversation

@Gonals

@Gonals Gonals commented Jul 21, 2026 •

Copy link
Copy Markdown
Contributor

Explanation of Change

Fixed Issues

$ https://github.com/Expensify/Expensify/issues/662378
PROPOSAL:

Tests

  1. Go to Workspaces->Company Cards-> Add cards and add some cards via CSV:
    uploads_1594908368712-Best+Example+CSV+for+Domains.csv

  2. Wait until the cards show up

  3. In the feed, go to settings -> Import from spreadsheet and add more cards from a differently structured file:
    fake_transactions-2026-01-16 10_56_55.629.csv

  4. Confirm the auto-mapping of columns makes sense. It won't be perfect, as some columns are different, but there should not be attributes double assigned or things like date being assigned to the amount column.

  • Verify that no errors appear in the JS console

Offline tests

None

QA Steps

Same as tests

  • Verify that no errors appear in the JS console

PR Author Checklist

  • I linked the correct issue in the ### Fixed Issues section above
  • I wrote clear testing steps that cover the changes made in this PR
    • I added steps for local testing in the Tests section
    • I added steps for the expected offline behavior in the Offline steps section
    • I added steps for Staging and/or Production testing in the QA steps section
    • I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
    • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
    • I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
  • I included screenshots or videos for tests on all platforms
  • I ran the tests on all platforms & verified they passed on:
    • Android: Native
    • Android: mWeb Chrome
    • iOS: Native
    • iOS: mWeb Safari
    • MacOS: Chrome / Safari
  • I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
  • I followed proper code patterns (see Reviewing the code)
    • I verified that comments were added to code that is not self explanatory
    • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
    • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
  • If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
  • I followed the guidelines as stated in the Review Guidelines
  • I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
  • If a new CSS style is added I verified that:
    • A similar style doesn't already exist
    • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
  • If new assets were added or existing ones were modified, I verified that:
    • The assets are optimized and compressed (for SVG files, run npm run compress-svg)
    • The assets load correctly across all supported platforms.
  • If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
  • If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
  • If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
  • If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
  • If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
    • I verified that all the inputs inside a form are aligned with each other.
    • I added Design label and/or tagged @Expensify/design so the design team can review the changes.
  • I added unit tests for any new feature or bug fix in this PR to help automatically prevent regressions in this user flow.
  • If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
Android: mWeb Chrome
iOS: Native
iOS: mWeb Safari
MacOS: Chrome / Safari

@Gonals Gonals self-assigned this Jul 21, 2026
@codecov

codecov Bot commented Jul 21, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Looks like you've decreased code coverage for some files. Please write tests to increase, or at least maintain, the existing level of code coverage. See our documentation here for how to interpret this table.

Files with missing lines Coverage Δ
src/components/ImportSpreadsheetColumns.tsx 100.00% <100.00%> (ø)
src/libs/actions/ImportSpreadsheet.ts 76.71% <100.00%> (-3.29%) ⬇️
src/components/ImportColumn.tsx 93.75% <75.00%> (+64.36%) ⬆️
...e/companyCards/addNew/CompanyCardsImportedPage.tsx 56.70% <90.00%> (+54.57%) ⬆️
src/libs/importSpreadsheetUtils.ts 49.75% <45.78%> (-44.36%) ⬇️
... and 11 files with indirect coverage changes

@Gonals Gonals changed the title [HOLD] Improve column auto-mapping when the file structure is different Improve column auto-mapping when the file structure is different Aug 5, 2026
@Gonals
Gonals marked this pull request as ready for review August 5, 2026 12:45
@Gonals
Gonals requested review from a team as code owners August 5, 2026 12:45
@melvin-bot
melvin-bot Bot requested review from a team, JmillsExpensify and abdulrahuman5196 and removed request for a team August 5, 2026 12:45
@melvin-bot

melvin-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

@abdulrahuman5196 Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

@melvin-bot
melvin-bot Bot requested review from ikevin127 and removed request for a team August 5, 2026 12:45
@melvin-bot

melvin-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

@ikevin127 Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba05f09ca3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".


// Whether each column may auto-detect its role from its header. Flows that compute the whole mapping in one
// coordinated pass (e.g. company cards) set this to false to avoid duplicate pre-selections.
shouldAutoDetectColumns?: boolean;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ CONSISTENCY-10 (docs)

Per STYLE.md, component props must be documented with a /** ... */ block comment above the prop, not a // line comment. The newly added shouldAutoDetectColumns prop is documented with // comments.

Convert the // comment into a JSDoc block:

/**
 * Whether each column may auto-detect its role from its header. Flows that compute the whole mapping in one
 * coordinated pass (e.g. company cards) set this to false to avoid duplicate pre-selections.
 */
shouldAutoDetectColumns?: boolean;

Reviewed at: ba05f09 | Please rate this suggestion with 👍 or 👎 to help us improve! Reactions are used to monitor reviewer efficiency.

@Gonals Gonals changed the title Improve column auto-mapping when the file structure is different [HOLD] Improve column auto-mapping when the file structure is different Aug 5, 2026
@ikevin127

Copy link
Copy Markdown
Contributor

This one is for @abdulrahuman5196 to review best on first C+ assignment, if I'm needed due to Abdul not being available - ping me, meanwhile I'll unsubscribe 🙌

@abdulrahuman5196

Copy link
Copy Markdown
Contributor

I can review, Is this ready for review? Hold is added

@JmillsExpensify JmillsExpensify left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No product review required on this one.

@Gonals Gonals changed the title [HOLD] Improve column auto-mapping when the file structure is different Improve column auto-mapping when the file structure is different Aug 27, 2026
@Gonals

Gonals commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

@abdulrahuman5196 All yours

@abdulrahuman5196

Copy link
Copy Markdown
Contributor

@codex Review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

We were unable to download your code in a timely manner.
ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@abdulrahuman5196

Copy link
Copy Markdown
Contributor

@codex review

@melvin-bot
melvin-bot Bot requested a review from chiragsalian September 13, 2026 18:21
@melvin-bot

melvin-bot Bot commented Sep 13, 2026

Copy link
Copy Markdown

@chiragsalian Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

@melvin-bot melvin-bot Bot changed the title Improve column auto-mapping when the file structure is different [Payment due @abdulrahuman5196] Improve column auto-mapping when the file structure is different Sep 13, 2026
@melvin-bot

melvin-bot Bot commented Sep 13, 2026

Copy link
Copy Markdown

🎯 @abdulrahuman5196, thanks for reviewing and testing this PR! 🎉

A payment issue will be created for your review once this PR is deployed to production.
E/E issue linked to the PR - https://www.github.com/Expensify/Expensify/issues/662378.

If payment is not needed (e.g., regression PR review fix etc), react with 👎 to this comment to prevent the payment issue from being created.

@chiragsalian

Copy link
Copy Markdown
Contributor

Looks like you have a valid AI review you didnt handle - #96654 (comment).

The rest of the code LGTM

@chiragsalian

Copy link
Copy Markdown
Contributor

Bump any update here?

@Gonals

Gonals commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

Bump any update here?

All yours! Sorry about the delay! Work got in the way of work!

@chiragsalian chiragsalian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chiragsalian
chiragsalian merged commit 920d627 into main Sep 17, 2026
34 of 35 checks passed
@chiragsalian
chiragsalian deleted the alberto-betterMaps branch September 17, 2026 16:11
@github-actions

Copy link
Copy Markdown
Contributor

🚧 chiragsalian has triggered a test Expensify/App build. You can view the workflow run here.

@OSBotify

Copy link
Copy Markdown
Contributor

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

@OSBotify

Copy link
Copy Markdown
Contributor

🚀 Deployed to staging by https://github.com/chiragsalian in version: 9.4.82-0 🚀

platform result
🕸 web 🕸 success ✅
🤖 android 🤖 cancelled 🔪
🍎 iOS 🍎 cancelled 🔪

@MelvinBot

MelvinBot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Yes — help site changes are required. Draft PR: Docs updates for E/A#96654 Company card CSV column auto-mapping behavior

One article documents this flow, and it described the old behavior: Import-Company-Card-Transactions-From-a-Spreadsheet.md said "Previously mapped fields auto-fill to save time" and never mentioned that mappings are pre-filled from the file's column headers at all.

What the article now says, matching getCompanyCardColumnMappings:

  • Mappings are pre-filled from the header row of the file you just uploaded.
  • A field is pre-filled on one column at most.
  • The saved layout is only a positional fallback for fields the headers don't identify, and only onto columns still set to Ignore.
  • Unrecognized columns stay on Ignore rather than getting a guessed field.
  • Unique ID is still never filled in.
Changes made and how labels were verified

Changes (all in docs/articles/new-expensify/connect-credit-cards/Import-Company-Card-Transactions-From-a-Spreadsheet.md):

  • New ## How Expensify fills in your field mappings section.
  • Step 7 of the import steps now says to review the pre-filled mappings.
  • Rewrote the auto-fill note under ## How to upload additional transactions to an existing card feed.
  • Updated the ## Can I change field mappings after importing transactions? FAQ, plus two new FAQs: why fields arrive already filled in, and why a field mapped last time can come back empty.
  • Added field/column mapping keywords and updated internalScope.

Labels verified against the strings the app renders in src/languages/en.ts — Card number, Card name, Date, Merchant, Amount, Currency, Unique ID, Ignore, Import, Add cards, Import transactions from file, Import spreadsheet. A live-UI pass confirmed the Add cards button but couldn't reach the mapping screen: the add-cards flow requires an account-validation magic code, and file upload isn't drivable on the web test harness.

No other article needed changes. The shouldAutoDetectColumns default stays true, so per-column header detection is unchanged for members, categories, tags, merchant rules, per diem, and personal card imports.

Not run: cspell on the changed file — the sandbox blocked the command. The added wording is ordinary English, but CI will confirm.

@Gonals, please review the linked help site PR and confirm it reflects the current behavior. Then mark the linked help site PR Ready for review


view run · view recording

@OSBotify

Copy link
Copy Markdown
Contributor

🚀 Deployed to staging by https://github.com/chiragsalian in version: 9.4.86-0 🚀

platform result
🕸 web 🕸 success ✅
🤖 android 🤖 success ✅
🍎 iOS 🍎 success ✅

@MelvinBot

Copy link
Copy Markdown
Contributor

🤖 Yes, help site changes were required. Draft PR: Docs updates for E/A#96654 header-first company card CSV column mapping

One article needed updating — Import Company Card Transactions From a Spreadsheet. It described the auto-fill only as "previously mapped fields auto-fill," which no longer matches what admins see now that mappings are matched from the uploaded file's column headers first and fall back to saved column positions second.

What the docs PR changes:

  • Adds a ## How Expensify fills in company card field mappings section covering the header-first order, the saved-position fallback, columns left as Ignore, and the limits.
  • Rewrites step 7 of the import flow to say review the mappings rather than set them from scratch.
  • Updates the re-upload note and the "Can I change field mappings" FAQ to name both sources of the auto-fill.
  • Adds three FAQs: whether mappings survive a differently structured file, why a column still shows Ignore, and a reworded duplicate-mapping question — the old one asked about mapping the same column twice, but the error the app raises is about mapping a single field to multiple columns.

No other article needs changing. shouldAutoDetectColumns={false} is set only on src/pages/workspace/companyCards/addNew/CompanyCardsImportedPage.tsx:264, so the member, category, tag, per diem, merchant rules, and personal card import flows keep their old per-column detection.

Two accuracy details worth confirming

Both were checked against src/libs/importSpreadsheetUtils.ts:243-290:

  • Unique ID is still never auto-filled. findColumnName has no case returning EXTERNAL_ID, and the saved-mapping fallback skips it explicitly, so the article's existing Unique ID guidance stays correct. Worth noting because the code comment at line 276 reads as though the detection pass could claim it from a header — today it can't.
  • The header row is read regardless of the File contains column headers toggle. Detection always reads row 0 and the mapping pass doesn't depend on containsHeader, so the article ties header matching to "your file has a header row" rather than to that toggle. Flag it if that was meant to be gated.

Label verification: Company cards and Add cards were confirmed against dev NewDot web. The rest — Import transactions from file, Import spreadsheet, Card number, Card name, Date, Merchant, Amount, Currency, Unique ID, Ignore, and Import — were confirmed against src/languages/en.ts, because the live Add cards flow is gated behind an account-validation magic code the test session couldn't clear.

The docs PR has no linked issue: this PR's own linked issue is internal, so it isn't referenced from a public docs PR. Add one if the automation needs it.

@Gonals, please review the linked help site PR and confirm it reflects the current behavior. Then mark the linked help site PR Ready for review

@OSBotify

Copy link
Copy Markdown
Contributor

🚀 Deployed to staging by https://github.com/chiragsalian in version: 9.4.89-0 🚀

platform result
🕸 web 🕸 success ✅
🤖 android 🤖 success ✅
🍎 iOS 🍎 success ✅

@OSBotify

Copy link
Copy Markdown
Contributor

🚀 Deployed to production by https://github.com/luacmartins in version: 9.4.89-0 🚀

platform result
🕸 web 🕸 success ✅
🤖 android 🤖 success ✅
🍎 iOS 🍎 failure ❌

Bundle Size Analysis (Sentry):

@melvin-bot

melvin-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

🤖 Payment issue created: #101623

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants