Add Amazon Seller Central and Vendor Central templates - #1
Open
jthordal wants to merge 2 commits into
Open
Conversation
Two new integrations, each in both weld/ and dbt/ dialects, following the shopify/ structure: raw -> staging -> core -> analytics, plus data tests. Amazon Seller Central (6 core models). Rebuilds the Business Reports, then goes past them to the settlement report - the only table that knows what Amazon actually paid. Handles the four things that break Amazon reporting: report generations duplicating on every sync, the three ASIN reports being the same sales at three granularities, traffic being measured on ASINs rather than SKUs, and settlement data being deleted after 89 days. Amazon Vendor Central (4 core models). Sales where Amazon is the customer, so shipped_cogs is your revenue and shipped_revenue is Amazon's. Every sales report arrives four times (distributor view x selling program) describing overlapping ASIN sets, so both columns are in the grain everywhere and the analytics layer pins MANUFACTURING + RETAIL. Seller Central columns are verified against Weld's published connector schema. Vendor Central columns follow Amazon's own report schemas flattened by Weld's naming convention, which was validated against the Seller Central ERD. All 122 SQL files parse as BigQuery; refs, cross-layer columns and test columns are checked. Not run end to end - reconcile before relying on the output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds two new Amazon integrations (amazon-seller-central/ and amazon-vendor-central/) to the SQL templates library, each delivered in both Weld and dbt dialects and organized into raw → staging → core → analytics layers.
Changes:
- Add Amazon Vendor Central staging/core/analytics models plus dbt equivalents and data tests.
- Add Amazon Seller Central staging/core/analytics models plus dbt equivalents and data tests.
- Update root README to list the two new integrations.
Reviewed changes
Copilot reviewed 91 out of 91 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Adds Amazon Seller/Vendor Central to integration index |
| amazon-vendor-central/README.md | Documents Vendor Central model purpose, grains, and usage notes |
| amazon-vendor-central/weld/staging/sales.sql | Staging wrapper for vendor sales report (casts, renames, de-dupes) |
| amazon-vendor-central/weld/staging/traffic.sql | Staging wrapper for vendor traffic (glance views) report |
| amazon-vendor-central/weld/staging/margin.sql | Staging wrapper for NPPM (Amazon’s margin) report |
| amazon-vendor-central/weld/staging/inventory.sql | Staging wrapper for vendor inventory report |
| amazon-vendor-central/weld/staging/forecast.sql | Staging wrapper for vendor forecasting report (latest generation per window) |
| amazon-vendor-central/weld/staging/real_time_sales.sql | Staging wrapper for real-time vendor sales report (hourly) |
| amazon-vendor-central/weld/core/sales_over_time.sql | Core daily vendor sales rollup with wholesale vs retail separation |
| amazon-vendor-central/weld/core/sales_by_asin.sql | Core ASIN/day join of sales + traffic + margin |
| amazon-vendor-central/weld/core/inventory_health.sql | Core vendor inventory scorecard with sales velocity-derived ratios |
| amazon-vendor-central/weld/core/forecast_vs_actuals.sql | Core forecast window scoring vs shipped/ordered actuals |
| amazon-vendor-central/weld/analytics/sales_over_time.sql | Analytics contract over core sales_over_time with default view/program filter |
| amazon-vendor-central/weld/analytics/sales_by_asin.sql | Analytics contract over core sales_by_asin with default view/program filter |
| amazon-vendor-central/weld/analytics/inventory_health.sql | Analytics contract over core inventory_health with default view/program filter |
| amazon-vendor-central/weld/analytics/forecast_vs_actuals.sql | Analytics contract over core forecast_vs_actuals filtering out IN_PROGRESS |
| amazon-vendor-central/dbt/README.md | Describes dbt drop-in structure, materialization, and assumptions |
| amazon-vendor-central/dbt/models/staging/sources.yml | Defines dbt sources for Vendor Central raw tables |
| amazon-vendor-central/dbt/models/staging/stg_amazon_vendor__sales.sql | dbt staging vendor sales wrapper (materialized table) |
| amazon-vendor-central/dbt/models/staging/stg_amazon_vendor__traffic.sql | dbt staging vendor traffic wrapper |
| amazon-vendor-central/dbt/models/staging/stg_amazon_vendor__margin.sql | dbt staging vendor margin wrapper |
| amazon-vendor-central/dbt/models/staging/stg_amazon_vendor__inventory.sql | dbt staging vendor inventory wrapper |
| amazon-vendor-central/dbt/models/staging/stg_amazon_vendor__forecast.sql | dbt staging vendor forecast wrapper |
| amazon-vendor-central/dbt/models/staging/stg_amazon_vendor__real_time_sales.sql | dbt staging real-time vendor sales wrapper |
| amazon-vendor-central/dbt/models/core/core_amazon_vendor__sales_over_time.sql | dbt core vendor sales_over_time (materialized table) |
| amazon-vendor-central/dbt/models/core/core_amazon_vendor__sales_by_asin.sql | dbt core sales_by_asin (partitioned table) |
| amazon-vendor-central/dbt/models/core/core_amazon_vendor__inventory_health.sql | dbt core inventory_health (partitioned table) |
| amazon-vendor-central/dbt/models/core/core_amazon_vendor__forecast_vs_actuals.sql | dbt core forecast_vs_actuals (materialized table) |
| amazon-vendor-central/dbt/models/core/core_amazon_vendor__sales_over_time.yml | dbt schema tests + accepted values + expression checks for core models |
| amazon-vendor-central/dbt/models/analytics/analytics__amazon_vendor_sales_over_time.sql | dbt analytics view contract with default view/program filter |
| amazon-vendor-central/dbt/models/analytics/analytics__amazon_vendor_sales_by_asin.sql | dbt analytics view contract with default view/program filter |
| amazon-vendor-central/dbt/models/analytics/analytics__amazon_vendor_inventory_health.sql | dbt analytics view contract with default view/program filter |
| amazon-vendor-central/dbt/models/analytics/analytics__amazon_vendor_forecast_vs_actuals.sql | dbt analytics view contract excluding IN_PROGRESS |
| amazon-vendor-central/dbt/tests/assert_no_duplicate_report_generations.sql | dbt test: staging sales de-dupe invariant |
| amazon-vendor-central/dbt/tests/assert_distributor_views_are_not_summed.sql | dbt test: quantify overlap between distributor views |
| amazon-vendor-central/dbt/tests/assert_traffic_is_not_multiplied_by_views.sql | dbt test: prevent traffic inflation across view/program variants |
| amazon-vendor-central/dbt/tests/assert_cogs_does_not_exceed_retail.sql | dbt test: catch shipped_cogs vs shipped_revenue swapped mapping |
| amazon-seller-central/README.md | Documents Seller Central model purpose, grains, and usage notes |
| amazon-seller-central/weld/staging/orders.sql | Staging wrapper for order line report (casts + discount sign convention) |
| amazon-seller-central/weld/staging/returns.sql | Staging wrapper for MFN returns (sign conventions + metadata) |
| amazon-seller-central/weld/staging/fba_returns.sql | Staging wrapper for FBA returns (unit-level, signed quantity) |
| amazon-seller-central/weld/staging/settlement.sql | Staging wrapper for settlement report with ledger classification |
| amazon-seller-central/weld/staging/settlement_period.sql | Staging wrapper for settlement period totals/deposit metadata |
| amazon-seller-central/weld/staging/sales_and_traffic_by_date.sql | Staging wrapper for business report by date (de-dupe generations) |
| amazon-seller-central/weld/staging/sales_and_traffic_by_asin.sql | Staging wrapper for business report by ASIN/SKU (de-dupe generations) |
| amazon-seller-central/weld/staging/listings.sql | Staging SKU dimension mapping SKU ↔ ASIN plus listing attributes |
| amazon-seller-central/weld/staging/fba_inventory.sql | Staging wrapper for FBA inventory snapshot with grouped buckets |
| amazon-seller-central/weld/staging/fba_reimbursements.sql | Staging wrapper for FBA reimbursements (casts) |
| amazon-seller-central/weld/staging/sku_cost.sql | Stub model for user-supplied SKU cost with validity window |
| amazon-seller-central/weld/core/product_sales_over_time.sql | Core event-based sales equation (SALE/RETURN rows) |
| amazon-seller-central/weld/core/sales_over_time.sql | Core reconstruction of Business Report sales/traffic by date |
| amazon-seller-central/weld/core/settlement_ledger.sql | Core pivot of settlement ledger into P&L-like columns |
| amazon-seller-central/weld/core/asin_profitability.sql | Core join of sales/fees/returns/COGS into contribution metrics |
| amazon-seller-central/weld/core/traffic_and_conversion.sql | Core traffic rolled up to ASIN grain with correct denominators |
| amazon-seller-central/weld/core/inventory_health.sql | Core FBA inventory + trailing sales velocity cover metrics |
| amazon-seller-central/weld/analytics/sales_over_time.sql | Analytics contract over core sales_over_time |
| amazon-seller-central/weld/analytics/product_sales_over_time.sql | Analytics contract over core product_sales_over_time (excludes cancelled lines) |
| amazon-seller-central/weld/analytics/settlement_ledger.sql | Analytics contract over core settlement_ledger (keeps account-level charges) |
| amazon-seller-central/weld/analytics/asin_profitability.sql | Analytics contract over core asin_profitability |
| amazon-seller-central/weld/analytics/traffic_and_conversion.sql | Analytics contract over core traffic_and_conversion |
| amazon-seller-central/weld/analytics/inventory_health.sql | Analytics contract over core inventory_health |
| amazon-seller-central/dbt/README.md | Describes dbt drop-in structure, materialization, and assumptions |
| amazon-seller-central/dbt/models/staging/sources.yml | Defines dbt sources for Seller Central raw tables + google_sheets COGS input |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__orders.sql | dbt staging orders wrapper (materialized table) |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__returns.sql | dbt staging MFN returns wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__fba_returns.sql | dbt staging FBA returns wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__settlement.sql | dbt staging settlement wrapper (materialized table) |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__settlement_period.sql | dbt staging settlement period wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__sales_and_traffic_by_date.sql | dbt staging business report by date wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__sales_and_traffic_by_asin.sql | dbt staging business report by ASIN/SKU wrapper (materialized table) |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__listings.sql | dbt staging listings/SKU dimension wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__fba_inventory.sql | dbt staging FBA inventory snapshot wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__fba_reimbursements.sql | dbt staging reimbursements wrapper |
| amazon-seller-central/dbt/models/staging/stg_amazon_seller__sku_cost.sql | dbt staging stub for user-supplied SKU costs |
| amazon-seller-central/dbt/models/core/core_amazon_seller__product_sales_over_time.sql | dbt core sales equation at event grain (partitioned table) |
| amazon-seller-central/dbt/models/core/core_amazon_seller__sales_over_time.sql | dbt core business report sales/traffic by date (materialized table) |
| amazon-seller-central/dbt/models/core/core_amazon_seller__settlement_ledger.sql | dbt core settlement ledger pivot (partitioned table) |
| amazon-seller-central/dbt/models/core/core_amazon_seller__asin_profitability.sql | dbt core profitability join (partitioned table) |
| amazon-seller-central/dbt/models/core/core_amazon_seller__traffic_and_conversion.sql | dbt core ASIN-grain traffic + conversion (partitioned table) |
| amazon-seller-central/dbt/models/core/core_amazon_seller__inventory_health.sql | dbt core inventory health (materialized table) |
| amazon-seller-central/dbt/models/core/core_amazon_seller__product_sales_over_time.yml | dbt schema tests + expression checks for seller core models |
| amazon-seller-central/dbt/models/analytics/analytics__amazon_seller_sales_over_time.sql | dbt analytics view contract |
| amazon-seller-central/dbt/models/analytics/analytics__amazon_seller_product_sales_over_time.sql | dbt analytics view contract excluding cancelled lines |
| amazon-seller-central/dbt/models/analytics/analytics__amazon_seller_settlement_ledger.sql | dbt analytics view contract keeping account-level charges |
| amazon-seller-central/dbt/models/analytics/analytics__amazon_seller_asin_profitability.sql | dbt analytics view contract |
| amazon-seller-central/dbt/models/analytics/analytics__amazon_seller_traffic_and_conversion.sql | dbt analytics view contract |
| amazon-seller-central/dbt/models/analytics/analytics__amazon_seller_inventory_health.sql | dbt analytics view contract |
| amazon-seller-central/dbt/tests/assert_settlement_ties_to_deposits.sql | dbt test: ledger sums to Amazon’s deposit total |
| amazon-seller-central/dbt/tests/assert_sales_equation_holds.sql | dbt test: net = gross + discounts + returns |
| amazon-seller-central/dbt/tests/assert_product_sales_track_business_report.sql | dbt test: line-grain model tracks business report at month grain |
| amazon-seller-central/dbt/tests/assert_no_duplicate_report_generations.sql | dbt test: de-dupe generations invariant for business report by date |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+169
to
+170
| LEFT JOIN {{staging.amazon_seller.listings}} l | ||
| ON l.amazon_seller = f.amazon_seller AND l.sku = f.sku |
Comment on lines
+174
to
+175
| LEFT JOIN {{ ref('stg_amazon_seller__listings') }} l | ||
| ON l.amazon_seller = f.amazon_seller AND l.sku = f.sku |
Comment on lines
+50
to
+72
| CASE | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemprice' | ||
| AND LOWER(CAST(amount_description AS STRING)) IN ('principal', 'shipping', 'giftwrap', 'shippingcharge') | ||
| THEN 'revenue' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemprice' | ||
| AND LOWER(CAST(amount_description AS STRING)) LIKE '%tax%' | ||
| THEN 'tax_collected' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemprice' | ||
| AND LOWER(CAST(amount_description AS STRING)) LIKE '%discount%' | ||
| THEN 'promotion' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'promotion' THEN 'promotion' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemfees' THEN 'selling_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'orderfee' THEN 'selling_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemwithheldtax' THEN 'tax_withheld' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'orderwithheldtax' THEN 'tax_withheld' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) LIKE 'fba%' THEN 'fba_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'servicefee' THEN 'service_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'costofadvertising' THEN 'advertising' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) LIKE '%reimbursement%' THEN 'reimbursement' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'other-transaction' THEN 'other' | ||
| WHEN CAST(amount_type AS STRING) IS NULL THEN 'unclassified' | ||
| ELSE 'other' | ||
| END AS ledger_category, |
Comment on lines
+77
to
+78
| LOWER(CAST(transaction_type AS STRING)) IN ('refund', 'chargeback', 'guaranteeclaim') | ||
| AS is_refund |
Comment on lines
+52
to
+74
| CASE | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemprice' | ||
| AND LOWER(CAST(amount_description AS STRING)) IN ('principal', 'shipping', 'giftwrap', 'shippingcharge') | ||
| THEN 'revenue' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemprice' | ||
| AND LOWER(CAST(amount_description AS STRING)) LIKE '%tax%' | ||
| THEN 'tax_collected' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemprice' | ||
| AND LOWER(CAST(amount_description AS STRING)) LIKE '%discount%' | ||
| THEN 'promotion' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'promotion' THEN 'promotion' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemfees' THEN 'selling_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'orderfee' THEN 'selling_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'itemwithheldtax' THEN 'tax_withheld' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'orderwithheldtax' THEN 'tax_withheld' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) LIKE 'fba%' THEN 'fba_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'servicefee' THEN 'service_fee' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'costofadvertising' THEN 'advertising' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) LIKE '%reimbursement%' THEN 'reimbursement' | ||
| WHEN LOWER(CAST(amount_type AS STRING)) = 'other-transaction' THEN 'other' | ||
| WHEN CAST(amount_type AS STRING) IS NULL THEN 'unclassified' | ||
| ELSE 'other' | ||
| END AS ledger_category, |
Comment on lines
+79
to
+80
| LOWER(CAST(transaction_type AS STRING)) IN ('refund', 'chargeback', 'guaranteeclaim') | ||
| AS is_refund |
Comment on lines
+28
to
+32
| SUM(shipped_units) AS shipped_units_30d, | ||
| SAFE_DIVIDE(SUM(shipped_units), 30.0) AS shipped_units_per_day | ||
| FROM {{staging.amazon_vendor.sales}} | ||
| WHERE date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) | ||
| GROUP BY 1, 2, 3, 4, 5 |
Comment on lines
+32
to
+36
| amazon_vendor, marketplace, distributor_view, selling_program, asin, | ||
| SUM(shipped_units) AS shipped_units_30d, | ||
| SAFE_DIVIDE(SUM(shipped_units), 30.0) AS shipped_units_per_day | ||
| FROM {{ ref('stg_amazon_vendor__sales') }} | ||
| WHERE date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) |
The FBA returns report carries no marketplace column, so it was resolved by joining staging.amazon_seller.listings on (amazon_seller, sku). For any account listing the same SKU in more than one marketplace that join fans out, producing one RETURN row per marketplace and silently multiplying FBA returns - and the comment claimed the opposite, that the marketplace would come back NULL. Collapse listings to exactly one row per SKU first, resolving marketplace only where it is unambiguous and returning NULL otherwise. A NULL marketplace on a RETURN row is visible and auditable; a wrong one is neither, and a duplicated one is worse than both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two new integrations alongside
shopify/, each shipping both dialects (weld/anddbt/) and following the same raw → staging → core → analytics structure.Amazon Seller Central — 6 core models
Rebuilds the Business Reports, then goes past them to the settlement report, which is the only table that knows what Amazon actually paid you.
sales_over_timeproduct_sales_over_timesettlement_ledgerasin_profitabilitytraffic_and_conversioninventory_healthIt handles the four things that break Amazon reporting in practice:
SUM(sessions)over SKUs over-counts and every conversion rate derived from it is understated by that factor.Amazon Vendor Central — 4 core models
Sales where Amazon is the customer, which changes everything: there are no orders, revenue is recognised on shipment, and
shipped_cogs(Amazon's cost of goods) is your revenue whileshipped_revenueis Amazon's. The field naming invites the most expensive mix-up in vendor reporting, so the models and READMEs are explicit about it.Every sales and inventory report arrives four times —
distributor_view×selling_program— describing overlapping ASIN sets. Both columns are in the grain of every core model to force the choice, and the analytics layer pinsMANUFACTURING + RETAIL.assert_distributor_views_are_not_summed.sqlmeasures how much a given catalogue actually overlaps.Data tests
Eight, weighted toward the ones that check against something external. The most valuable is
assert_settlement_ties_to_deposits.sql:settlement_description.total_amountis Amazon's own figure for what it paid out and it appears on the bank statement, so if the classified ledger ties to it the fee classification is complete.Provenance of the schemas
amzn/selling-partner-api-models) flattened by Weld's naming convention. That convention was validated by round-tripping Amazon's Sales & Traffic schema against the Seller Central ERD, where every scalar matched.Validation
All 122 SQL files parse as BigQuery. Model refs, cross-layer column references, analytics filters and every test/schema-test column were checked programmatically. Not run end to end against a live warehouse — the READMEs say to reconcile a closed month, and a settlement against a bank statement, before relying on the output.
🤖 Generated with Claude Code