[KrzysztofPajak]

Type: feature

Issue

On a large catalog every category page is slow, whatever the page or sort order. Measured on a test database with 50,073 products, 47,532 of them visible in one category (Release build, warm cache, one client, p50):

	before
category, page 1	3,145 ms
category, page 2	3,212 ms
category, other sort order	3,327 ms

Next to the page itself (11–18 ms), GetSearchProductsQueryHandler runs two queries that walk every matching product: the count for the pager (~0.85 s) and the aggregation of filterable specification option ids (~2.8 s). They ran on every request, although neither depends on the page index or the order. Rewriting the aggregation ($filter/$map, a different grouping, a forced collection scan) did not get it below ~2.5 s: the cost is the number of documents, not the shape of the query.

Solution

A storefront listing page opts in with GetSearchProductsQuery.CacheListingSummary (category, brand, collection, vendor and product tag pages). It reads its total and filter options from a cache entry and still queries the page live:

• CacheKey.PRODUCTS_LISTING_SUMMARY_KEY = Grand.product.listing-{hash}. The hash is taken over a JSON array of every request value that changes which products match: categories, featured flag, brand, collection, store, vendor, warehouse, product type, visibility/new/home-page/published flags, price, rating, tag, the allowed customer groups, filtered specs, specification options, keywords and the search mode. The page and the order are left out. Sets are sorted, and JSON escapes any separator inside a value and formats numbers invariantly.
• Lifetime: 5 minutes (ListingSummaryCacheMinutes).
• A short (last) page still gives an exact total. A full page never reports fewer products than it shows, and an empty page past the end never more than precede it, so a lagging total cannot contradict the page shown.
• On a miss, the count and the options run in parallel. The cache semaphore makes concurrent requests for the same listing wait for one computation instead of repeating the scan. That computation does not run under the first request's token, so one aborted request does not discard it for the others; each waiter still stops on its own cancellation.
• Everything else counts live as before: staff lists (ShowHidden), search, autocomplete, the sitemap, featured and new products.

Invalidation: PRODUCTS_LISTING_PATTERN_KEY is cleared by ProductService.UpdateProduct/UnPublishProduct, by product category/collection mapping insert/update/delete, by product specification attribute mapping insert/update/delete and by product tag attach/detach. Insert/delete product and UpdateProductField already clear the whole Grand.product. prefix. Every other write (stock, bulk imports, availability dates passing) is bounded by the 5-minute lifetime, as is a summary computed while an invalidation arrives: until then the pager may show a page too many, or a filter option may yield no products. The products on the page are always current.

After (same database and harness):

	before	after
category, page 1	3,145 ms	42 ms
category, page 2	3,212 ms	33 ms
category, other sort order	3,327 ms	34 ms
category filtered by one option, second request	–	142 ms

The pager total and the filter options match the database: 47,532 unfiltered, 47,529 with the filter.

Not addressed here:

• The first request for a listing still pays the full cost (6.6 s cold after a restart).
• Deep pages remain slow (page 3000: 8–14 s, a skip over 36k documents). They are reachable by crawlers through the server-rendered pager, so they need a separate fix.

Breaking changes

GetSearchProductsQueryHandler takes a new ICacheBase constructor parameter. DI resolves it, so only code that constructs the handler directly (tests, custom plugins) needs to pass it. GetSearchProductsQuery gets CacheListingSummary (default false). Behavior: the total and filter options may lag behind writes that do not clear them, by at most 5 minutes.

Testing

1. dotnet test src/Tests/Grand.Business.Catalog.Tests: all pass. The new tests cover:
   • a full page reuses the cached total;
   • a short page is exact;
   • a full page past a lagging total keeps the page within the total;
   • clearing the listing prefix recounts;
   • an aborted first request still caches the shared total for the next one;
   • a tag containing the field separator does not share a key with a different customer group set;
   • not opting in and ShowHidden count live;
   • customer groups and categories do not share a total.
2. Open a category on the storefront, then add a product to it in the admin (Catalog → Products → edit → Categories). Reload the category: the total and the product appear at once.
3. Open a category with a specification filter applied, and check that the filter options and the total match the products shown.
4. On a large catalog, compare the category response time on page 1 and page 2 before and after.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.