From 73f1e4f2bb7e8b5883d2ade85b4057923272a2e3 Mon Sep 17 00:00:00 2001 From: Tim Veluwenkamp Date: Mon, 22 Dec 2025 15:02:02 +0100 Subject: [PATCH 1/5] + Improving Varnish Cache Hit Rate --- .../tools/improving-varnish-cache-hit.rate.md | 135 ++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md diff --git a/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md b/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md new file mode 100644 index 00000000..41ae7db6 --- /dev/null +++ b/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md @@ -0,0 +1,135 @@ +--- +myst: + html_meta: + description: Learn how to improve your Varnish cache hit rate on Hypernode by identifying automatic cache purges, analyzing hit/miss patterns, and optimizing URL normalization to boost performance and efficiency. + title: Improving Varnish Cache Hit Rate on Hypernode +--- + +# Improving Varnish Cache Hit Rate + +An improved **Varnish hit rate** ensures that more pages are served directly from cache. This reduces backend resource usage on your Hypernode and allows your shop to handle more concurrent visitors without performance degradation. +A low hit ratio usually indicates a caching misconfiguration or cache invalidation happening too frequently. + +--- + +## Understanding Cache Hit Rate + +- **HIT:** The response is served directly from Varnish cache +- **MISS:** The request is forwarded to the backend (for example, PHP/Magento) + +A consistently low hit rate means that: + +- Pages are bypassing Varnish +- Cache entries are being purged too often +- URLs are fragmented due to parameters or inconsistencies + +--- + +## Common Issue: Automatic Cache Busting + +If you notice that your cache is cleared at consistent times, this often points to an automated process that flushes the cache. + +### Typical causes + +- External integrations that frequently update stock or pricing +- Magento cron jobs triggering full cache purges +- Extensions that invalidate more cache than necessary + +### What to do + +- Review all third-party integrations that update catalog or pricing data +- Inspect Magento cron jobs for full cache invalidation tasks +- Reduce the scope or frequency of full cache purges +- Prefer targeted purges (specific URLs or entities) instead of clearing the entire cache + +--- + +## Checking Cacheability and Hit/Miss Behavior + +When Varnish is frequently purged or bypassed, your hit rate will drop. You can analyze this behavior using the tools below. + +### Useful tools + +#### `varnishlog` +View detailed logs of Varnish requests and responses. Look for recurring **MISS** patterns on URLs that should be cacheable. + +#### `varnishstat` +Provides counters for cache hits, misses, and backend requests. + +#### Hypernode Insights (if available) +Use hit/miss graphs to identify when cache invalidations occur and correlate them with deployments or cron activity. + +--- + +## Checking Varnish Headers Using cURL + +You can verify whether a page is cached directly from your own terminal. + +```bash +curl -I https://www.example.com/ \ + | egrep 'Age:|Via:|X-Cache|X-Magento-Cache-Debug|Cache-Control' +``` + +### What to look for + +- **Age** header increasing → cached response +- **X-Cache: HIT** → served from Varnish +- **Cache-Control** headers that allow caching +- Absence of **Set-Cookie** for cacheable pages + +--- + +## Checking Cache Statistics on Your Hypernode + +### Snapshot of hits and misses + +Untested example (single run): + +```bash +varnishstat -1 -f MAIN.cache_hit,MAIN.cache_miss,MAIN.backend_req +``` + +### Tested (live overview of cached URLs) + +```bash +varnishncsa -F '%U%q %{Varnish:hitmiss}x' | grep hit +``` + +This helps identify which URLs are effectively cached and which are not. + +--- + +## Handling Marketing and Tracking URL Parameters + +Marketing parameters such as `utm_source`, `utm_medium`, or `gclid` can dramatically increase the number of unique cache entries. +Each parameter variation creates a new cache object, lowering your overall hit rate. + +### Best practice + +Normalize URLs so that these parameters do not influence caching decisions. + +#### Examples of parameters to strip + +- `utm_*` +- `gclid` +- `fbclid` + +This normalization should happen before Varnish decides whether a request is cacheable. + +> **Tip:** The [`elgentos/magento2-varnish-extended`](https://github.com/elgentos/magento2-varnish-extended) extension improves handling of marketing parameters and enhances the default Magento 2 VCL. + +--- + +## URL Normalization + +Different URL variations can fragment your cache and reduce efficiency. + +### Common normalization examples + +- **Trailing slashes** + `/category` → `/category/` +- **Lowercase query parameters** + `?Color=Red` → `?color=red` +- **Remove session IDs or irrelevant parameters** + +By normalizing URLs, similar requests map to the same cache object, reducing duplication and improving hit rates. \ No newline at end of file From 9eed62c8f0cad2f88cf224d1767889642cddfa07 Mon Sep 17 00:00:00 2001 From: Tim Veluwenkamp Date: Tue, 23 Dec 2025 09:45:44 +0100 Subject: [PATCH 2/5] readability improved for improving-varnish-cache-hit.rate --- .../tools/improving-varnish-cache-hit.rate.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md b/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md index 41ae7db6..eb7c2b8a 100644 --- a/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md +++ b/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md @@ -71,11 +71,10 @@ curl -I https://www.example.com/ \ ``` ### What to look for - -- **Age** header increasing → cached response -- **X-Cache: HIT** → served from Varnish -- **Cache-Control** headers that allow caching -- Absence of **Set-Cookie** for cacheable pages +- **Age** header increasing → cached response. +- **X-Cache: HIT** → served from Varnish. +- **Cache-Control** → verifies cacheability. +- No **Set-Cookie** header → indicates a cacheable page. --- @@ -83,7 +82,7 @@ curl -I https://www.example.com/ \ ### Snapshot of hits and misses -Untested example (single run): +single run (untested): ```bash varnishstat -1 -f MAIN.cache_hit,MAIN.cache_miss,MAIN.backend_req @@ -101,7 +100,7 @@ This helps identify which URLs are effectively cached and which are not. ## Handling Marketing and Tracking URL Parameters -Marketing parameters such as `utm_source`, `utm_medium`, or `gclid` can dramatically increase the number of unique cache entries. +Marketing (query) parameters such as `utm_source`, `utm_medium`, or `gclid` can dramatically increase the number of unique cache entries. Each parameter variation creates a new cache object, lowering your overall hit rate. ### Best practice From 5590c0952736c0e16ad1d919750ea4b6df2bf1d9 Mon Sep 17 00:00:00 2001 From: Tim Veluwenkamp Date: Thu, 26 Feb 2026 11:25:58 +0100 Subject: [PATCH 3/5] Added varnish hitrate docs --- .../tools/improving-varnish-cache-hit-rate.md | 207 ++++++++++++++++++ .../tools/improving-varnish-cache-hit.rate.md | 134 ------------ 2 files changed, 207 insertions(+), 134 deletions(-) create mode 100644 docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md delete mode 100644 docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md diff --git a/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md b/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md new file mode 100644 index 00000000..9ed519bc --- /dev/null +++ b/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md @@ -0,0 +1,207 @@ +--- +myst: + html_meta: + description: Learn how to improve your Varnish cache hit rate on Hypernode by identifying automatic cache purges, analyzing hit/miss patterns, and optimizing URL normalization to boost performance and efficiency. + title: Improving Varnish Cache Hit Rate on Hypernode +--- + +# Improving Your Varnish Cache Hit Rate + +A higher Varnish cache hit rate means more pages are served directly from cache. +This reduces backend resource usage on your Hypernode, improves page load speed, and helps your shop handle more concurrent visitors without performance degradation. + +A low hit rate often indicates that cache isn’t being reused effectively, typically caused by misconfiguration, frequent invalidation, or too much variation in URLs. + +This guide takes you step-by-step from verifying that your cache is active to diagnosing and improving your hit ratio. + +## Before You Begin + +Typical cache hit rates: +- **Below 10%** → Cache is barely reused +- **30–70%** → Improvement possible (depends on shop type) +- **Above 80%** → Generally healthy + +Keep in mind: +- Staging environments typically have low hit rates +- B2B webshops often have lower hit rates due to personalization + + +# Step 1 — Verify Varnish Is Enabled + +Ensure Varnish is properly enabled on your Hypernode and configured in your +application (e.g. Magento 2). + +For Magento 2, verify: +- Varnish is selected as the caching application +- The correct VCL is generated and loaded +- Full Page Cache (FPC) is enabled + +For a complete guide on how to configure Varnish in Magento 2 see: +https://docs.hypernode.com/ecommerce-applications/magento-2/how-to-configure-varnish-for-magento-2-x.html#how-to-configure-varnish-for-magento-2-x + +```{tip} +Tip: The [elgentos/magento2-varnish-extended](https://github.com/elgentos/magento2-varnish-extended) extension improves Magento’s +default VCL configuration and marketing parameter handling. +``` + +# Step 2 — Check if Pages Are Being Cached + +Use `curl` to inspect response headers: + +```console +curl -I https://yourdomain.com +``` + +Look for: +- `X-Cache: HIT` → Served from Varnish +- `X-Cache: MISS` → Served from backend +- `Age` → How long the object has been cached +- `X-Magento-*` headers → Useful Magento cache debug info only visible when developer mode is enabled. + +If most responses return `MISS`, caching is not being reused effectively. + +You can also inspect these headers in your browser via: +Developer Tools → Network tab → Select request → Response Headers + +--- + +# Step 3 — Measure Your Cache Hit Rate + +Run: + +```console +varnishstat -1 -f MAIN.cache_hit -f MAIN.cache_miss +``` + +This shows: +- `MAIN.cache_hit` → Cached responses served +- `MAIN.cache_miss` → Requests sent to backend + +A high miss count relative to hits indicates room for improvement. + +For live monitoring: + +```console +varnishstat +``` + +--- + +# Step 4 — Common Causes of Low Hit Rates + +## 1. Pages Bypassing Varnish + +Some pages are intentionally not cached: +- Checkout +- Customer account pages +- Requests containing `Set-Cookie` headers + +This is expected behavior. + +## 2. Frequent Cache Invalidations + +If cache clears happen frequently, reuse becomes impossible. + +Common causes: +- Stock or pricing integrations +- Magento cron jobs performing full cache purges +- Extensions invalidating excessive cache entries + +Best practice: +Perform targeted purges (specific URLs or cache tags) instead of full cache +flushes. + +--- + +## 3. Marketing & Tracking Parameters + +Tracking parameters create separate cache entries for identical content. + +Examples: +- `utm_source` +- `utm_medium` +- `gclid` +- `fbclid` + +Example problem: +- /product-x +- /product-x?utm_source=google + +These generate separate cache objects unless normalized. + +Solution: + +Strip non-essential tracking parameters in VCL. + +```{tip} +The [elgentos/magento2-varnish-extended](https://github.com/elgentos/magento2-varnish-extended) module improves this behavior. +``` +--- + +## 4. URL Normalization Issues + +Different URL formats fragment your cache. + +Examples: +- `/category` vs `/category/` +- `?Color=Red` vs `?color=red` +- Unsorted query parameters +- Session IDs in URLs + +Normalize URLs to ensure identical content maps to a single cache object. + +--- + +## 5. Non-Cacheable Magento Blocks + +In Magento, a single block marked as non-cacheable can disable Full Page Cache +for the entire page. + +Search for non-cacheable blocks: + +```console +grep -R "cacheable=\"false\"" app/code vendor +``` +If found: +- Verify the block truly needs to be dynamic +- Remove cacheable="false" if unnecessary +- Use AJAX or Customer Data Sections for dynamic content + +Even one unnecessary non-cacheable block can severely impact hit rate. + +# Optional — Enable Magento Developer Mode for Debugging + +Developer mode provides more detailed error output: +```console +magerun2 deploy:mode:set developer +``` + +Or: +```console +php bin/magento deploy:mode:set developer +``` + +# Debugging Tools + +## varnishlog +Inspect detailed request handling: +```bash +varnishlog +``` +Look for recurring MISS patterns on pages that should be cacheable. + +## varnishncsa +Show hit/miss per URL: +```bash +varnishncsa -F '%U%q %{Varnish:hitmiss}x' +``` +Filter for hits: +```bash +varnishncsa -F '%U%q %{Varnish:hitmiss}x' | grep hit +``` + +## Hypernode Insights (If Available) +Use Hypernode Insights to: +- Monitor hit/miss ratios +- Detect purge spikes +- Correlate cache drops with deployments or cron jobs \ No newline at end of file diff --git a/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md b/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md deleted file mode 100644 index eb7c2b8a..00000000 --- a/docs/hypernode-platform/tools/improving-varnish-cache-hit.rate.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -myst: - html_meta: - description: Learn how to improve your Varnish cache hit rate on Hypernode by identifying automatic cache purges, analyzing hit/miss patterns, and optimizing URL normalization to boost performance and efficiency. - title: Improving Varnish Cache Hit Rate on Hypernode ---- - -# Improving Varnish Cache Hit Rate - -An improved **Varnish hit rate** ensures that more pages are served directly from cache. This reduces backend resource usage on your Hypernode and allows your shop to handle more concurrent visitors without performance degradation. -A low hit ratio usually indicates a caching misconfiguration or cache invalidation happening too frequently. - ---- - -## Understanding Cache Hit Rate - -- **HIT:** The response is served directly from Varnish cache -- **MISS:** The request is forwarded to the backend (for example, PHP/Magento) - -A consistently low hit rate means that: - -- Pages are bypassing Varnish -- Cache entries are being purged too often -- URLs are fragmented due to parameters or inconsistencies - ---- - -## Common Issue: Automatic Cache Busting - -If you notice that your cache is cleared at consistent times, this often points to an automated process that flushes the cache. - -### Typical causes - -- External integrations that frequently update stock or pricing -- Magento cron jobs triggering full cache purges -- Extensions that invalidate more cache than necessary - -### What to do - -- Review all third-party integrations that update catalog or pricing data -- Inspect Magento cron jobs for full cache invalidation tasks -- Reduce the scope or frequency of full cache purges -- Prefer targeted purges (specific URLs or entities) instead of clearing the entire cache - ---- - -## Checking Cacheability and Hit/Miss Behavior - -When Varnish is frequently purged or bypassed, your hit rate will drop. You can analyze this behavior using the tools below. - -### Useful tools - -#### `varnishlog` -View detailed logs of Varnish requests and responses. Look for recurring **MISS** patterns on URLs that should be cacheable. - -#### `varnishstat` -Provides counters for cache hits, misses, and backend requests. - -#### Hypernode Insights (if available) -Use hit/miss graphs to identify when cache invalidations occur and correlate them with deployments or cron activity. - ---- - -## Checking Varnish Headers Using cURL - -You can verify whether a page is cached directly from your own terminal. - -```bash -curl -I https://www.example.com/ \ - | egrep 'Age:|Via:|X-Cache|X-Magento-Cache-Debug|Cache-Control' -``` - -### What to look for -- **Age** header increasing → cached response. -- **X-Cache: HIT** → served from Varnish. -- **Cache-Control** → verifies cacheability. -- No **Set-Cookie** header → indicates a cacheable page. - ---- - -## Checking Cache Statistics on Your Hypernode - -### Snapshot of hits and misses - -single run (untested): - -```bash -varnishstat -1 -f MAIN.cache_hit,MAIN.cache_miss,MAIN.backend_req -``` - -### Tested (live overview of cached URLs) - -```bash -varnishncsa -F '%U%q %{Varnish:hitmiss}x' | grep hit -``` - -This helps identify which URLs are effectively cached and which are not. - ---- - -## Handling Marketing and Tracking URL Parameters - -Marketing (query) parameters such as `utm_source`, `utm_medium`, or `gclid` can dramatically increase the number of unique cache entries. -Each parameter variation creates a new cache object, lowering your overall hit rate. - -### Best practice - -Normalize URLs so that these parameters do not influence caching decisions. - -#### Examples of parameters to strip - -- `utm_*` -- `gclid` -- `fbclid` - -This normalization should happen before Varnish decides whether a request is cacheable. - -> **Tip:** The [`elgentos/magento2-varnish-extended`](https://github.com/elgentos/magento2-varnish-extended) extension improves handling of marketing parameters and enhances the default Magento 2 VCL. - ---- - -## URL Normalization - -Different URL variations can fragment your cache and reduce efficiency. - -### Common normalization examples - -- **Trailing slashes** - `/category` → `/category/` -- **Lowercase query parameters** - `?Color=Red` → `?color=red` -- **Remove session IDs or irrelevant parameters** - -By normalizing URLs, similar requests map to the same cache object, reducing duplication and improving hit rates. \ No newline at end of file From a5c49dc7f06cf9132951ed89960e26d278c41e3c Mon Sep 17 00:00:00 2001 From: Tim Veluwenkamp Date: Thu, 26 Feb 2026 11:41:26 +0100 Subject: [PATCH 4/5] Improved readability and links in varnish-hitrate-docs --- .../tools/improving-varnish-cache-hit-rate.md | 19 ++++--------------- 1 file changed, 4 insertions(+), 15 deletions(-) diff --git a/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md b/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md index 9ed519bc..6a2ce992 100644 --- a/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md +++ b/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md @@ -28,20 +28,19 @@ Keep in mind: # Step 1 — Verify Varnish Is Enabled -Ensure Varnish is properly enabled on your Hypernode and configured in your +Ensure Varnish is properly enabled on your vhost and configured in your application (e.g. Magento 2). For Magento 2, verify: +- That varnish is enabled on the vhost - Varnish is selected as the caching application - The correct VCL is generated and loaded - Full Page Cache (FPC) is enabled -For a complete guide on how to configure Varnish in Magento 2 see: -https://docs.hypernode.com/ecommerce-applications/magento-2/how-to-configure-varnish-for-magento-2-x.html#how-to-configure-varnish-for-magento-2-x +For a step-by-step guide on activating and configuring Varnish in Magento 2, please refer to our [documentation here](https://docs.hypernode.com/ecommerce-applications/magento-2/how-to-configure-varnish-for-magento-2-x.html#how-to-configure-varnish-for-magento-2-x) ```{tip} -Tip: The [elgentos/magento2-varnish-extended](https://github.com/elgentos/magento2-varnish-extended) extension improves Magento’s -default VCL configuration and marketing parameter handling. +Tip: The [elgentos/magento2-varnish-extended](https://github.com/elgentos/magento2-varnish-extended) extension improves Magento’s default VCL configuration and marketing parameter handling. ``` # Step 2 — Check if Pages Are Being Cached @@ -63,8 +62,6 @@ If most responses return `MISS`, caching is not being reused effectively. You can also inspect these headers in your browser via: Developer Tools → Network tab → Select request → Response Headers ---- - # Step 3 — Measure Your Cache Hit Rate Run: @@ -85,8 +82,6 @@ For live monitoring: varnishstat ``` ---- - # Step 4 — Common Causes of Low Hit Rates ## 1. Pages Bypassing Varnish @@ -111,8 +106,6 @@ Best practice: Perform targeted purges (specific URLs or cache tags) instead of full cache flushes. ---- - ## 3. Marketing & Tracking Parameters Tracking parameters create separate cache entries for identical content. @@ -130,13 +123,11 @@ Example problem: These generate separate cache objects unless normalized. Solution: - Strip non-essential tracking parameters in VCL. ```{tip} The [elgentos/magento2-varnish-extended](https://github.com/elgentos/magento2-varnish-extended) module improves this behavior. ``` ---- ## 4. URL Normalization Issues @@ -150,8 +141,6 @@ Examples: Normalize URLs to ensure identical content maps to a single cache object. ---- - ## 5. Non-Cacheable Magento Blocks In Magento, a single block marked as non-cacheable can disable Full Page Cache From 8682f6ae034502a18b1d20809b4a2370ec2d559a Mon Sep 17 00:00:00 2001 From: Tim Veluwenkamp Date: Thu, 26 Feb 2026 19:23:01 +0100 Subject: [PATCH 5/5] feedback based improvements to varnish-hitrate-docs --- .../tools/improving-varnish-cache-hit-rate.md | 30 ++++++++++++++----- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md b/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md index 6a2ce992..e9ee4077 100644 --- a/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md +++ b/docs/hypernode-platform/tools/improving-varnish-cache-hit-rate.md @@ -7,8 +7,8 @@ myst: # Improving Your Varnish Cache Hit Rate -A higher Varnish cache hit rate means more pages are served directly from cache. -This reduces backend resource usage on your Hypernode, improves page load speed, and helps your shop handle more concurrent visitors without performance degradation. +A higher Varnish cache hit rate means more requests are served directly from cache. +This reduces resource usage on your Hypernode, improves page load speed, and helps your shop handle more concurrent visitors without performance degradation. A low hit rate often indicates that cache isn’t being reused effectively, typically caused by misconfiguration, frequent invalidation, or too much variation in URLs. @@ -51,13 +51,21 @@ Use `curl` to inspect response headers: curl -I https://yourdomain.com ``` -Look for: -- `X-Cache: HIT` → Served from Varnish -- `X-Cache: MISS` → Served from backend -- `Age` → How long the object has been cached -- `X-Magento-*` headers → Useful Magento cache debug info only visible when developer mode is enabled. +Review the following headers: + +- **`Set-Cookie`** + If `PHPSESSID` is present, Varnish will not cache the response. + +- **`Cache-Control`** + Should **not** contain `private`, `no-store`, or `no-cache`. + +- **`Age`** + Indicates how long (in seconds) the object has been cached. -If most responses return `MISS`, caching is not being reused effectively. +- **`X-Magento-*`** + Provides Magento cache/debug information (visible in developer mode). + +If most responses return `MISS` (for example in `X-Magento-Cache-Debug` or similar headers), caching is not being reused effectively. You can also inspect these headers in your browser via: Developer Tools → Network tab → Select request → Response Headers @@ -82,6 +90,12 @@ For live monitoring: varnishstat ``` +Look for: +- `VCL_call HIT` → Served from Varnish +- `VCL_call MISS` → Served from backend +- `Age` → Indicates how long (in seconds) the object has been cached. +- `X-Magento-*` headers → Provides Magento cache/debug information (visible in developer mode). + # Step 4 — Common Causes of Low Hit Rates ## 1. Pages Bypassing Varnish