Skip to content

feat: Add gzip decompression and NDJSON support for Application Insig…#2

Merged
Rahulkumar010 merged 1 commit into
Rahulkumar010:mainfrom
ChNorman:fix/ndjson-support
Feb 13, 2026
Merged

feat: Add gzip decompression and NDJSON support for Application Insig…#2
Rahulkumar010 merged 1 commit into
Rahulkumar010:mainfrom
ChNorman:fix/ndjson-support

Conversation

@ChNorman

@ChNorman ChNorman commented Feb 4, 2026

Copy link
Copy Markdown
Contributor

Problem

The official Microsoft Application Insights SDK sends telemetry with two characteristics that azurinsight didn't handle:

  1. Gzip compression: Telemetry is sent with content-encoding: gzip header
  2. NDJSON format: Multiple telemetry items are batched as newline-delimited JSON, not a single JSON array

Without this support, the emulator would fail to parse Application Insights SDK telemetry with errors like:

  • "SyntaxError: Unexpected non-whitespace character after JSON at position X"
  • Telemetry silently lost or partially captured

Solution

Added gzip decompression middleware that:

  1. Detects gzipped requests by checking content-encoding: gzip header
  2. Decompresses the payload using Node.js built-in zlib.gunzipSync
  3. Parses NDJSON format by splitting on newlines and parsing each line as separate JSON
  4. Maintains backward compatibility by detecting single-line JSON and parsing normally
  5. Positioned correctly before bodyParser middleware to intercept and pre-process the request body

Testing

  • ✅ Verified with actual Application Insights SDK telemetry
  • ✅ Handles both single and batched telemetry items
  • ✅ Proper error handling for malformed payloads

Impact

This change makes azurinsight fully compatible with the official Microsoft Application Insights SDK, enabling developers to use it as a local development emulator without SDK-specific workarounds.

…hts SDK

## Problem
The official Microsoft Application Insights SDK sends telemetry with two characteristics that azurinsight didn't handle:

1. **Gzip compression**: Telemetry is sent with `content-encoding: gzip` header
2. **NDJSON format**: Multiple telemetry items are batched as newline-delimited JSON, not a single JSON array

Without this support, the emulator would fail to parse Application Insights SDK telemetry with errors like:
- "SyntaxError: Unexpected non-whitespace character after JSON at position X"
- Telemetry silently lost or partially captured

## Solution
Added gzip decompression middleware that:

1. **Detects gzipped requests** by checking `content-encoding: gzip` header
2. **Decompresses the payload** using Node.js built-in `zlib.gunzipSync`
3. **Parses NDJSON format** by splitting on newlines and parsing each line as separate JSON
4. **Maintains backward compatibility** by detecting single-line JSON and parsing normally
5. **Positioned correctly** before bodyParser middleware to intercept and pre-process the request body

## Testing
This fix has been production-tested in the SentimentIQ project with:
- ✅ 15/15 Application Insights integration tests passing
- ✅ Verified with actual Application Insights SDK telemetry
- ✅ Handles both single and batched telemetry items
- ✅ Proper error handling for malformed payloads

## Impact
This change makes azurinsight fully compatible with the official Microsoft Application Insights SDK, enabling developers to use it as a local development emulator without SDK-specific workarounds.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@Rahulkumar010

Copy link
Copy Markdown
Owner

Excellent contribution! 🎉

@ChNorman - Thank you for adding gzip/NDJSON support! This is a significant step makes azurinsight toward full compatibility with the official Microsoft Application Insights SDK. Great work on the implementation and testing! 👏

Really appreciate your contribution!

@Rahulkumar010 Rahulkumar010 merged commit 97971e7 into Rahulkumar010:main Feb 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants