Skip to content

Comments

Add timeout enforcement and comprehensive tests for gojq middleware#961

Merged
lpcox merged 2 commits intomainfrom
claude/review-gojq-module
Feb 15, 2026
Merged

Add timeout enforcement and comprehensive tests for gojq middleware#961
lpcox merged 2 commits intomainfrom
claude/review-gojq-module

Conversation

@Claude
Copy link
Contributor

@Claude Claude AI commented Feb 15, 2026

Implements Priority 2 recommendations from the Go Fan module review for the gojq middleware, adding timeout protection and comprehensive test coverage.

Changes

Timeout Protection

  • Added DefaultJqTimeout constant (5 seconds) to prevent query hangs
  • Enhanced applyJqSchema() to automatically enforce timeout when context lacks a deadline
  • Preserves existing context deadlines when present
  • Improved error handling for timeout, cancellation, and compilation failures

Documentation

  • Updated code comments to reference gojq v0.12.18 features (536M element array limit, improved concurrency, enhanced type error messages)
  • Enhanced init() function documentation explaining fail-fast behavior
  • Added detailed timeout behavior documentation in applyJqSchema()

Test Coverage

  • Added 9 comprehensive timeout-related tests covering:
    • Default timeout application for contexts without deadlines
    • Context deadline preservation
    • Large array processing (10,000 elements)
    • Deeply nested structures (10 levels)
    • Compilation error scenarios
    • Context cancellation behavior

Code Quality

  • Fixed 4 unused require declarations in internal/sys/sys_test.go

Example

// Context without deadline automatically gets 5-second timeout
result, err := applyJqSchema(context.Background(), jsonData)

// Existing deadlines are preserved
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
result, err := applyJqSchema(ctx, jsonData)

All middleware tests pass (30+ test cases). The implementation leverages pre-existing query compilation caching (10-100x speedup) while adding robust timeout protection.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /tmp/go-build1592018226/b275/launcher.test /tmp/go-build1592018226/b275/launcher.test -test.testlogfile=/tmp/go-build1592018226/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a 64/src/math/exp_net/http/httptrace ache/go/1.25.7/x-lang=go1.25 get go-sdk/internal/-v ache/Python/3.12-buildid ache/go/1.25.7/xTaEENvXcKh3a5hEPOj9o/TaEENvXcKh3a5hEPOj9o abis�� 64/src/runtime/cgo1.25.7 HEAD x_amd64/vet --global fips140/alias (dns block)
    • Triggering command: /tmp/go-build3921665002/b275/launcher.test /tmp/go-build3921665002/b275/launcher.test -test.testlogfile=/tmp/go-build3921665002/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true /opt/hostedtoolcache/go/1.25.7/x64/src/net -I 2018226/b260/config.test --gdwarf-5 --64 -o 2018226/b260/conHEAD e=/t�� t0 m0s docker-compose (dns block)
    • Triggering command: /tmp/go-build1222286590/b275/launcher.test /tmp/go-build1222286590/b275/launcher.test -test.testlogfile=/tmp/go-build1222286590/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true MJ37E2qau -tests ker/docker-init go HEAD 64/pkg/tool/linux_amd64/compile ortcfg -d aw-mcpg/internal/testutil/mcptest/config.go aw-mcpg/internal/testutil/mcptest/driver.go /usr/local/sbin/bash g_.a go x_amd64/vet bash (dns block)
  • invalid-host-that-does-not-exist-12345.com
    • Triggering command: /tmp/go-build1592018226/b260/config.test /tmp/go-build1592018226/b260/config.test -test.testlogfile=/tmp/go-build1592018226/b260/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true sonrpc2/conn.go sonrpc2/frame.go .12/x64/bin/as ; s#git --global bin/git 64/pkg/include ortc�� 64/src/runtime/cgo 64/src/crypto/internal/fips140de--gdwarf2 inux.go esnew.go ocknew.go nix_cgo.go nix_cgo_res.go (dns block)
    • Triggering command: /tmp/go-build1856060892/b260/config.test /tmp/go-build1856060892/b260/config.test -test.testlogfile=/tmp/go-build1856060892/b260/testlog.txt -test.paniconexit0 -test.timeout=10m0s rev-�� --abbrev-ref HEAD cal/bin/git tion_pool.go er.go 64/pkg/tool/linux_amd64/vet base64 -d (dns block)
  • nonexistent.local
    • Triggering command: /tmp/go-build1592018226/b275/launcher.test /tmp/go-build1592018226/b275/launcher.test -test.testlogfile=/tmp/go-build1592018226/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a 64/src/math/exp_net/http/httptrace ache/go/1.25.7/x-lang=go1.25 get go-sdk/internal/-v ache/Python/3.12-buildid ache/go/1.25.7/xTaEENvXcKh3a5hEPOj9o/TaEENvXcKh3a5hEPOj9o abis�� 64/src/runtime/cgo1.25.7 HEAD x_amd64/vet --global fips140/alias (dns block)
    • Triggering command: /tmp/go-build3921665002/b275/launcher.test /tmp/go-build3921665002/b275/launcher.test -test.testlogfile=/tmp/go-build3921665002/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true /opt/hostedtoolcache/go/1.25.7/x64/src/net -I 2018226/b260/config.test --gdwarf-5 --64 -o 2018226/b260/conHEAD e=/t�� t0 m0s docker-compose (dns block)
    • Triggering command: /tmp/go-build1222286590/b275/launcher.test /tmp/go-build1222286590/b275/launcher.test -test.testlogfile=/tmp/go-build1222286590/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true MJ37E2qau -tests ker/docker-init go HEAD 64/pkg/tool/linux_amd64/compile ortcfg -d aw-mcpg/internal/testutil/mcptest/config.go aw-mcpg/internal/testutil/mcptest/driver.go /usr/local/sbin/bash g_.a go x_amd64/vet bash (dns block)
  • slow.example.com
    • Triggering command: /tmp/go-build1592018226/b275/launcher.test /tmp/go-build1592018226/b275/launcher.test -test.testlogfile=/tmp/go-build1592018226/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true g_.a 64/src/math/exp_net/http/httptrace ache/go/1.25.7/x-lang=go1.25 get go-sdk/internal/-v ache/Python/3.12-buildid ache/go/1.25.7/xTaEENvXcKh3a5hEPOj9o/TaEENvXcKh3a5hEPOj9o abis�� 64/src/runtime/cgo1.25.7 HEAD x_amd64/vet --global fips140/alias (dns block)
    • Triggering command: /tmp/go-build3921665002/b275/launcher.test /tmp/go-build3921665002/b275/launcher.test -test.testlogfile=/tmp/go-build3921665002/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true /opt/hostedtoolcache/go/1.25.7/x64/src/net -I 2018226/b260/config.test --gdwarf-5 --64 -o 2018226/b260/conHEAD e=/t�� t0 m0s docker-compose (dns block)
    • Triggering command: /tmp/go-build1222286590/b275/launcher.test /tmp/go-build1222286590/b275/launcher.test -test.testlogfile=/tmp/go-build1222286590/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true MJ37E2qau -tests ker/docker-init go HEAD 64/pkg/tool/linux_amd64/compile ortcfg -d aw-mcpg/internal/testutil/mcptest/config.go aw-mcpg/internal/testutil/mcptest/driver.go /usr/local/sbin/bash g_.a go x_amd64/vet bash (dns block)
  • this-host-does-not-exist-12345.com
    • Triggering command: /tmp/go-build1592018226/b284/mcp.test /tmp/go-build1592018226/b284/mcp.test -test.testlogfile=/tmp/go-build1592018226/b284/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/runtime/cgo Vl8znsXtV x_amd64/cgo rt-size '1280, 7/opt/hostedtoolcache/go/1.25.7/x64/pkg/tool/linux_amd64/compile fips140/check cal/bin/npx x_amd64/cgo ortc�� 0938582/b193/_pk-p mon/httpcommon.ggithub.com/github/gh-aw-mcpg/internal/logger/sanitize x_amd64/vet unset --global ache/Python/3.12--version x_amd64/vet (dns block)
    • Triggering command: /tmp/go-build1856060892/b284/mcp.test /tmp/go-build1856060892/b284/mcp.test -test.testlogfile=/tmp/go-build1856060892/b284/testlog.txt -test.paniconexit0 -test.timeout=10m0s rev-�� --abbrev-ref HEAD ache/go/1.25.7/x64/pkg/tool/linux_amd64/compile /opt/hostedtoolc/opt/hostedtoolcache/go/1.25.7/x64/pkg/tool/linux_amd64/link 0938582/b194/ ache/go/1.25.7/x/tmp/go-build1856060892/b260/config.test ache/go/1.25.7/x-importcfg de 1665002/b293/_pk-s HEAD 1665002/b293=> /guard/context.ggrep /guard/guard.go 64/pkg/tool/linu(create|run) git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[go-fan] Go Module Review: gojq - JSON Query Processing</issue_title>
<issue_description># 🐹 Go Fan Report: github.com/itchyny/gojq

Module Overview

gojq is a pure Go implementation of jq - the powerful JSON query language and processor. It provides both a CLI tool and a Go library for programmatically processing JSON data with jq queries. This module is critical for the MCP Gateway's middleware layer, enabling sophisticated JSON transformations and schema generation on MCP tool responses.

  • Version: v0.12.18 (latest)
  • Repository: https://github.com/itchyny/gojq
  • Stars: 3,692 ⭐
  • License: MIT
  • Last Update: Jan 31, 2026 (13 days ago - very active!)

Current Usage in gh-aw-mcpg

Based on GitHub code search, gojq is used in 2 files:

Files

  • internal/middleware/jqschema.go - Main implementation for jq-based schema generation
  • internal/middleware/jqschema_bench_test.go - Performance benchmarks

Key APIs Used

The middleware likely leverages:

  • gojq.Parse() - Parse jq query strings
  • gojq.Compile() - Compile queries into executable code
  • Code.Run() - Execute compiled queries on JSON data
  • Iterator pattern for efficient result processing

Context

The middleware uses gojq for:

  • JSON Schema Generation: Transform MCP tool response payloads into JSON schemas
  • Payload Processing: Handle large JSON responses from backend MCP servers
  • Performance: Active benchmarking indicates optimization focus

Research Findings

Recent Updates (v0.12.18 - December 2025)

🎉 Major improvements in latest release:

  1. New Functions

    • trimstr/1 - Efficient prefix/suffix removal (better than string slicing)
    • toboolean/0 - Clean type conversion
  2. Performance & Scale

    • 🚀 Array index limit increased to 536,870,912 (2^29 elements) - huge improvement!
    • 🚀 Stopped numeric normalization for concurrent execution - better parallel performance
    • ✨ Support for binding expressions with binary operators (1 + 2 as $x | -$x)
  3. Bug Fixes

    • 🐛 Fixed last/1 to be included in builtins/0
    • 🐛 Fixed --indent 0 to preserve newlines
    • 🐛 Fixed string repetition to emit error when result is too large

Very Recent Activity (January 2026)

  • Jan 31, 2026: Fixed type error messages for split() and match() functions
  • Jan 7, 2026: Updated copyright year and GitHub Actions
  • Ongoing: Active maintenance with regular updates

Best Practices from gojq Documentation

  1. Compile Once, Run Many: Compile queries once and reuse for massive performance gains
  2. Iterator Pattern: Use Run() which returns an iterator for memory-efficient processing
  3. Error Handling: Check both compilation errors (syntax) and runtime errors (types, null access)
  4. Custom Functions: Extend jq with Go functions using gojq.WithFunction()
  5. Variables: Pass variables to queries for dynamic behavior
  6. Memory Management: Be mindful of large arrays (now supports up to 536M elements!)

Improvement Opportunities

🏃 Quick Wins (High Impact, Low Effort)

1. Leverage New v0.12.18 Functions

Impact: Medium | Effort: Low

  • Use trimstr/1 instead of manual string slicing for prefix/suffix removal
  • Use toboolean/0 instead of custom type conversion logic
  • Benefit: Simpler, more readable jq queries with better performance

Example:

# Before
.[1:] | if . == "true" then true else false end

# After (with v0.12.18)
trimstr("x") | toboolean

2. Utilize Increased Array Index Limit

Impact: High | Effort: Low

v0.12.18 dramatically increased the array index limit to 536,870,912 elements (2^29):

  • Review any artificial limits or pagination in payload processing
  • Large MCP tool responses can now be handled directly without chunking
  • Benefit: Simpler code, better performance for large datasets

3. Improve Error Messages

Impact: Medium | Effort: Low

Recent fixes improved type error messages for split() and match():

  • Ensure error handling captures and logs these enhanced messages
  • Add context about which MCP server/tool caused the error
  • Benefit: Faster debugging and troubleshooting

✨ Feature Opportunities (High Impact, Medium/High Effort)

1. Query Compilation Caching 🔥

Impact: High | Effort: Medium

Problem: If jq queries are recompiled on every request, it wastes significant CPU.

Solution: Implement a compilation cache using sync.Map:

var compiledQueries sync.Map // Thread-safe cache

func getOrCompileQuery(queryStr string) (*gojq.Code, error) {
    // Check cache first
    if cached, ok := compiledQueries.Load(queryStr); ok {
        return cached.(*gojq.Code), nil
    }
    
    // Parse and compile
    query, err := gojq.Parse(queryStr)
    if err != nil {
        return nil, fmt.Errorf("failed to parse j...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes github/gh-aw-mcpg#921

Implements Priority 2 improvements from Go Fan review:
- Added automatic 5-second timeout for jq queries
- Enhanced documentation referencing gojq v0.12.18 features
- Added comprehensive unit tests for timeout behavior
- Fixed unused require declarations in sys_test.go

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
@Claude Claude AI changed the title [WIP] Review gojq module for JSON query processing Add timeout enforcement and comprehensive tests for gojq middleware Feb 15, 2026
@Claude Claude AI requested a review from lpcox February 15, 2026 14:25
@lpcox lpcox marked this pull request as ready for review February 15, 2026 15:27
Copilot AI review requested due to automatic review settings February 15, 2026 15:27
@lpcox lpcox merged commit c7a6fec into main Feb 15, 2026
5 checks passed
@lpcox lpcox deleted the claude/review-gojq-module branch February 15, 2026 15:27
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request implements Priority 2 recommendations from the Go Fan module review for the gojq middleware, adding robust timeout protection and comprehensive test coverage. The changes enhance the reliability of JSON query processing by preventing hangs from malformed queries or excessively large payloads.

Changes:

  • Added 5-second default timeout enforcement for jq query execution to prevent hangs
  • Enhanced documentation referencing gojq v0.12.18 features (536M element array limit, improved concurrency, better error messages)
  • Added 9 comprehensive timeout-related tests covering default timeout application, deadline preservation, large arrays, nested structures, compilation errors, and context cancellation
  • Fixed 4 unused require declarations in sys_test.go through formatting cleanup

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
internal/middleware/jqschema.go Added DefaultJqTimeout constant (5s), enhanced applyJqSchema() to automatically enforce timeout when context lacks deadline, improved error handling for timeouts/cancellation, updated documentation with gojq v0.12.18 features
internal/middleware/jqschema_test.go Added TestApplyJqSchema_TimeoutBehavior and TestApplyJqSchema_ContextTimeout test functions with 9 comprehensive test cases covering timeout behavior, large data processing, nested structures, and error scenarios
internal/sys/sys_test.go Code formatting improvements (indentation standardization) and removal of 4 unused require declarations in test functions that only use assert

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link
Contributor

Add timeout enforcement and comprehensive tests for gojq middleware
[test-improver] Improve tests for sys package
GitHub MCP (last 2 merged PRs): ✅
Serena activate_project: ✅
Playwright github title: ✅
File write: ✅
Bash cat: ✅
Agent-finished (make agent-finished): ❌
Overall status: FAIL

AI generated by Smoke Codex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants