Skip to content

feat: gate subnet blocks on minimum unique-IP count with fraction/absolute threshold support#77

Merged
chinrichs-godaddy merged 11 commits into
godaddy:masterfrom
chinrichs-godaddy:feat/subnet-unique-ip-gate
Jul 13, 2026
Merged

feat: gate subnet blocks on minimum unique-IP count with fraction/absolute threshold support#77
chinrichs-godaddy merged 11 commits into
godaddy:masterfrom
chinrichs-godaddy:feat/subnet-unique-ip-gate

Conversation

@chinrichs-godaddy

Copy link
Copy Markdown
Collaborator

Summary

  • New behaviour: A subnet is only blocked when a configurable minimum number of distinct source IPs have been observed within it, preventing a single abusive IP from triggering a block for its entire /24 (or whatever subnetMaskBits is set to).
  • New option minSubnetUniqueIps (default 0.5): accepts a fraction (0, 1) — resolved as a percentage of the subnet's host count — or an absolute count ≥ 1. 0.5 on a /24 requires 128 unique IPs before the subnet can be blocked. Set to 1 to restore legacy behaviour.
  • New cluster option clusterMinSubnetUniqueIps (default 0.5): same fraction/absolute semantics as the local option. Cluster nodes publish per-subnet unique-IP counts to Redis via ZADD GT (max across nodes), and readThresholds gates subnet blocks on both the rate threshold and this count. The sliding window uses max(current, floor(prev × prevWeight)) for continuity at window boundaries.
  • Both options are resolved to absolute integers inside the ConnectQOS constructor via the new resolveUniqueIpThreshold helper. Metrics and ClusterSync always receive integers.
  • The local gate uses a "skip if count=0" design: direct isBadSubnet(subnetString) calls (no request context, so no IP timestamps populated) fall through to rate-only evaluation, preserving backward compatibility.

Changes

File What changed
src/metrics.ts Added minSubnetUniqueIps option, #subnetIpTimestamps LRU, getSubnetUniqueIpCount(), updated OnTrackFn signature to pass optional ip
src/connect.ts Added resolveUniqueIpThreshold() helper, wired minSubnetUniqueIps defaults, added unique-IP gate in getSubnetStatus, passes ip through trackRequesttrackSubnetonTrackClusterSync.recordHit
src/cluster.ts Added clusterMinSubnetUniqueIps option, #subnetIpSets per-window tracking, ZADD GT publish, two extra zrangebyscore reads in readThresholds, isSubnetUniqueIpThresholdMet(), stale key cleanup
README.md Documented both new options with fraction/absolute semantics in the main Options list and Cluster Options table
test/metrics.test.ts Updated onTrack subnet test; added 4 tests for getSubnetUniqueIpCount
test/connect.test.ts Added minSubnetUniqueIps: 1 to existing subnet tests; added 2 new scenario tests
test/cluster.test.ts Added zadd to mock pipeline; added 3 new tests for ZADD GT publish and readThresholds gating

Test plan

  • npm test — 126 tests pass, 0 failures
  • Verified: single-IP flood does not trigger subnet block when minSubnetUniqueIps > 1
  • Verified: subnet block fires once threshold unique IPs are observed
  • Verified: stale IPs (beyond maxAge) are not counted
  • Verified: cluster ZADD GT published with correct count per subnet per window
  • Verified: cluster gate prevents block when unique IP count < threshold
  • Verified: cluster gate allows block when both rate and unique IP threshold are met
  • Verified: clusterMinSubnetUniqueIps: 1 disables gate (existing cluster subnet test unchanged)
  • All existing subnet tests pass with minSubnetUniqueIps: 1 (legacy behaviour preserved)

🤖 Generated with Claude Code

…olute threshold support

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a “minimum unique source IPs per subnet” gate for subnet blocking (local and cluster modes) to avoid blocking an entire subnet due to a single abusive IP, with the threshold configurable as either a fraction of subnet size or an absolute count.

Changes:

  • Adds local unique-IP tracking in Metrics and gates ConnectQOS subnet blocking on a resolved integer minSubnetUniqueIps threshold.
  • Adds cluster-wide unique-IP aggregation (published via Redis ZADD GT) and gates cluster subnet blocks on both rate threshold and unique-IP count.
  • Updates documentation and expands test coverage for the new behavior and Redis publish/read logic.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/metrics.ts Adds minSubnetUniqueIps, per-subnet IP last-seen tracking, and getSubnetUniqueIpCount(); extends onTrack signature.
src/connect.ts Resolves unique-IP thresholds, wires through IP context, and adds unique-IP gating for subnet status and cluster subnet blocks.
src/cluster.ts Tracks/publishes per-subnet unique-IP counts and gates subnet blocking on unique-IP threshold during readThresholds().
README.md Documents minSubnetUniqueIps and clusterMinSubnetUniqueIps with fraction/absolute semantics.
test/metrics.test.ts Updates onTrack expectations and adds getSubnetUniqueIpCount() tests.
test/connect.test.ts Adjusts existing subnet tests for legacy behavior and adds new unique-IP gating scenarios.
test/cluster.test.ts Extends Redis mocks and adds tests for ZADD GT publish and read-time gating.
Comments suppressed due to low confidence (1)

src/connect.ts:353

  • When callers use isBadSubnet(req) / getSubnetStatus(req) (i.e., a request object), the code resolves the subnet from the request IP but does not pass that IP into Metrics.trackSubnet(...). This means the unique-IP gate won’t collect per-IP timestamps for this API surface, and subnets can still be blocked by a single abusive IP even when minSubnetUniqueIps > 1.
    if (track && status === ActorStatus.Good) {
      // only track if we're NOT throttling
      // forward cache to avoid additional lookup
      this.#metrics.trackSubnet(sourceStr, sourceInfo as CacheItem);

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/metrics.ts Outdated
Comment thread src/metrics.ts
Comment thread src/cluster.ts Outdated
Comment thread src/cluster.ts Outdated
…enabled, prune stale entries, skip ZADD when subnet rate disabled, track IP in getSubnetStatus(req)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Comment thread src/metrics.ts Outdated
Comment thread src/metrics.ts
Comment thread src/cluster.ts Outdated
Comment thread src/cluster.ts Outdated
… LRU, fix window-boundary data loss in cluster unique-IP publishing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/cluster.ts:333

  • The new per-window subnet_ips sorted sets are never trimmed to maxTrackedActors, unlike the other window ZSETs. In a multi-node cluster, each node can contribute different subnet members, so subnet_ips:w{n} can grow without bound for the duration of a window and increase Redis memory usage / read costs in readThresholds. Consider trimming subnet_ips with ZREMRANGEBYRANK using the same trimIndex logic as the other actor ZSETs (for each published window key).
    // Publish unique IP counts per subnet using ZADD GT so Redis retains the maximum
    // across all nodes. Publish under each window's own key so data is not lost when
    // sync() fires after a window boundary with no intervening recordHit() call.
    if (this.#clusterMinSubnetUniqueIps > 1 && this.#clusterMaxSubnetRate > 0 && this.#subnetIpSetsByWindow.size > 0) {
      for (const [w, windowSets] of this.#subnetIpSetsByWindow) {
        for (const [subnet, ipSet] of windowSets) {
          pipe.zadd(this.#subnetIpsKey(w), 'GT', ipSet.size, subnet);
        }
      }
    }

    // Trim sorted sets to maxTrackedActors (keep highest scores = top offenders).
    // Only trim types with active deltas — no unnecessary pipeline commands for idle types.
    // When the set has fewer than maxTrackedActors entries, ZREMRANGEBYRANK resolves
    // stop to a negative index that underflows past the start and is a no-op per Redis spec.
    const trimIndex = -(this.#maxTrackedActors + 1);
    if (ipDeltas.size > 0) pipe.zremrangebyrank(this.#windowKey('ip', win), 0, trimIndex);
    if (subnetDeltas.size > 0) pipe.zremrangebyrank(this.#windowKey('subnet', win), 0, trimIndex);
    if (hostDeltas.size > 0) pipe.zremrangebyrank(this.#windowKey('host', win), 0, trimIndex);

Without this trim, multiple nodes contributing different subnets via ZADD GT
could cause subnet_ips:w{n} keys to grow beyond maxTrackedActors, increasing
Redis memory usage and read costs in readThresholds. Reuses the existing
trimIndex and hoists it before the subnet_ips publish block.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comment thread README.md Outdated
Comment thread src/cluster.ts Outdated
… in README

- Cap ipSet growth at clusterMinSubnetUniqueIps: once the threshold is reached we
  have all the information needed to evaluate the gate; additional IPs waste memory
  with no behavioral benefit.
- Update README to accurately describe clusterMinSubnetUniqueIps: each node tracks
  and publishes its own observed count; Redis retains the maximum across nodes via
  ZADD GT (not the cluster-wide union). Add Redis >= 6.2 requirement note.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Comment thread src/connect.ts
Comment thread src/cluster.ts Outdated
Comment thread src/connect.ts Outdated
…hold check

IPv6 subnet keys are the IP address itself (resolveSubnetFromIp returns IPv6
unchanged), so the per-subnet unique-IP count is always 1. With the default
threshold of 128, the gate was permanently suppressing every IPv6 subnet block.

- connect.ts: skip the local unique-IP gate when the subnet key contains ':'
- cluster.ts: skip IP set accumulation in recordHit and the gate check in
  readThresholds for IPv6 subnet keys (same ':' test)
- connect.ts: remove the redundant isSubnetUniqueIpThresholdMet() call from the
  cluster subnet block check; blockedSubnets is already populated with the gate
  applied, and the extra check incorrectly suppresses IPv6 blocks after the above fix

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Comment thread src/cluster.ts
Comment thread README.md
Comment thread README.md Outdated
…es; clarify README

isSubnetUniqueIpThresholdMet() was returning false for IPv6 subnet keys and when
clusterMaxSubnetRate=0, because #subnetUniqueIpCounts is never populated in those
cases. Both are non-applicable (gate skipped) so the correct answer is true.

README: add note that minSubnetUniqueIps gate requires a request object (direct
isBadSubnet(string) calls fall back to rate-only); update clusterMinSubnetUniqueIps
description to reflect sliding-window continuity rather than "within a window".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comment thread src/metrics.ts Outdated
Comment thread src/metrics.ts Outdated
IPv6 subnet keys are the IP itself so unique-IP tracking for them has no
benefit — the gate is already skipped for IPv6 everywhere else. Add the
same !source.includes(':') guard to trackSubnet to avoid the unnecessary
Map allocation and per-insert pruning walk for IPv6 traffic.

Also reword the prune-on-insert comment to not overstate the risk as
"unbounded" — the map is bounded by maxAge pruning and subnet address space.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Comment thread src/metrics.ts
…ne cost

Previously the per-subnet ipMap grew to the full unique-IP count (up to the
subnet host count, e.g. 4096 for /20), making the prune-on-insert walk O(n).
Prune stale entries first, then only add new IPs when below minSubnetUniqueIps;
already-tracked IPs always get their timestamps refreshed. This keeps both the
map size and the prune walk bounded at minSubnetUniqueIps entries.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comment thread src/metrics.ts
Comment thread src/cluster.ts
…le-counting

Same physical client seen as both 1.2.3.4 and ::ffff:1.2.3.4 was counted as
two distinct IPs, which could prematurely satisfy clusterMinSubnetUniqueIps or
minSubnetUniqueIps. Extracted normalizeIp() from util.ts (reusing the existing
IPV4_MAPPED_REGEX) and apply it before inserting into Metrics.#subnetIpTimestamps
and ClusterSync.#subnetIpSetsByWindow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@chinrichs-godaddy
chinrichs-godaddy merged commit eb4337a into godaddy:master Jul 13, 2026
2 checks passed
@chinrichs-godaddy
chinrichs-godaddy deleted the feat/subnet-unique-ip-gate branch July 13, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants