rules: rate limiting

[mrl5]

closes #18

overview

• implements sliding window algorithm (described in https://blog.cloudflare.com/counting-things-a-lot-of-different-things/)
• allocates two fixed size buffers (std::collections::HashMap). Size is configurable by capacity param
• buffers are controlled by single thread. tokio::sync::mpsc::channel is used for communication with request handlers aka "hot path"

pingoo.yml interface

rules:
  rate_limit_api_routes:
    expression: http_request.path.starts_with("/api/")
    actions:
+     - action: limit
+   limit:
+     max: 10
+     period: 60      # in seconds
+     capacity: 1024  # number of IPs that can be tracked

benchmark

Implementation was tested with pong.rs service compiled with --release profile. CPU used was AMD Ryzen 7 3700X 8-Core Processor (16 threads)

TL;DR

Rate limiter introduces ~6.83 us of latency (133.55us - 126.72us)

bench 1 -- allow all requests but count

wrk -H "User-Agent: curl/8.4.0" -H "Accept: */*" -c10 -t4 -d30s --latency http://127.0.0.1:8080/foo

Running 30s test @ http://127.0.0.1:8080/foo
  4 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   133.55us   86.13us   4.30ms   93.02%
    Req/Sec    15.04k   812.64    18.18k    68.83%
  Latency Distribution
     50%  118.00us
     75%  143.00us
     90%  184.00us
     99%  292.00us
  1800124 requests in 30.10s, 178.54MB read
Requests/sec:  59805.02
Transfer/sec:      5.93MB

bench 2 -- rate limit all requests

wrk -H "User-Agent: curl/8.4.0" -H "Accept: */*" -c10 -t4 -d30s --latency http://127.0.0.1:8080/foo

Running 30s test @ http://127.0.0.1:8080/foo
  4 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    65.21us  121.49us   7.93ms   99.40%
    Req/Sec    32.29k     3.19k   72.72k    78.60%
  Latency Distribution
     50%   49.00us
     75%   68.00us
     90%  110.00us
     99%  149.00us
  3858445 requests in 30.10s, 636.59MB read
  Non-2xx or 3xx responses: 3858445
Requests/sec: 128189.65
Transfer/sec:     21.15MB

reference 1 -- upstream pingoo (commit 2716047) + pong

wrk -H "User-Agent: curl/8.4.0" -H "Accept: */*" -c10 -t4 -d30s --latency http://127.0.0.1:8080/foo

Running 30s test @ http://127.0.0.1:8080/foo
  4 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   129.42us   93.51us   5.50ms   93.55%
    Req/Sec    15.59k     0.93k   23.08k    70.22%
  Latency Distribution
     50%  114.00us
     75%  138.00us
     90%  177.00us
     99%  284.00us
  1864309 requests in 30.10s, 184.91MB read
Requests/sec:  61938.41
Transfer/sec:      6.14MB

reference 2 -- this PR + pong, rate limiting disabled

wrk -H "User-Agent: curl/8.4.0" -H "Accept: */*" -c10 -t4 -d30s --latency http://127.0.0.1:8080/foo

Running 30s test @ http://127.0.0.1:8080/foo
  4 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   126.72us   82.22us   4.36ms   93.00%
    Req/Sec    15.80k     0.90k   23.36k    68.97%
  Latency Distribution
     50%  112.00us
     75%  136.00us
     90%  174.00us
     99%  272.00us
  1890160 requests in 30.10s, 187.47MB read
Requests/sec:  62797.52
Transfer/sec:      6.23MB

reference 3 -- just pong

wrk -H "User-Agent: curl/8.4.0" -H "Accept: */*" -c10 -t4 -d30s --latency http://127.0.0.1:3000/foo

Running 30s test @ http://127.0.0.1:3000/foo
  4 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    31.24us   12.10us   1.78ms   87.73%
    Req/Sec    59.76k     3.49k   69.01k    73.67%
  Latency Distribution
     50%   30.00us
     75%   36.00us
     90%   42.00us
     99%   55.00us
  7161754 requests in 30.10s, 601.04MB read
Requests/sec: 237936.46
Transfer/sec:     19.97MB

bench disclaimer

for the record bench 1 -- allow all requests but count required this modification, so it was kind of custom build:

index 1e70882..9296708 100644
--- a/pingoo/rate_limiter.rs
+++ b/pingoo/rate_limiter.rs
@@ -166,7 +166,7 @@ impl RateLimiter {
         };

         let approx = get_approx(prev_sum, curr_bucket.starts_at.elapsed(), self.sampling_period);
-        Ok(u64::from(self.limit) >= approx + u64::from(curr_sum))
+        Ok(u64::from(self.limit) >= approx + u64::from(curr_sum) || true)
     }