Skip to content
c4miloPublic

About

HTTP/1.1, HTTP/2 and HTTP/3 in Zig, client and server, written from the RFCs: no heap, no I/O of its own, deterministic simulation, TLA+ and Lean models. QUIC included.

Topics

Resources

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

colibri

main license zig

An HTTP/1.1, HTTP/2 and HTTP/3 library for Zig, client and server, with QUIC underneath HTTP/3. It is written from the RFCs, and every check it makes on a peer's input cites the section that requires it.

colibri owns no socket, no allocator and no clock. Your program reads the octets, hands them to colibri, and gets back events. colibri writes the octets it owes into buffers your program owns, and your program sends them. Every function that needs the current time takes it as a parameter. Because of this, one seed of the simulator replays a whole connection byte for byte.

The name is Spanish for hummingbird.

At a glance

Protocols h11 (RFC 9112), h2 (RFC 9113) with HPACK (RFC 7541), h3 (RFC 9114) with QPACK (RFC 9204), and QUIC versions 1 and 2 (RFC 9000, 9001, 9002, 9369)
Roles Client and server, for every protocol
Memory No allocator anywhere in src/, tests included. Your program owns every struct and buffer, and each size is a comptime constant.
I/O None. colibri parses octets you already read and writes into buffers you own. A call that would block returns what it needs instead.
Time A value you pass. No source file reads a clock.
TLS and packet protection Vtables: tls_provider.Provider for h11 and h2, and tls_provider.QuicProvider and crypto.Suite for QUIC. The tls module fills all three from chapulin, the library's TLS stack, and a program may fill them itself.
Language Zig 0.16.0
License Apache-2.0

Status

colibri 0.8.0 is the latest release. 0.1.0 was the first, and carried the TLS that design §8 step 16 put in the library. 0.2.0 has each TLS session draw from a source its caller passes. 0.3.0 ends a connection inside colibri on a TLS or QUIC error, and adds h2's trailers. 0.4.0 adds the server and client modules, whose Channel chooses h3 over QUIC or TCP for each server, and qlog. 0.5.0 adds the server over QUIC and Alt-Svc advertising, and content codings: the server codes a response in gzip or deflate, and the client decodes one. The client also retires a QUIC connection near its idle timeout. 0.6.0 runs AES-256-GCM first where the CPU has the AES instructions, lets a TLS client choose the suites it offers, and sends a failed record's alert before it closes. 0.7.0 speaks QUIC version 2, and its server switches a client that lists it (RFC 9368). The server ends a connection whose peer is too slow or silent, and an h2 peer that resets streams too fast. One build runs the AES instructions where the program's probe of its CPU finds them and ChaCha20 elsewhere, and both ciphers run faster. 0.8.0 adds a router to the server: literal paths, parameters, a trailing wildcard, and HEAD answered by a route's GET. The router refuses a table that hides a route. The server holds its TCP connections in its endpoint and runs its deadlines and rate limits over h3 too. Each module exports its names from its root alone, and a channel builds its TLS from one value. The client sends a client.Request and reads a client.Response in place of client.HttpExchange, and client.ChannelOf sizes a channel. It sends an idempotent request again once after an unanswered close, and holds a long h11 head over TLS. After a KeyUpdate that asks for a reply, the client and the server read on, and send the reply before their next application data record (RFC 9846 §4.7.3). The client decodes zstd and br, a TLS client can take no tickets, and colibri carries chapulin e6e3c1e7, whose clock is a function.

Protocol Built Checked against
h11 Client and server, pipelining, chunked, and gzip and deflate bodies decoded Go's net/http, h2o and curl, in cleartext and over TLS; the HTTP Garden against 35 other servers
h2 Client and server, HPACK with the dynamic table h2spec 2.6.0: 144 of 146 in cleartext with h2 alone and over TLS, the other 2 test an RFC 7540 rule RFC 9113 dropped; curl, nghttp, Go, nghttpd and h2o
h3 and QUIC Client and server, QPACK with the dynamic table, Retry, resumption, key update, loss recovery and congestion control The QUIC Interop Runner against quic-go, ngtcp2, neqo and quinn; h3spec 0.1.13: 49 examples, 0 failures; h2load --h3: 1,000 of 1,000 requests; aioquic in both directions; QPACK against ls-qpack

Still to come:

  • published benchmarks from Linux (step 13).

docs/design.md §8 records each check: what ran, on what machine, and what it printed.

Using colibri

Add colibri to your build.zig.zon:

zig fetch --save git+https://github.com/c4milo/colibri#v0.8.0

Then import the modules you use. Each of the fifteen library modules is exported by name: core, wire, http, tls_provider, tls, crypto, qlog, hpack, qpack, quic, h2, h3, h11, server and client.

const colibri = b.dependency("colibri", .{ .target = target, .release = true });
exe.root_module.addImport("server", colibri.module("server"));
exe.root_module.addImport("client", colibri.module("client"));
exe.root_module.addImport("tls", colibri.module("tls"));

.release = true builds ReleaseSafe. colibri offers Debug and ReleaseSafe only, because its assertions stay on in production.

Most programs use two modules. client sends requests and server answers them, behind one set of calls for h11, h2 and h3. Each connection runs its own TLS handshake, and ALPN picks the version. For each exchange a client places a request, and a response that holds the memory the answer goes into. These lines are from examples/client_exchange.zig:

const get: client.Request = .{ .method = "GET", .path = "/greeting" };
var get_response: client.Response align(@alignOf(client.Response)) = .{
    .wanted = &wanted,
    .values = &wanted_values,
    .body = &get_body,
};

It hands both to a pool. The pool asks for a QUIC or a TCP transport, and sends the request over the first connection that completes its handshake:

const known: client.Values = .{ .addresses = &server_addresses, .port = server_port };
// `init` builds the TLS configuration of each transport inside the pool, which stays in
// place from here on. It keeps the pool's TCP connections and its exchanges in `pool_memory`,
// which `.{}`, the default counts, sizes. The receive pool is what makes the pool offer QUIC.
try pool.init(&pool_memory, &client_config, known, .{}, receive_pool.storage());
// `request` only takes a request and its response. The pool asks for a transport, and
// sends the request over the first connection that completes its handshake, so a program asks
// at once and never waits.
_ = try pool.request(&get, &get_response);
_ = try pool.request(&post, &post_response);

The program moves datagrams and octets between its sockets and the pool's send_datagram, send_stream and receive, and every exchange ends in one finished event, which names the request and the response:

.finished => |ended| {
    // The event names the request and the response, whose memory is the program's again.
    std.debug.print("client: {s}: {s}, {d}\n", .{
        ended.request.path, @tagName(ended.response.outcome), ended.response.status,
    });
    finished += 1;
    // Every exchange has ended, so the client shuts the pool down.
    if (finished == requests_count) pool.shutdown();
},

A server holds its TCP and QUIC connections in one endpoint, and calls accept for each TCP socket it accepts. It gets one event from each receive, and answers a request by its id, in h11, h2 and h3 alike. It finds the route of a method and a path through a table that it declares at compile time, with server.router. These lines are from examples/server_exchange.zig:

fn answer(id: server.Id, content: []const u8) !void {
    var digits: [8]u8 = undefined;
    const length = std.fmt.bufPrint(&digits, "{d}", .{content.len}) catch unreachable;
    try endpoint.respond(id, .{
        .status = 200,
        .fields = &.{
            .{ .name = "content-type", .value = content_type },
            .{ .name = "content-length", .value = length },
        },
        .end = false,
    });
    // `write_body` returns the octets it took. It takes fewer than it was given when the room or
    // the peer's window runs out, and the request's `writable` event then says when to write the
    // rest.
    const taken = try endpoint.write_body(id, .{ .octets = content, .end = true });
    assert(taken == content.len);
}

docs/usage.md walks through both modules.

The protocol modules are there for a program that wants one version by itself. An h11 client writes each request into a buffer it owns, then sends it. These lines are from examples/h11_exchange.zig, where link stands in for the program's socket:

const get_len = try client.write_request(&output, "GET", "/greeting", &.{
    .{ .name = "Host", .value = host },
});
try link.send(.client, output[0..get_len]);

It then hands colibri the octets that arrived, and gets at most one event per call:

const input = try link.receive(.client);
const step = try client.receive(input, &.{});
if (step.event) |event| switch (event) {
    .response => |response| {
        try read_status(response.line.status.code);
        if (ends_with_head(response.body)) ended += 1;
    },
    .data => |data| try read_body(data),
    .end => ended += 1,
    else => {},
};
link.consume(.client, step.consumed);

A call that consumes nothing and returns no event means colibri needs more octets. A response with no body ends with its head, so no .end follows it. An event's slices point into the input, so the example uses them before it lets the link drop the octets step.consumed counts. The response's field lines are in client.section until the next call. h2 and h3 follow the same shape: octets in, at most one event out, and the frames colibri owes written into your buffer. docs/usage.md walks through each protocol, TLS, and the buffers each one needs.

examples/ holds whole programs, each a client and a server run over Rotor's loop: a server that serves TCP and QUIC through one endpoint, a client whose pool tries QUIC and then TCP, and h11 and h2 by themselves. zig build examples runs them, and CI does too.

How it is checked

  • RFC citations. A check that exists because an RFC requires it carries the RFC and the section on the line that makes it. A lint refuses a validation without one.
  • Deterministic simulation. A seeded network delays, drops, reorders and duplicates datagrams, and marks them with ECN. Checks drive h11, h2, h3, QUIC, loss recovery and QPACK through it, and each seed replays byte for byte in Debug and in ReleaseSafe.
  • Models. Ten TLA+ specifications in spec/tla/ cover flow control, QPACK's tables, key installation, connection IDs, closing, probe timeouts and more. Each model also has configurations that remove one rule, and TLC must find those violated. The h3 simulator writes its runs as traces that TLC checks against the h3 model.
  • Proofs. spec/lean/ holds Lean 4 proofs of QUIC's variable-length integer, packet number and ACK ranges, HPACK's and QPACK's prefixed integer and Huffman code, and QPACK's Required Insert Count encoding. The Zig tests read vectors the proved definitions produce.
  • Golden corpus. src/golden/ holds exact bytes with a manifest that names each case's length, checksum and expected verdict, including the request smuggling shapes of RFC 9112.
  • Mutation testing. Every check is broken on purpose, and a test must fail. Each result is recorded as CAUGHT or NOT CAUGHT in the commit or in docs/design.md.
  • Fuzzing. Every reader of a peer's octets has a fuzz property, run over its corpus and every input of up to two octets. Input checks in the simulator edit what colibri's own writers produce, to reach longer inputs.
  • Lints. zig build lint refuses a heap allocation, a system call, a clock or random read outside the test endpoints, an unbounded loop, a function above cognitive complexity 15, a source file above 500 lines, and a slice indexed by a value a peer chose.

CI runs tools/ci.sh on every push to main: the lints, every module's tests in both build modes, the examples, the simulator in both build modes, the TLA+ models and the h3 traces, the Lean proofs, h2spec and the interop scripts in cleartext and over TLS, h3spec, the TLS handshakes against Go, and the QUIC checks against colibri and aioquic. Every push also runs the module tests on arm64, on Linux and on macOS. The HTTP Garden and the QUIC Interop Runner, against quic-go, ngtcp2, neqo and quinn, run every Monday.

Platforms

CI runs every check on Ubuntu 24.04 on x86-64, the module tests on Ubuntu 24.04 and macOS 26 on arm64, and the QUIC Interop Runner on Ubuntu 26.04 on x86-64, because it needs tshark 4.5.0 or newer. macOS on arm64 is also the development host, where every check runs, the TLS and QUIC ones included. colibri makes no system call, so it builds wherever Zig 0.16.0 does, but only these platforms are checked. Published performance numbers come from Linux alone.

Documentation

Document What it covers
docs/usage.md Adding colibri, driving each protocol, TLS, and the buffers you own
examples/ Whole programs for the server and client modules, and for h11 and h2 by themselves, which CI runs
docs/design.md The module graph, the wire formats, and the numbered build plan with each step's record
docs/decisions.md Every design decision, with the alternatives it beat
docs/invariants.md The numbered invariants, each one a runtime assertion
docs/chapulin.md What colibri needs from chapulin, its TLS stack
docs/rfcs/ The RFCs colibri is written from, unmodified, with their checksums

Limits

colibri does not build these, and docs/decisions.md says why for each:

  • server push, in h2 and h3;
  • priority scheduling, though it parses the priority fields the RFCs still require;
  • extended CONNECT;
  • 0-RTT;
  • connection migration;
  • HTTP datagrams and multipath QUIC;
  • middleware: events are polled, so a program's own loop is where a check before a request runs.

Contributing and security

CONTRIBUTING.md states the rules a change must follow and the checks it must pass. Report a vulnerability as SECURITY.md describes, not in the public tracker.

License

Apache-2.0. See LICENSE.

About

HTTP/1.1, HTTP/2 and HTTP/3 in Zig, client and server, written from the RFCs: no heap, no I/O of its own, deterministic simulation, TLA+ and Lean models. QUIC included.

Topics

Resources

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages