wasi-http 0.3: instance reuse means middleware world cannot implement CDN-Loop

[pchickey]

The wasi-http 0.3 middleware world, as well as the work done to make instance reuse the default, are both features I'm strongly in favor of. The trouble comes with the intersection of those features, which I think breaks some assumptions. @cfallin first brought this issue up in #918.

CDN-Loop is a header that we expect many wasi-http embedders to require in their implementations of wasi-http middleware worlds. In short, the implementation will read the CDN-Loop header out of the incoming request, parse out the values specific to their CDN, possibly choose to block the request if those values exceed some limit, and otherwise the implementation will hand the request to a wasi-http middleware world's exported handler. Then, the implementation of the wasi-http middleware's imported handler needs to take the request and add a CDN-Loop header with all of the values from the inciting request's header, plus an added (on first hop) or modified (when appending a hop) value specific to the CDN. This is straightforward to implement in wasi-http 0.2, and indeed many implementations do so today.

However, with instance reuse, once there are multiple simultaneous requests passed to the exported handler, it is no longer possible for an implementation to associate a call of middleware's imported handler with the inciting request of middleware's exported handler. This ambiguity makes impossible to implement CDN-Loop correctly, and if implemented incorrectly CDN-Loop could end up either incorrectly blocking benign requests, or failing to block requests that should be blocked, neither of which a CDN can tolerate.

Inside any correct middleware component, with instance reuse or not, there is an association between an incoming request and the outgoing request, in order to turn the outgoing request's response to the incoming request's response. Instance reuse, and the current shape of the middleware wit world, makes it impossible for the implementation of the middleware world to maintain that association. Host magic to solve this doesn't exist, and even if it did, we want this exact sort of thing to be implementable with virtualization anyway.

CDN-Loop is just one example in the broader category of problems caused by loosing association between the inbound request and outbound request in a middleware or proxy. This problem presents in other ways specific to the embeddings that @cfallin and I maintain for F5, as well, but CDN-Loop is a representative example that exists across many vendors, so I figured its the best way to discuss it in the standard.

[cfallin]

Thanks @pchickey for writing this up -- Pat and I have discussed this and a few potential solutions, starting from the discussion in #918. A few thoughts:

• I think that this issue is separable from the logging issue in wasi-http 0.3: request and response should have a tracing identifier #918. While the latter -- incoming-request attribution for logged messages -- is a "nice-to-have correlation", this issue is a fundamental capability-model issue.
• The cleanest rendering IMHO is to make the plumbing explicit in the dataflow. I had proposed something pretty ugly in wasi-http 0.3: request and response should have a tracing identifier #918 with a parent-request: option<borrow<request>> on a request constructor. Instead of that, I think maybe what we could have as a general shape is a factory pattern: the outgoing request method is no longer a freestanding function import, but a method on a resource, and that resource (outgoing-request-factory) is a capability ("the ability to make outgoing requests") that is plumbed into the entrypoint somehow.
  • Within the guest, the incoming request associated with some dynamic scope could be held in (guest-side) thread-local storage or other implicit interpreter/runtime context. There is no conflict here wrt expected APIs in common guest languages IMHO: syscall or library interfaces of various sorts already force various kinds of impedance matching, and there should be an expectation in a capabilities-based system that one needs to wire through capabilities to provide the needed services.
  • The main downside of the option-of-parent-request, that one would forget to associate the parent, is no longer an issue if we take the pure capabilities approach: one needs, by the static type system, to have the factory that the host can associate with some request context.
  • This is also nicely virtualizable, because it's purely in the WIT type system as ordinary interfaces, with no magic.
  • The main downside is that it results in a kind of non-compositional entrypoint: the naive dependency-injection design here is to have handle(outgoing-request-factory: ..., logger-factory: ..., other-side-service-factory: ...)
    • An antidote to that is to have a root-request-context resource, and use interface composition ("mixins") to request the specific factories. So the specified entry point for a wasi-http handler is handle(incoming-request, root-context), and then one does root-context.outgoing-request-factory() to get the capability to make outgoing requests. I strongly favor this design so far.
• An alternative is to wire through the association on the host side, but (i) ambient dynamic state is IMHO an unfortunate design choice in a system that otherwise strives to be a capabilities system, and (ii) as @alexcrichton notes here there are complications in trying to build that plumbing.

[alexcrichton]

I talked about this with @pchickey today a bit and wanted to write down some conclusions we reached.

The main thing was that I talked to @lukewagner about my comment here and we've concluded that it's probably what we should do to solve this. Specifically Wasmtime is already maintaining an async call stack of sorts and coupled with the ability to learn the "id" of a starting task I think it would solve this. Wasmtime can, for example, provide the ability to create an async task, not yet started, and then learn some sort of identifier from it. From this state the task can be started. Subsequently when wasm calls the host it should be possible for Wasmtime to deduce what the "root" async task is, and then the host can correlate these two events to provide a signal of what to do. Effectively, using async tasks, the host can correlate outgoing requests to whatever incoming request originated the task.

In talking with @pchickey we concluded that this would likely solve the issue at hand. There's some niche cases where it doesn't always work, but this is something we concluded is like inevitable regardless of the design. For well-behaved guests that don't intentionally go out of their way it should be able to provide this correlation.

With this the hope is then to be able to use this correlation to avoid needing to change anything related to the component model or WASIp3 WITs or such. I'll talk a bit more about this in the WASI meeting tomorrow and with some folks in the morning, but I wanted to be sure to at least leave a high-level summary comment before doing so to avoid completely blind-siding folks at least.

[alexcrichton]

I've attempted to further summarize, and document, the outcomes of all discussions in #920. I've currently scheduled that to close this upon merge, so please take a look!

[cfallin]

Thanks @alexcrichton. I don't have an invite to the WASI subgroup and didn't realize this would be discussed there, but I think it would be nice to see explicit commentary on the capabilities-based design noted above, especially in a longer-term context. It sounds like there is not appetite for creating explicit plumbing (in a canonical capabilities-system way) whereby the handler is given a capability to make outbound requests in its context? If not, why not? Implicit, ambient state, tied to tasks, seems to be strictly more complex to me. I understand pragmatism around getting near-term launches unblocked but I also want to make sure my ideas and concerns are not ignored in a longer-term design context.

[pchickey]

The argument against plumbing through the association between inbound and outbound request via capabilities (i.e. plumbing through a resource from one to the other) is that while that can solve this particular problem where both end of the association exist in the same packge, it binds the invariant permanently to that package's interface (making it unsuited for reuse in other packages), and also cannot express invariants created by another package that uses another interface. Its just not a composition mode that was thought about when wit was designed, so there aren't mechanisms to do it currently - it would perhaps take something like an effect system in order to express these invariants across package boundaries, and may be tricky to implement in a way that doesn't break separate compilation of Canonical ABI imports/exports.

I agree that having to solve this issue by plumbing through a capability implicitly on the task abstraction is not ideal, but its the best mechanism we have a reasonable path to today. Long term its probably worth considering how the design of the component model's type system can express this differently.

[cfallin]

Thanks. I suspect there is still a way that preserves separate compilation and composition over multiple packages. Consider:

• The handle entry (or commandline main, or any world's entry point) takes a root-capability; that is a core WASI concept.
• Any package P that today defines ambient-action(...) -> ... as a free function in an interface (e.g.: outbound HTTP requests) does the following:
  • puts that method instead on an interface action-cap { action(...) -> ... }
  • defines a ambient-action-capability(root: borrow<root-capability>) -> action-cap free function

That's a one-for-one trade in free toplevel function (ambient-action -> ambient-action-capability), so we still get the linker's ability to late-bind and preserve separate compilation. And the toplevel for any world does not need to know about the various system capabilities -- we get the N-to-M binding without having to make all aware of all. Basically, it inherits all of the characteristics of the existing situation, because we replace a toplevel function with a toplevel function -- the only difference is that we thread through root-capability. What do you think?

[cfallin]

Concretely, one possible shape (but not the only one), rendered in wit:

package wasi:capability;

resource root-capability {}

# ---

package wasi:http;

use wasi:capabillity/root-capability;

interface handler {
  handle: async func(root: borrow<root-capability>, request: request) -> result<response, error-code>;
}

resource client {
  get: static func(root: borrow<root-capabililty>) -> client; # maybe result<>?
  send: async func(root: borrow<root-capability>, request: request) -> result<response, error-code>;
}

Note that:

• Some other API that also needs to be "scoped" to the particular handle invocation could implement its own equivalent of the client resource, with no involvement from wasi:http, either at the type level or with extra params -- everything is hung off of the root-capability;
• Separate compilation is preserved, because of the above;
• The handle function on handler (incoming entry point) and send signature on client are symmetric (pass root cap in, pass root cap out).

One could summarize this approach as "thread a context handle through, and make actions methods on instances created from context, rather than freestanding functions".

[pchickey]

I stand corrected, I hadn't thought of @cfallin's way to shape this so that a capability works with separate compilation.

I guess the biggest thing to consider with regard to @cfallin's proposal is where in the wasi interfaces root: borrow<root-capability> isn't appropriate as the first argument. get-environment() -> list<(string, string)> probably not because its supposed to be the same for the life of the instance, but what about get-stdout, or other logging facilities (as we discovered with #918)? I'm not sure where to start or stop applying that approach, which doesn't necessarily count against the approach but against my having thought of a rule for where to draw the line...

[alexcrichton]

Ah apologies @cfallin this all happened pretty quickly, but regardless definitely happy to follow-up here.

Your example WIT is what I would envision as well for passing around capabilities, more-or-less an explicit argument in various locations. The problems I personally see with this sort of approach are:

• Libraries in languages generally aren't architected for this. For example we wanted to ideally provide stdin/stdout/stderr as capabilities to the main function of a component, but that's just not how any standard library is really organized. For the context of HTTP this would mean that reqwest, or any other library that wants to support WASI, would have to have a WASI-specific API not present on any other platform to thread around this capability and/or object. Overall I feel that we've historically found that it's basically not worth the complexity budget to try to get ecosystems to integrate in WASI-specific ways and it's much easier to integrate as-is into ecosystems.
• Assuming we did in fact do all the integration and plumbing everywhere, this can quickly become a related problem when the next capability needs to be transferred. For example wasi-otel may want a context of some form transferred around in addition to the wasi-http capability. This might instead nudge the host to have a sort of "catch all" capability which represents everything that needs to get passed around. This, to me, feels like it's going back to the world of "every interface takes a wasi:io object somewhere" where many interface would take this catch-all capability (to avoid having unique capabilities that everyone's passing around).
• In the limit, to me this sounds very similar to a free-standing get-the-capability: func() -> own<thing> function. That'd be one-per-capability and per-interface, in theory, and would be easy to integrate into existing libraries. This sort of design though ends up being basically the same as using the call-stack and that's probably how the host would implement it.

Overall I feel that explicitly threading capabilities, while conceptually nice, generally doesn't work well for foundational primitives. For example standard I/O, making HTTP requests which many preexisting libraries already do, and tracing spans are examples of pretty foundational things where it'd be "cleanest" to thread objects around but AFAIK most languages don't do that. Using the async call stack is indeed just an approximation of what capabilities would allow (some examples in #920), but I feel at least personally that it's the best tradeoff of practicality and accuracy.

[cfallin]

Libraries in languages generally aren't architected for this

I touched on this above already but I do not see this as changing user-visible APIs at all. Rather I would suspect that the guest would use some sort of dynamic context inside guest space to hold the root capability. That could either be guest-side TLS or state attached to an interpreter/language context. If you like, imagine a get_root_capability / set_root_capability facility somewhere inside wasi-libc or wstd or whatever that my-special-api-library uses to get the root cap for the "current execution".

That may seem like a little bit of sleight-of-hand vs. holding it "on the outside" (it's TLS either way!) but there are two big advantages: (i) it reduces the complexity of the host-side VM and required builtins, and I believe CM already has a problem with an unwieldy implementation complexity; (ii) it's virtualizable. Putting it in the WIT types and making context a real resource handle lets one reason about it, use it, pass it around like anything else, without special ambient state that requires its own primitives to access and update.

Assuming we did in fact do all the integration and plumbing everywhere, this can quickly become a related problem when the next capability needs to be transferred. For example wasi-otel may want a context of some form transferred around in addition to the wasi-http capability. This might instead nudge the host to have a sort of "catch all" capability which represents everything that needs to get passed around.

As noted above, wasi-http doesn't need to know about wasi-otel, and vice-versa. The "catch all" capability that you hypothesize is in fact root-capability, no? There's no need to transfer around a bunch of capabilities.

In the limit, to me this sounds very similar to a free-standing get-the-capability: func() -> own function. That'd be one-per-capability and per-interface, in theory, and would be easy to integrate into existing libraries. This sort of design though ends up being basically the same as using the call-stack and that's probably how the host would implement it.

Yes, exactly! :-) The key is that it's on the guest side, using existing facilities for guest-side state, and at the wit interface level (but not the user-visible level), things are explicit. That keeps the host simple(r) and keeps things virtualizable.

[alexcrichton]

Ah sorry I missed your note of that, but I believe this sort of scheme still suffers from the inability to practically virtualize as well as a murky definition of root-capability on the host.

For virtualization this sounds a lot like wasi:io where we ended up concluding for WASIp2 that virtualizing more-or-less wasn't possible because if you virtualize one thing you have to virtualize the entire world due to the pervasive dependency on wasi:io. In this scheme the definintion of root-capability suffers the same problem -- if you virtualize wasi:http/handler then you also have to virtualize anything else that works with root-capability or similar.

For a murky definition, I'm not sure what we would do on the host for something like root-capability. One option would be like HashMap<TypeId, Box<Any>> to represent pretty much anything, and another option would be a sort of "trusted integer" which the host keys on (similar to the exported task ID if using the async call graph). If the latter, this is pretty close to the six-to-one-half-dozen or the other here.

---

Personally I'd say that adding this to WIT does not keep the hosts simpler. Hosts already have to maintain an async call graph and already have the knowledge of root tasks and such. Exposing this in Wasmtime is expected to be almost trivial and mostly just some bikeshedding. Alternatively all WIT interfaces need to be rethought and have borrow<root-capability> injected anywhere the host might want to handle things like observability. This doesn't just affect WASI WITs but also all downstream wits for embedders as well. Arguably something like wasi:sockets may want telemetry depending on the embedding as well, but on the other hand it seems weird to put it there.

Overall I see this need to attribute import-calls-to-export-calls as something that's so pervasive that solving it in WIT is the wrong layer of abstraction. It's sort of like how we could build an async system in WASIp2, but it was always known to be objectively the wrong place to do so.

[lukewagner]

From a capability-based security perspective: at the wasm level (as opposed to language level) our unit of isolation here is the component instance. That means that we have to assume that everything inside the unit is "colluding" (via shared instance-wide memory and table state) which means that trying to enforce something finer-grained at the wasm level isn't buying us anything. So for example, if, as a host (or parent component), I give a component instance ci 3 root-capabilitys, I have to assume that any import call made by ci can be passed any of those 3 root-capabilitys and that I have no way to know whether the given root-capability was "right" or "wrong" because the caller is a soup of core wasm; I have to assume it was right. If I need a finer degree of isolation, I need to create finer units of isolation (= component instances).

Importantly, from this capability perspective, I believe the async-call-stack-based approach is no different: every async call into ci creates a fresh "task" which you can indeed view as a capability (it can't be forged, and its meaning is set and interpreted by the host or, one day, parent component), and any import call that ci makes can only be given one of these tasks. And as above, I can't know whether "the calling task" was "right" or "wrong". The main difference is just that this capability is threaded implicitly through calls and thread creation by the runtime using machinery it already needed for several other purposes. There's also a temporary limitation which is that the guest can't easily control "the current task", but that could be fixed by an (I think) simple addition like the one sketched in CM/#659.

Given this more-theoretical capability equivalence, deciding between the approaches I think comes down to a more pragmatic question of ergonomics and ease of use which, for the reasons Alex mentioned, I think favors the async call stack approach. Also, all else being equal: the simpler and more readable our WIT interfaces are, the better, and I think plumbing root-capabilitys everywhere is a non-negligible regression; it's an abstract concept that you need a lot of context to understand, and if the goal is that it's plumbed by out-of-sight machinery, now WIT interfaces can't simply be defined and called by regular developers. If all this paid for itself by a clearly stronger security posture, that'd be worth discussing, but, because of the above, I don't believe that it does.

[cfallin]

if you virtualize wasi:http/handler then you also have to virtualize anything else that works with root-capability or similar.

For a murky definition, I'm not sure what we would do on the host for something like root-capability. One option would be like HashMap<TypeId, Box> to represent pretty much anything, and another option would be a sort of "trusted integer" which the host keys on (similar to the exported task ID if using the async call graph). If the latter, this is pretty close to the six-to-one-half-dozen or the other here.

Yes, I had expected (sorry for not sketching details) that the root-capability would provide some sort of index so that a guest-space-implemented virtualized component for just one API could key its resources off of that.

I agree that this is pretty close to "trusted CM builtin to provide the current task ID" if accessible from Wasm; it sounds like your proposal above was to make this accessible only on the host side? That was one of my roots of concern about virtualizability. If you see root-capability as a reified, unforgeable task ID, I guess they're pretty similar.

---

Separately, there is a category of use that "actual task ID" does not cover that "task ID as unforgeable handle" does: when the guest wants to do smart (clever? too clever?) things managing its own contexts. For example do we have a guarantee that in all cases, for all guest task-scheduler implementations, we will have a one-to-one mapping between CM async tasks and the identity we wish to use for outbound calls? (We certainly have at least one CM async task per toplevel handle() entry, so maybe that's the case, but it's worth noting that the explicit plumbing is strictly more expressive here.)