SAF pre-requisite: Consistent multi-level support

[handrews]

I have some thoughts on next steps for SAFs (#5310), which (as previously presented) are API features that are standardized (by someone else) and require an exception to the essential OAS description rules.

One point I mentioned in the slides is that SAFs operate at multiple scales, or levels. To fit this in our "standardized, yet requiring an exception" framework, it would help if our "essential rules" were more consistent and comprehensive.

Not adding anything truly new

My favorite response to 3.2 was this one from Dave Shanley of libopenapi that started with:

To be honest, the updates were quite simple. They used existing models and patterns, there were no new complex relationships, no new paradigms that would require new algorithms. I could use all of my existing scaffolding without adding any new engine parts.

The changes proposed here might be a little more burdensome as they touch more challenging areas of parsing and processing. But fundamentally, they are taking things we already do and doing them more consistently.

That's the frame I want to emphasize here. From a user perspective, these are big changes. From an implementor perspective, they should actually make things easier in the long run.

Filling out the field levels matrix

We already have fields that work at multiple levels, with two different but well-defined ways of overriding. But they are... kinda randomly distributed. Which is weird. Why not make them all behave the same?

• It's easier to think about (I've had to look up which level security skips multiple times in the past month)
• It's allows more code re-use in implementations (everything that "inherits" does it the same)
• Every single one of these has been requested. Repeatedly.
• It means that SAF exceptions for any/multiple of these fields will be working from the same base behavior

Here are the relevant fields and levels:

field:	servers	security	parameters	headers	responses
global	Y	Y	+	+	+?
path	Y	+	Y	+	+?
op	Y	Y	Y	+	Y
response	n/a	n/a	n/a	Y	n/a
override:	whole	whole	single	single	single

The override row shows the interactions — whether the entire field overrides higher levels, or whether the override behavior is at the single item level. Both behaviors already exist, so again this is nothing new.

What goes where

• "global" means global to the API, and is represented by the OpenAPI Object of the entry document only.
  • We could consider a document-local level, but that is a significant change in mindset so I recommend against it. I can elaborate if necessary.
• "path" means the Path Item Object
• "op" is a bit odd because it could mean truly operation-scope, or request-scope, or response-scope:
  • responses is operation-scope, on the Operation Object
  • servers, security, and parameters are request-scope, also on the Operation Object (the Request Body Object is just a required flag and a map of Media Type Objects, and there's only at most one of it)
  • headers is response-scope, and would be added to the Responses Object (plural), which represents the complete set of response behaviors for the operation; adding it here ensures that it associates properly when responses is added at other levels

Why a ? on responses?

The responses field is a bit more complex as it would include the headers field, both directly and in individual Response Objects. While I think multi-level resposnes is both one of the most-requested features and potentially important to SAFs (recall the special 202 Accepted response handling for Prefer: respond-async), we might want to figure out the others and then look at how we feel about responses. I suspect it will be much easier to decide about it then.

This is an area where we might be going beyond "existing models and patterns," so we should make sure it's worth the effort.

What about response headers that reply to some but not all responses?

I think the appropriate approach is to consider all response header descriptions as "if relevant." We should think through how the required field on the Header Object functions in such an environment. But if a header is relevant to most responses, we should be able to set it on the Responses Object even if, for example, it won't appear on a 503 Service Unavailable response from an intermediary.

Order of steps

This would introduce a fairly uniform set of processing steps for all major aspects of an operation (ignoring responses for now, as explained above):

• Locate the entry document's OpenAPI Object, and use the fields present there as the starting point for all operations
• For each Path Item Object at its point of use (after resolving $refs), use the fields to override or merge those inherited from the OpenAPI Object
• For each Operation Object under a given Path Item Object, use the fields to override or merge those inherited: This gives you your request description
• For the Operation Object's Responses Object, merge headers
• For each Response Object under the Responses Object, merge headers: This gives you your response description.

Note that we already have to do the first three, but with different sets of fields at each level. The last two are new, but are important given the reliance of many SAFs on response headers, and the difficulty of locating the correct individual response(s) to check for such headers.

So what if we add responses?

The main wrinkle there is managing the interaction of multiple levels of headers and responses. Basically, does a lower-level response completely replace the higher-level response for the same status, or are its headers fields merged?

I think the answer is "completely replace", but it needs thought.

[handrews]

The point of filling out this matrix is that we'll have SAFs that work at the different levels, using these various mechanisms. Having the mechanisms appear and interact consistently will make the interaction of SAFs that use them more clear.

[handrews]

An example of how this makes SAFs a bit more regular:

API keys are a SAF, even though they're kind-of a pseudo-standard. There's no one API key specification, but everyone knows what you mean: some sort of credential token that is passed as a request header, query parameter, or cookie (but not in the path).

If both parameters and security are available at all three levels (global, path, and operation), then we can define a clear interaction for the API Key SAF and regular query/header/cookie parameters at each level, as well as a clear interaction across levels. And then API designers can get any of the possible outcomes.

Right now, it is asymmetric because security is available globally but not per-path, and parameters are available per-path and operation, but not globally. So processing these together involves some weird bouncing about the parse tree, picking some things from here, others from there.

[handrews]

Update from today's TDC call:

• Let's tackle each field separately, starting with security, then parameters, then headers
  • No major objections to parameters but the existing behavior is vexatious; we can't remove it, though, so this probably doesn't make it that much worse. But we can take time with this by starting with security instead.
  • headers requires some thought on how response headers interact with status codes to which they are not relevant
  • No immediate objections to responses, but general agreement that we should figure out the others and then see where we are before deciding what, if anything, to do there
• It's not necessary to do all of this work prior to higher-level work on SAFs
  • API Keys SAF already outlined, although we need to figure out what we're doing with Security Schemes or their replacements in general before we can commit to something
  • Deprecation and Sunset are great first SAFs to work through as they are very simple, and also are ones with valid use cases at each scale
  • Rate Limiting adds a bit more complexity by having a runtime-volatile element
  • HTTP Authentication would be a good framework-style SAF as it supports most other security approaches (although not API Keys or mTLS)

[handrews]

I've opened:

• PR v3.3: Add security field at the Path Item Object level #5319 for adding path-level security (issue Security Requirement Consistency with Server Objects #1416)
• Issue v3.3: Global parameters #5320 for figuring out how global-level parameters would work. I identified a few odd corners, but I think in all cases the answer is "keep doing what we're doing" and if we need to add more functionality we can do it in a future release based on real feedback.

[lornajane]

I'd add that we did not necessarily commit to filling ALL the gaps in the matrix of what you can do where, but agreed on a starting point for security at path item level.