[CEP XXXX] OCI Storage of Conda Artifacts

[beckermr]

This draft CEP has an updated specification for storing conda packages as OCI artifacts. It is an updated form of the specification in PR #70, given the feedback on the previous PR.

Rendered CEP

[jaimergp]

A few cosmetic comments, but unless I understood wrong, there's a problem with the _ OCI-encoding.

[beckermr]

Good catch @jaimergp! I reran my notebooks with _ going to _U and everything works great.

Interestingly enough, we apparently have no build strings with a double underscore in either defaults or conda-forge! All OCI-encoded conda artifact names I produced using _ -> __ from both of those channels passed the OCI regexes from the Distribution Spec. Fun!

[beckermr]

Thanks for the additional comments @jaimergp! Any thing else you can see?

[jaimergp]

Nothing too big, just a couple of observations:

• The copyright statement needs to be put back in.
• A References section that compiles the different URLs mentioned would be welcome.
• I would wait until this CEP is approved to assign a number. There are a few ongoing PRs that might get voted before this one and then it would be confusing. e.g. there's a PR named "CEP 17", but CEP 17 ended up being this one. This might be reflecting a problem in how we mint CEP numbers. Happy to discuss further!

[beckermr]

pre-commit.ci autofix

[jaimergp]

This is the same as the one proposed in #116, right? We should update this part to refer to that one once accepted.

[beckermr]

I was thinking on this more and I think we should not use the m prefix and instead disallow current repodata on OCI channels. My reasons are

• Older conda clients that had performance improvements from current repodata won't be able to read OCI channels directly anyways. They can fall back to repodata and the client should be upgraded anyways.
• Older conda clients that access an OCI channel via a web proxy could request current_repodata.json and the web proxy could translate to repodata_current.json for use with an OCI channel if we wanted.
• The CEP in [CEP 26] Identifying Packages and Channels in the conda Ecosystem #116 specifies that the URL of a conda channel needs to have <channel base URL>/noarch/repodata.json as a valid address. As long as we stick to the tag "latest" being the current most recent image, then we can meet this spec.
• We will likely save ourselves some pain by only having to deal with prefixing package names with c as opposed to having to prefix everything.

Thoughts @jaimergp?

[jaimergp]

All of that is currently valid and sound. I'm just worried that we are being lucky now with no conflicts in the filename prefixes (I do second dropping current_repodata.json, we don't need it these days, and we can consider it an Anaconda.org implementation detail if there's such a need).

I just don't see the pain in prefixing metadata files with m. If we don't, we might run into a situation where we want to add a new type of files, and the only way out would be to add OCI-specific sub-subdirs like conda-forge/linux-64/packages/ and conda-forge/linux-64/metadata/, and that seems like more painful.

But if you think the burden of prefixing m to everything is not worth it, I won't block it. I'd be happy to hear what others think too.

[beckermr]

Hmmm. Maybe the right thing is to distinguish between the abstract url given to conda versus the storage location on disk. We can specify that more clearly in the CEP.