Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

MatchSpec is a fundamental conda construct, used in various places for things like dependency specifications and search (though arguably MatchSpec should not be used for the latter). However, to the best of my knowledge, there is currently no formal specification for MatchSpec beyond "what's in the code" (e.g., comments in conda/models/match_spec.py and tests/models/test_match_spec.py) and random bits of institutional/community knowledge.

We should provide such an official specification so developers and users in the conda ecosystem have a shared understanding of what MatchSpec is, and what it does and does not support.

Why is this needed?

Because "is this a valid MatchSpec?" is (a) an oddly difficult or (b) alarming question to answer; as in, conda install '*=*=*' does not result in an immediate InvalidMatchSpec error.

What should happen?

At least one CEP detailing what we currently support (and maybe don't support) as MatchSpecs. Note that we will probably want to evolve this specification to remove cases like *=*.

Additional Context

No response

With all of the upcoming plugins work and the possibilities related to them, it would be beneficial to come up with a core set of standards and best practices that have been discussed and agreed upon.

Possible topics to discuss include:

• Versioning
• Terminology
• Tests
• Verbosity / debug output
• Output format
• Preserving certain conda CLI arguments (e.g., -n, -p, -c, etc.)

Note: Standards that would only apply to certain plugins (e.g., a solver or authentication plugin) may be discussed within a separate CEP for those particular plugins.

While working on conda-libmamba-solver, we tried hard to maintain backwards compatibility with the classic solver, but getting 100% the same behaviour proved to be tricky. One of the reasons is that it's difficult to separate what is supposed to be a "expected behaviour" from an "implementation detail".

There are some foundational principles for that behaviour that can be found across comments, issues, PRs and other contextual metadata, but there's no detailed specification for what should happen under X circumstances.

I'd like to start a discussion around the need of standardising the behaviour of conda as a CLI tool and the outcomes of a number of actions. Such a document should allow us to clearly answer to questions like:

Given a fresh Python 3.7 environment, should conda install numpy=*=*py39* update the Python version implicitly or should the user also add python=3.9 to the list of specs?

Spec'ing these things out would allow the community to have a wider view of conda actually does (and you'd be surprised to learn how many implicit variables are part of that process), and what future iterations of the ecosystem should do instead. For example, I'd like to see how feasible is to steer the logic towards more explicit behaviour.

Topics to discuss include:

• Scope of the spec: which parts of conda need standardisation?
• Programmatic enforcement: can we create a tool agnostic test suite?
• Command-line interfaces: do we want to have a single CLI standard across tools?

This doesn't mean we need to tackle all of those issues in a single CEP. Actually, using several could be beneficial.

conda takes the directory name as the name of the environment too. This is used for CLI (conda activate xyz) and UX (the (xyz) prompt in the shell). This proposal would add a magic file under conda-meta/ that proposes a name different that the environment directory.

Why is this needed or useful?

• conda-build temporary environments could be renamed underneath for better UX (e.g. the name of the package being built)
• pixi envs could be rendered as the project name instead of (.pixi/env).
• conda-store environments (which are named with hashes to implement atomic updates and rollbacks) could have a user-friendly name.
• Environment names could use spaces and other characters that are usually not very path friendly.

The idea would be very simple. If a conda-meta/env-name is present, parse the file contents and use that as the name.

Exploratory work is current underway for providing a series of hooks that will enable third parties to override how conda does its networking:

• conda/conda#11821

With this CEP, we aim to clearly articulate our arguments for doing this in an effort to get consensus from the steering council. This CEP will outline the new plugin hooks that we would like to add to conda as well as a full explanation of how they work and what there intended uses will be.

⚠️ This work is still being actively researched. All findings about what this particular CEP will need will be added here as comments.

Not sure if/how/what we'll formalize into CEP(s). But I'm starting to think about how we could use conda to deliver large data sets and other "non-code" binary blobs. Think reference genomes for bioconda packages, corpora for nltk, pre-trained models for ${your favorite new gen AI package}, etc.

We can currently do things like just generating "huge" (multi-GB) packages or (ab)using post-link/activation scripts to run wget some-suspect-url, but I'm wondering if, as a community, we can come up with more clever solutions.

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

It would be beneficial to linking to CEPs if it would be hosted under a domain like ceps.conda.org or similar. See https://peps.python.org for an example.

Why is this needed?

No response

What should happen?

No response

Additional Context

No response

The conda capitalization rules were first standardized in a CEP in #31. However, we determined that it would likely be better to not bring the issue to a vote and just put it in the lower level CEP documentation. This ticket is for that work within conda-incubator.

Related Cases

• conda/conda-docs#803
• conda/conda#11711

It would help to clarify the process of CEP review/feedback, so that when a PR is transitioning from draft to non-draft status as a pull request, community members would be able to see that draft review can start.

Note: This was taken from the conda wiki before it was deleted.

In transitioning to Conda-forge recipes for the Anaconda distribution, we have needed to resolve several package name differences between the conda-forge and defaults channels. There are instances of old defaults names being replaced by conda-forge names, and also instances of conda-forge names being replaced by defaults names.

Maintaining connections:

Generally, it is sufficient to use metapackages to point from one package name to another. For example, conda-forge uses the boost-cpp name for the boost package, while defaults prefers libboost. The defaults recipe can provide both in one recipe with the {{ pin_subpackage() }} jinja2 function:

outputs:
  - name: libboost
    script: install-libboost.sh   # [unix]
    script: install-libboost.bat  # [win]
    requirements:
      build:
        - {{ compiler('c') }}
        - {{ compiler('cxx') }}
      host:
        - icu               # [unix]
        - bzip2             # [unix]
        - zlib

  # Metapackages to mirror conda-forge's name. It is my goal to deprecate
  # these names and eventually stop providing the packages.
  # TODO :: Aim to remove these by Oct 10th 2018.
  - name: boost-cpp
    requirements:
      run:
        - {{ pin_subpackage('libboost', exact=True) }}

Removing metapackages

This CEP is more about what to do after that, though. From the boost example above, Ray left a comment that he'd like to stop providing the boost-cpp metapackage by Oct 10th 2018. Out of concern for reducing workload and freeing up old names that may be desirable, we need a way to deprecate package names, and communicate to people that packages may either go away or change behavior in the future. If they have pins to old versions, they'll be fine. If they are not pinned, their installations or builds may break.

Proposal:

Add new metadata in meta.yaml:

build:
  end_of_life:
    date: YYYY-MM-DD
    reason: split into devel and lib packages; users should choose appropriately
    alternate_packages:
      - somepkg-devel
      - libsomepkg

Teach conda about that metadata, and have it show warnings when it is installing a package marked with end_of_life. These warnings should also show up when conda-build is using conda to create environments.

as a general rule, provide 12 months from the date of adding end_of_life metadata until actually replacing that package with something that has different functionality.

In various cases, users would like to take advantage of various CPU-specific features or instructions for improved performance, e.g., AVX* or AES/crytography.

xref issues:

• conda-forge archspec-enabled packages

cc: @ltalirz

CEP 1 should explain in detail what the steps and what the possible outcomes are. Bonus points if we can get it as a flowchart.

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

Allow fixing mistakes in run_exports metadata

Why is this needed?

In July 31 2024 meeting, I expressed interest in adding run_exports hotfixing capabilities. @jezdez pointed out that the existing hotfixing capability for repodata.json is not well documented, and also differs between Anaconda and Conda-forge.

@jaimergp proposed conda/conda-index#102 and implemented this in CEP 12 and conda/conda-index#110. As Jaime mentioned, this opens the door to hotfixing run_exports metadata when conda-index gets run.

At time of writing, I'm not sure that the actual process of generating and applying patches has been done. In discussing this, @wolfv said that he'd really like to have run_exports information be co-located with the repodata.json file. This costs some file size space in repodata.json, so it has been undesirable. However, with sharding by package name, the size impact will be relatively small, and the simplicity of having the run_exports metadata in the main repodata.json for each shard is worthwhile.

What should happen?

• Document the current process that is used when hotfixing repodata.json for conda-forge
• Propose a process for hotfixing run_exports
• Define a migration path for combining run_exports metadata into repodata.json (may require sharding to be implemented first)

Additional Context

No response

run_exports has been immensely helpful. Just to summarize it run_exports:

• Allows packages with knowledge of what they need to express it for easy downstream consumption
  • Packages that are part of the build can add dependency to run
    • weak only happens when the dependency is added to host (not build)
    • strong happens when a dependency is added to build or host
  • Dependencies added can express ABI compatibility

However there are still some cases that are not covered. In particular there are cases where a dependency added to build needs to add itself to host, but there is no need to add it to run (in fact this would often be better to avoid). IOW run_exports/strong does not meet this need. Some examples include:

• Header-only dependencies of a package
  • conda/conda-build#3796 (comment)
• Tighter alignment in build & host of compiler dependencies (like OpenMP)
  • conda-forge/openmp-feedstock#126
• Compiler runtime libraries that are statically linked (like cudart)
  • conda-forge/staged-recipes#21723 (comment)

Solving these cases would involve a host_exports that allows adding other dependencies to host. Could imagine to use cases for a package providing host_exports

• host_exports/weak (default)
  • Add dependencies to host if providing package is added to host
  • Solves the header-only dependency case above for a library
• host_exports/strong (opt-in)
  • Add dependencies to host if providing package is in build or host
  • Handles the compiler dependency alignment problem (with OpenMP)
  • Handles statically linked compiler libraries (like cudart)
  • Of most use for compiler use cases

Note: I was going to post this in conda/schemas directly but I can't tag the steering team from there, hence the issue here.

We have a few CEPs coming up (see #51 #8 #50) where it has been suggested that we document the proposed standards with an accompanying JSON schema in the conda/schemas repo.

As I start writing some of these schemas down, I will be using pydantic to write reusable models that can auto export JSON documents. I wonder if there's an interest in doing this retroactively for existing models that have not been standardized yet, so if this catches up, I'd like to propose the following repository structure:

• examples/: Real-life examples of the documented file types, that can be used in test suites if necessary.
• schemas/: Publishable JSON schema documents.
• models/: Python scripts that generate JSON schemas into schemas/. Most likely using pydantic.
• scripts/: Other helper utilities (e.g. refresh all JSON schemas from their models, publish to a HTML tree, etc).
• tests/: Automations to make sure the proposed models actually implement the existing "standards" (as in examples/), but also reject non-compliant files (e.g. extra keys due to typos).

Eventually I can foresee how the models directory becomes a collection of well structured modules with no extra dependencies (other than Pydantic) that would be best served as an importable Python package. Downstream packages (maybe including conda!) could benefit from using these models to transparently add typing their code paths, or at least compare their custom implementations to accepted community standards in their test suites.

I'd love to hear thoughts from @conda-incubator/steering! Thanks!

It would be good to have a CEP that spells out what is in the .conda format as this is missing atm. Especially as we increasingly rely on this and depend on a few tools to manage reading and writing these. Currently the info we have, which could be used for this CEP is...

• https://docs.google.com/document/d/1HGKsbg_j69rKXPihhpCb1kNQSE8Iy3yOsUU2x68x8uw
• https://docs.conda.io/projects/conda/en/stable/user-guide/concepts/packages.html#conda-file-format
• https://www.anaconda.com/blog/understanding-and-improving-condas-performance

Would be good to pull this together to provide a single point of truth.

Independently there are some things that we might want to consider to amend the specification like generating/reusing a Zstandard dictionary for faster and more compact compression/decompression and have per file format dictionaries (text files may benefit a lot from this for example).

Hi All,

I wasn't really sure where to put this, but I think it might become a good CEP candidate. We'll see where discussion goes.

Abstract

Vibrant Open Source ecosystems have places for the community to gather and engage online. Conda already has several places for doing this. This proposal argues that we might have too many such places and that we would benefit from consolidating and updating our channels for community communication.

Motivation

Conda currently has at least 4 conda-centric channels where people can ask questions, post answers, and publish news. In addition there are several nearby channels that aren't about conda per se, but are about parts of the conda ecosystem. Finally, there are a number of places around the web where people often post conda related content.

The central goal of this proposal is to reduce the number of "obvious" places where people think to post and look for questions and answers. These channels will eventually be promoted through things like conda.org and @condaproject on Twitter, for example.

There are many potential benefits to consolidating and updating:

1. There will be fewer places to search for previous questions and answers, making previous discussions easier to find.
2. Fewer communication channels mean a broader audience on each of the remaining channels.
3. Having fewer, but more obvious places to post queries may result in people who currently post to StackExchange or Reddit (for example), posting more of their queries in conda channels instead.
4. Having fewer channels increases the effectiveness of community building. It's harder to form a coherent community when there aren't obvious places to ask questions and search for information.
5. Reviewing and updating our channel platforms may reduce barriers to participating.

Communication channel consolidation is also an ideal time for us to define what each of our channels are for and to rethink our communication goals.

Specification

We propose:

1. Consolidating around a single "chat" platform for real-time discussions.
2. Consolidating around a single "forum" platform for long-form question and answer threads.
3. Creating a "conda announcements" mailing list.

And, a little more detail:

• Actual issues and feature requests would still end up in the various source repositories. Discussion about issues and feature requests may first happen in the chat or forum channels, before they show up in GitHub.
• We are not proposing (initially) that conda-forge, Bioconda and others migrate their specific channels to the consolidated conda channels. However, the hope is that if these channels are successful, then associated communities might be interested in doing so in the future. We would always welcome this.
• The announcements mailing list would be broadcast only. It would not be a forum for discussion, but rather a place for community news, open to use by all projects in the conda ecosystem
• All of this can be seen as a precursor to, and complementary to, a conda community hub site at conda.org. We hope to have that up later in 2022.

Alternatives

If we were to consolidate, what platforms would we consolidate on?

But, first…

A word about NumFOCUS

The conda Organization has applied for Fiscal Sponsorship from NumFOCUS, and this application is currently under review. If the application is accepted, then can NumFOCUS offer us any options as a sponsored project? According to NumFOCUS:

• They offer free Zoom Accounts
• Discounted Slack Pro Accounts. This is likely the US$1 to US$1.15 per person rate that non-profits receive.

That discounted Slack rate is included in the discussion below.

Chat platform

Conda currently has chat rooms in both Gitter and Slack. What are the pluses and minuses of these options, and what other options are there? Chat platforms tend to be used for a mix of real time and asynchronous communication. Individual posts tend to be shorter in chat than they do in forum platforms.

Common challenges with chat platforms include:

• Searchability
  • How easy is it to search content within the platform?
  • How easy is it for search engines to search content within the platform?
• Linkability
  • How easy is it to share links to particular posts/discussion?
• Threading
  • How much support is there to support threaded conversations, and to avoid overlapping parallel conversations in the same room?

Gitter

Gitter is a widely used platform and it is one of the two chat platforms currently used by conda.

Features and foibles:

• It is one of our current platforms so people understand how to use it.
• Free!
• Searchability does not exist in the Gitter Web view.
  • Search in the Gitter Desktop app is limited to the current channel and to the names of other channels.
• Gitter does not have mobile apps.
• Requires a login on Twitter, Github or GitLab to contribute, but anyone can read..
• Supports threading, but almost no one uses it.
• NOTE: The owners of the Gitter platform plan to deprecate it in favor of Element/Matrix (see below)

Slack

Slack is also a widely used platform that is one of the two chat platforms currently used by conda. It has a nicer user interface than Gitter.

• It is one of our current platforms so people understand how to use it.
• Free version is available - it's what we use now
  • But Slack is changing its free policy from "you can search the last 10,000 posts" to "you can search the last 3 month's posts."
  • Paid non-profit version costs $1 to $1.15 per User per month, which is not sustainable with our numbers.
• Search platforms like Google can't index the content.
• Requires a login to read it.
• Supports threading; has mixed adoption.
• Good search and community management features.
• Limited moderation features.
• Barrier of entry for new community members because of having to use invite links.

Element/Matrix

The same people who own Gitter (and plan to deprecate it) also own Element. Element is a front-end to Matrix-compliant message platforms. Matrix is a protocol for open communication, maintained by the non-profit Matrix.org foundation. Gitter is compatible with Element/Matrix, with one exception: You can't see private groups from Gitter in Element.

• Element feels a lot like Slack, so it is easy for most people to learn
• It's still another platform that many people will not have prior experience with
• It’s open source and free
• It's basically a better Gitter
• Supports threading
• Matrix-based rooms are published on the web but they do not appear to be indexed by Google.
  • Google can find the archive, but doesn't seem to search the archive.
• Element has a web-based interface, and an app-based interface.
  • You can view rooms when you aren't logged in
  • But the web interface does not appear to support in-room search, unless you are logged in.
  • And this path is not indexed by Google either.

Zulip

Zulip walks a middle ground between chat and forum platforms. Every conversation on it is threaded and has a title, but the feel of each thread is like a chat. It is possible that Zulip could be the single Chat/Forum platform for conda.

However, there are drawbacks, and these are summarized by this frequently heard statement from Zulip users:

I didn't like Zulip at first, but once I got used to it, I quite liked it.

Which says that Zulip is a good platform that is initially off-putting. And that's the challenge. We want a platform that encourages people to engage. Zulip does that, but only after a relatively steep learning (more of an unfamiliarity) curve. It's structured differently from platforms like Gitter and Slack, and that takes getting used to.

The driving question here is how many people would we lose before they learn the platform?

• Zulip Cloud Standard is free for open source projects.
• The public access option lets administrators configure selected streams to be web-public. Web-public streams can be viewed by anyone on the Internet without creating an account in your organization.
• Logged out visitors can browse all content in web-public streams, including using Zulip's built-in search to find conversations. Logged out visitors can only access the web-public streams in your organization, and the topics, messages, uploaded files, and emoji reactions in those streams.
• Web-public streams do not yet support search engine indexing. You can use zulip-archive to create an archive of a Zulip organization that can be indexed by search engines.
  • Well, that is kind of low budget web search support for chat/forum access.
  • For the forum it is essential that the posts get indexed, and that search results point into the chat/forum where people can start participating. Zulip does not give us this.

Discord

Discord is another widely used chat platform. It's also free. Downsides include that you have to join the channel to see content, and you need an invite to join.

• Discord an account to read content.
• It can't be indexed by search engines.
• Discord's internal search capability is not bad.
• Discord is available for free.
• Discord supports a large number of discussion rooms and private chats (as does Element, Gitter, and Slack)
• Has extensive moderation support.
• Many people will already be familiar with Discord.

Summary

We have several choices for a chat platform.

• Some of them are not readable unless you log in first.
• None of them are usefully indexed by Google (although some have more hope than others).
• Some have better search support than others.

Current Recommendation

Note: The "current recommendation" is whatever the current leaning is. Once discussion has settled, the current recommendation will be posted as the recommendation in the CEP PR. We will also not include the full discussion from this issue about different platforms that were not selected. The CEP will focus on the choice we make in this discussion.

First Choice: Element

Why?

• Slack's recent policy change makes it unaffordable to search chat history.
• Gitter is going away.
• Zulip has too steep of a learning curve.
• Discord requires invitation and login to see anything. It has no hope of being publicly searchable
• Element is free, has a future, is open source, does not require an invite or a login to reach content, and has a shallow learning curve.

Forum platform

Forum platforms complement chat platforms. Forums are particularly good at longer form questions, answering threads, and creating an online archive / knowledge base over time about the topic. All conversations are threaded in forums. However, there is more startup friction in forums than there is in chat. Typing a sentence or two in a chat platform feels less intimidating than creating a new post in a forum.

Discourse

Discourse is a widely used forum platform in the open source world. A hosted solution is available for free. Discourse is infinitely configurable and extensible and has good community management features. It will be familiar to many in our community, and will feel familiar to anyone who has used StackExchange.

• Is free for open source communities, with some limits:
  • 5GB total storage.
  • 50K / month page views, including web crawlers.
• Is indexed by search engines
• Has rewards based gamification (but not ranking gamification)
• Provides robust support for tagging and categorization of content.
• Many plugins are available (see Standard plugins)
  • This includes a plugin to post activity from Discourse into Slack, Discord, Zulip, Gitter, and Matrix.
• Familiar look and feel because of StackExchange and wide use of Discourse.

GitHub Discussions

GitHub Discussions is a way to have a discussion forum within the GitHub ecosystem.

• Free
• Indexed by search engines.
• Requires a GitHub login.
• Supports categorization of posts, including custom categories.
• Supports labels (tags)

See the conda Infrastructure discussion board for an example.

Zulip

Zulip, as mentioned above, is both a chat and a forum, or a forum that feels like a chat. See above for most pluses and minuses. One particular minus for Zulip as a forum is that posts are not searchable by search engines. This is a significant drawback for a forum because over time we want the forum to be a knowledge base. That doesn't work if Google can't find it.

Current Recommendation

First Choice: Discourse, followed by GitHub Discussions

Of these options, Discourse and GitHub Discussions ate the only choices because of Zulip's lack of search engine indexing. Discourse is a widely selected choice for communities like ours. GitHub Discussions may also be a viable option. Discussions is integrated with GitHub, but also does not appear to have the robust infrastructure and forum management options that Discourse has.

Email platform

What happens in chat and forums now, used to happen in mailing lists, and it was in almost all ways more painful in mailing lists. However, mailing lists still have a purpose as news distribution channels. Is there a new release of conda-lock or mamba? Is there an upcoming PackagingCon, or training about defining your tools in conda? Announcement lists are good for keeping the community up to date on that latest and greatest big news.

Announcement only lists have different requirements than discussion lists: There is no need to support threading and moderation for instance, as there is no discussion. Discussion happens in chat.

So, what are our options here?

Self Hosting

We could set up our own MailMan (MailMan2 please, not 3) or Sympa server. This gives us total control over the mailing list, and allows us to easily set up additional mailing lists when we need them.

However, there is currently nothing else in the plans that requires the conda Organization to host a live server that we are responsible for keeping up and running (and paid for). We are currently pretty sure that everything in conda.org can be run by GitHub. Mailing lists alone may not be enough justification to take on setting up and maintaining our own server.

Google Groups

Google Groups is another option. It has most of the advantages of having our own hosted mailing list server, but without all the drawbacks.

Google Groups is free to use, and we could probably even have the list name be [email protected].

There are limits, but those limits aren't published anywhere! Unless we want to pay for support, in which case they are published. It seems that the only way to understand the limits is to push them.

Other options for hosting?

Email Services

Services like MailChimp and Constant Contact offer very polished services for announcement only mailing lists. However, these services cost money, and polish is not really a priority for open source projects.

Options

There are a bunch of options out there for announce-only mailing lists. Here I present what the "best fit plan" looks like for a number of them. "Best fit plan" is defined as up to 2500 list members, and up to two emails per month (although I think that would be a heavy month).

It's also worth noting that it should be easy to migrate off of any of these services whenever we want to.

From these options, Sender.net appears to be the best. Its next step up is 5K subscribers for $18.25 / month, which is also better than the rest.

Current Recommendation

First Choice: Google Groups, followed by Sender.net if necessary.

Google Groups is widely used and can be configured to be announce-only.

Lean away from hosting a mail server ourselves. We can reconsider this if we ever need a server for another purpose.

What to do with existing channels?

All this work doesn't do any good if we aren't successful in moving people off of any of the channels we are trying to move people away from. How do we accomplish that?

Conda-Centric communications channels

Among the four conda-centric channels three can be combined into a chat and a forum channel. GitHub issues is the one channel that should stay in use as is.

Conda Gitter and Conda Slack should be combined. If we stay with either Gitter or Slack, then it's a matter of shutting down the other one and encouraging users to migrate. If we favor of a new platform, then we would shut down both and encourage migration.

The conda Google Group would be shut down and we would encourage members to use the new chat and forum platforms, and to subscribe to the announcements mailing list.

Nearby communication channels

For the nearby channels we don't need to do anything. But we should do some things like

• reach out to channel owners to let them know what we are doing, and seek their advice.
• let them know that they are welcome to join the conda channels now, or later, or not at all.
• Announce when new conda channels are ready. Stress that they are not replacing the nearby channel, but rather are for conda-centric discussion.

Across Town communication channels

The across town channels exist for reasons that are completely independent of conda. We can't make changes to these platforms, even if we wanted to. But, we can change how we recommend the community responds to items posted to these channels.

Depending on the channel (some examples):

• Twitter - usually point people to official channels, sometimes pointing them to where this question has been asked before. Twitter is not a great discussion channel
• Reddit, StackExchange, similar - Sometimes answer in place. Always include pointers to official channels

Resolution

Summarize our decisions here. Which we will do, once we create the CEP.

Recently we started tracking provenance metadata in conda-forge builds ( conda-forge/conda-smithy#1577 (comment) )

This follows the same metadata Anaconda appears to use as well ( conda-forge/conda-smithy#1577 (comment) )

Wonder if we should promote this metadata and document it somewhere. Idk if it should be a CEP or a guideline or example in the doc

Opening here to discuss and track

Motivation and rationale for example would allow community members to better understand the proposal.

Current practice are based on de facto implementations in conda and conda-index. For clarity, this should be standardized.

Roughly, current setup is:

• path/or/url/to/channel-name/subdir/repodata.json
• path/or/url/to/channel-name/subdir/package_name-version-build_string.tar.bz2

Some parts of conda require at least <subdir>/repodata.json to always be present.

In some cases, channel name is empty because the host directly exposes the subdirs in the root (e.g. http://my-custom-channel.org/noarch/repodata.json). This seems to be unintended and also affects how channel::package specs work.

We don't have clear rules for allowed characters or case sensitivity, either.

I wonder if this should be standardized in the context of a HTTP API explicitly (e.g. document expected result of GET requests), like the Open Container specification.

As discussed on Gitter and the 2024-03-27 community meeting, we should write a short CEP that defines what information the OS virtual packages (__linux, __osx, __win, etc.) should provide (e.g., what values go into the package name, version, and build). We decided a CEP was needed because there are multiple conda-compatible implementations (e.g., conda, mamba, pixi), and a CEP would allow these implementations to coordinate.

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

There was a conversation in a downstream tool from the conda-forge community that is worth discussing here as it relates to an existing CEP: conda-forge/conda-smithy#2030

Additional Context

• conda-forge/conda-smithy#2030

Note: Last year when the CEP process was relaunched to facilitate community inclusion and feedback, the pull request for CEP 2 was accidentally merged before calling for a vote following the conda governance policy, despite having received a few reviews from community members.

This vote is largely to rectify this procedural oversight and more closely follow the governance document.

@conda-incubator/steering
This issue falls under the Enhancement Proposal Approval policy, please vote and/or comment on this PR.

This PR needs 60% of the Steering Council to vote yea to pass.

To vote please leave 👍 (yay) or 👎 (nay) responses.

If you would like changes to the current language please leave a comment (in the PR).

This vote will end on 2022-05-24.

Closes #1

Since each CEP is a single file, could use a emoji to signify its status? You can add these to the front of the commit message/PR. This would allow us to quickly see at a glance what the status of things are. I tried an example in the CEP-4 branch

🔨 for Draft
🛠️ Proposed
✔️ Accepted
➖ Rejected
✅ Implemented
⚡ Deferred (Not sure on this one)

@jezdez what do you think?

https://github.com/orgs/channel-mirrors/packages has a mostly complete mirror of conda-forge and bioconda in GHCR.io. Each anaconda.org artifact (be it tar.bz2 .conda), has been pushed there under the following conventions:

• Name of the "container": <channel>/<subdir>/<package name>
• Label of the container: <version>-<build_string>

For each name+label combo, we have the following manifest of layers:

{
  "digest": "sha256:<hash>",
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "size": <size in bytes>,
  "config": {
    "digest": "sha256:<hash>",
    "mediaType": "application/vnd.unknown.config.v1+json",
    "size": <size in bytes>
  },
  "layers": [
    {
      "digest": "sha256:<hash>",
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "size": <size in bytes>
    },
    {
      "digest": "sha256:<hash>",
      "mediaType": "application/vnd.conda.package.v2",
      "size": <size in bytes>
    },
    {
      "digest": "sha256:<hash>",
      "mediaType": "application/vnd.conda.info.v1.tar+gzip",
      "size": <size in bytes>
    },
    {
      "digest": "sha256:<hash>",
      "mediaType": "application/vnd.conda.info.index.v1+json",
      "size": <size in bytes>
    }
  ]
}

I think the tar and conda layers correspond to tar.bz2 and .conda ones. Always present, but only one is populated. The info layers correspond to the compressed info/ contents and the info/index.json file, respectively.

This should be standardized in a CEP.