quansight-labs / conda-metadata-app Goto Github PK
View Code? Open in Web Editor NEWA streamlit app to query metadata from conda packages
Home Page: https://conda-metadata-app.streamlit.app/
License: MIT License
A streamlit app to query metadata from conda packages
Home Page: https://conda-metadata-app.streamlit.app/
License: MIT License
Currently, the app will (correctly IMO!) try to show the latest version and latest build. However, there's some mismatch between what the UI sees as the newest available version, and what's actually available in terms of artefact metadata. This often manifests in the following kind of errors (example is from just entering boost-cpp
):
Using the dropdown to navigate to an older build usually works, but it's not a great user experience (especially for those who don't know this quirk and wrongly conclude that app is broken). So I think we should probably catch that error and fall back to the last version for which we have metadata.
The second issue is that there seems to be some source of substantial delay with adding entirely new packages to the UI dropdown. For example, we added libboost
builds 4 days ago, and they're still not visible in the UI. In many ways, this is the opposite extreme of the above problem. It would be good to understand (for me at least) where this delay comes from, and if we can somehow get it down to under one day.
It should be possible to fetch .conda
metadata (.tar.bz2
can't be streamed) with JS only. This works on the browser:
import { RemoteZipPointer } from "@basisai/remote-zip";
(async () => {
const url = new URL("https://conda.anaconda.org/conda-forge/linux-64/des-fitvd-0.9.19-py39hf3d152e_2.conda");
const remoteZip = await new RemoteZipPointer({ url }).populate();
const fileListing = remoteZip.files(); // RemoteZipFile[]
console.log(fileListing);
})();
Resulting in:
[
{
"filename": "metadata.json",
"size": 31,
"modified": "2024-02-06T18:29:10",
"attributes": 25165824
},
{
"filename": "pkg-des-fitvd-0.9.19-py39hf3d152e_2.tar.zst",
"size": 93411,
"modified": "1980-01-01T00:00:00",
"attributes": 25165824
},
{
"filename": "info-des-fitvd-0.9.19-py39hf3d152e_2.tar.zst",
"size": 8325,
"modified": "1980-01-01T00:00:00",
"attributes": 25165824
}
]
Paired with a zst library, we should be able to fetch the info/
bits like conda-package-streaming
does. This could be implemented in the conda-forge.org website and only point here for the .tar.bz2 bits (which are provided by the OCI backend).
I think the current caching might be incurring in too much RAM usage (tricky to know on streamlit.cloud), because the app has required a reboot a couple times now just to load.
Maybe using something like diskcache
would help here.
Hello, I noticed that I can't bookmark another channel other than conda-forge. For example, opening https://conda-metadata-app.streamlit.app/?q=pkgs%2Fmain will instead redirect to https://conda-metadata-app.streamlit.app/?q=conda-forge.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.