This is based on the conversations referenced in #13

I want to try and have a conversation with the relevant crate authors/maintainers that create their own Array data types in rust. I would like to see if we can get an agreed upon basic data structure and API (Trait) for converting to/from the different types with zero* cost. This issue is intended to be the forum for those discussions. I'm hoping such an effort will be relatively easy and not limit the API's of each individual crate.

Once we get such a crate, we should document it on this site. I think telling the story of why there are different crates is really important because a lot of people expect rust numerical types have a standard in the same way as python does. If you can explain that the conversions are simple and basically free, then explain that they allow for useful features (like array multiplication with *) I think there will be a lot more support and the ecosystem can thrive with less friction.

This thread is for discussion on this topic and is open to anyone. Particularily welcome are the numerical library crate maintainers.

The proposed name of the crate from me is array-interop

* zero meaning no copying or moving (in memory) of the arrays, dimensions etc might be lost/gained.

Please add Cumath crate to category: GPU Computing

Cumath is cuda wrapper, the main crate provide a thin abstraction layer over cuda, cublas and curand

Please add crate to category: NLP (Natural Language Processing)

This category doesn't exist yet

Problem

I found it hard to know what color stands for in Rust Machine Learning Ecosystem section.

Can we clarify what each color means?

Prior art

In Are we web yet, we have following explicit label on each states.

• everything is awesome
• stable, tested and mature
• stuff's pretty great
• getting there
• stable but still maturing
• not yet stable, but progressing
• unstable/incomplete, needs work
• barely there, needs serious work
• basically nonexistent

Hi, I made a crate of utilities for differentiable graphs and would like it to be on are-we-learning-yet. I've implemented an MNIST classifier and a character prediction GRU-RNN as examples.

https://docs.rs/drug/0.0.1/drug/

Tract is a tiny, no-nonsense, self contained, TensorFlow and ONNX inference engine.

Please add https://github.com/pengowen123/eant2 to category: Evolution Algorithms

EANT2 is an Evolutionary algorithm that uses the Common Genetic Encoding (CGE) to encode neural networks and CMA-ES as a weight optimizer.

Essentially, EANT2 evolves the topology of the Neural Network and CMA-ES tries to find the optimal weights for each topology.

Please add crate statrs to category: Scientific Computing

First of all, thanks for this!

Secondly, I just finished writing a tutorial for doing simple spam or non-spam classification, end-to-end in Rust, aimed at Rust beginners. It works through downloading data, parsing it into a sparse matrix (using a bag-of-words model), fitting a simple linear classifier to it, and cross-validation.

It's at the Rustbridge repo here: https://github.com/rust-community/rustbridge/tree/master/workshops/machine_learning

https://github.com/frankmcsherry/timely-dataflow

I feel like this could belong here, maybe in data processing? Although it's possibly too low-level.

Please add these two crates to category: GPU Computing

wgpu-rs is used by emu (which is already on are-we-learning-yet - thank you). However, I believe wgpu-rs deserves its own mention. It supports some GPUs and others are work in progress: https://github.com/gfx-rs/wgpu#supported-platforms.

It contains of three crates:

1.:

• crate wgpu
• https://github.com/gfx-rs/wgpu-rs
• https://crates.io/crates/wgpu
• https://docs.rs/wgpu

2. and 3.:

• one GitHub for both crates: https://github.com/gfx-rs/wgpu, which is an "active mirror" of https://hg.mozilla.org/mozilla-central/file/tip/gfx/wgpu.
• first crate: wgpu-core:
  -- https://crates.io/crates/wgpu-core
  -- https://docs.rs/wgpu-core/
• second crate: wgpu-types:
  -- https://crates.io/crates/wgpu-types
  -- https://docs.rs/wgpu-types/

https://blog.rust-lang.org/2019/04/26/Mozilla-IRC-Sunset-and-the-Rust-Channel.html

Please add crate to category: Neural Networks

https://crates.io/crates/cntk

Please add crate revonet to category: Meta­heuristics

The crate contains implementation of the real-coded genetic algorithm for solving optimization problems and training of feed-forward neural networks.

Thank you,
Yury

Please add crate to category: Neural Networks

neuronika

Please add crate to category: Data Structures

Hey, so this is a request (shameless, I know) to add my reinforcement learning library rsrl (https://github.com/tspooner/rsrl) to your list of crates. It is still in alpha, but only because I have a few API changes in mind before a version 1.0 release. It currently works well and has many state-of-the-art algorithms implemented as well as the classics. It is somewhat inspired by the rlpy library and is intended for use in academia and industry.

I hope you find it interesting!

Cheers,
Tom

Note: I'm currently working on extracting some code from this crate into sub-crates, such as lfa for linear function approximation which will be worked on separately.

The wgpu crate still points to the old repo https://github.com/gfx-rs/wgpu-rs which got moved to https://github.com/gfx-rs/wgpu. Hence the commit info etc is outdated.

For the Scientific Computing Libraries - https://github.com/qcgpu/qcgpu-rust

Theres a site too - https://qcgpu.github.io

I did some quick scanning around the ecosystem to turn up a few things that could be added (but didn't take the time to figure out how they should be added, yet):

• Evokit: Github, r/rust announce, Podcast
• TiberiusBlog multiple blog posts about Rust ML - more recent that most the AWLY content
• RustLab 2019 Rust ML talk - more recent than most AWLY content

Please add bacon-sci to category: Scientific Computing

https://github.com/aftix/bacon

Please add crate to category: Neural Networks

Pro-tip: a quick PR to _data/crates.yaml often gets merged quicker.

Please add crate to category: Data Processing

Pro-tip: a quick PR to _data/crates.yaml often gets merged quicker.

I just had an excellent conversation with the developer of rulinalg and he mentioned the following:

Both ndarray and rulinalg already store the elements of their arrays/matrices in a single long contiguous vector. This means that you can already convert an ndarray Array2 into a rulinalg matrix essentially for free**

This is pretty essential information that I think should be included somewhere and your site seems like the best place.

Maybe there needs to be a library that specifically supports data type interop between libraries or something -- but it should definitely be documented.

leaf has been officially deprecated by their devs. It should not be mentioned anymore.

I got build CI working, but would still like to automate and cron the republishing. It seems the deploy script provider runs in a different context than the build task (i.e. w/o the bundler context), so rake publish doesn't quite work as a deploy script. I should probably be using the pages provider anyway. Calling it a night, but I want to remember where I left off got the next time I pick this up.

A generic k-means clustering library and command line tool for finding dominant colors in image buffers. It uses kmeans++ initialization and offers implementations of Lloyd's and Hamerly's algorithms for finding k-means.

https://github.com/okaneco/kmeans-colors
https://crates.io/crates/kmeans-colors
https://docs.rs/kmeans_colors/

Please add crate to category: GPU Computing

crate
repo
docs

Please add crate NeuroFlow to category: Neural Networks

Please add crate to category: Neural Networks

If you're open to multiple categories, I think dfdx could also be added to Reinforcement Learning (I have some examples of a Deep Q Network and Proximal Policy Optimization).

github: https://github.com/coreylowman/dfdx
crates.io: https://crates.io/crates/dfdx
docs.rs: https://docs.rs/dfdx/latest/dfdx/

Crates listings should include some richer data like:

• Last commit
• Number of contributors
• GH issue counts
• License

So crates.yaml will need replaced with a workflow that regularly regenerates all this data.

And sorting of crates should reflect some concept of popularity (e.g. downloads), stability (e.g. > v1.0.0), activity (contributors and recency of last commit), etc...

Please add crate to category: Data Structures

The bvh crate implements a Bounding Volume Hierarchy, is very fast and very well documented. It should be used for spatial queries. A new release with a dynamic and self-healing BVH will be made soon.

Please add crate to category: Scientific Computing

https://github.com/Nateckert/newton_rootfinder

Thanks in advance :)

data structures:

• numeric

scientific computing:

• linxal
• ndarray-linalg

Hey everyone :)

As part of the AeroRust unofficial wg I would like to know if it's ok for us to fork this project and use it as a boilerplate for areweinspaceyet.org

The page looks really great, and it would be a pretty cool way to bootstrap our space effort.

Thanks a lot for your time :)

Please add crate to category: Scientific Computing

mine this roundup from a 4 months ago: https://www.reddit.com/r/rust/comments/4dnq0h/gpu_programming_using_rust/

via gtani on reddit: https://www.reddit.com/r/rust/comments/4z7vpf/cataloging_the_rust_machine_learning_ecosystem/d6v22qc?st=isbcx10i&sh=e4fcfe52

I think it might be worth creating a category for packages that deal with sparse data well --- this is extremely important for all sorts of NLP and recommendation applications, where very large and very sparse matrices are commonplace.

Obviously, rustlearn supports this, or all of its classifiers (including random forests!) :)

I just submited a pull request to add the simplers_optimization black-box optimization crate to the metaheuristics category.

It is a Rust implementation of the Simple(x) optimization algorithm.

As of writing (9 December 2021), cobalt v0.17.5 is not publicly available through the Releases page, and the most recent v0.17.4 throws an error when attempting to build. It took a few minutes to figure out why (I thought maybe the README page had a typo on the version number) before I looked in the CI config file to find that Cobalt was actually being installed from the Git repository using

$ cargo install \
            --git https://github.com/cobalt-org/cobalt.rs \
            --rev 6047852d62aa505a4dc6a82715485d465be5c596

Until the next version is released (although Cobalt has a history of regular releases, I can not find a planned schedule), a note should be added to the README on how to install the required version.

Please add crate to category: Scientific Computing

This crate implements generalized linear models for ndarray array data. PR #92

Not sure if this belongs here, but a new section for Metaheuristics may be warranted.

https://crates.io/crates/metaheuristics

My rust-cuda project was announced recently, with the goal of making that nice GPU Computing box yellow or green :)

However, im not sure what specific crate to include, because the project is many crates that contribute to rust gpu computing.

Please add crate to category: Data Processing

If you think it belongs there that is: https://github.com/andygrove/datafusion-rs

Please add Prophet crate to category: Neural Networks

Prophet is an actively developed simple feedforward, backpropagation neural network with many basic features around those algorithms.

In its current form it is similar to the NN crate that kind of seems to be abandoned.

On the front page under "Additional resources" the first link leads to a 404 unfortunately:

https://suchin.io/2016/09/13/The-PlayRust-Classifier/

After the big rewrite in #96, the 6 or 7 crates that aren't on crates.io, don't have any data generated into crates_generated.yaml, and thus don't get rendered at all.

Please add crate to category: GPU Computing

RustaCUDA helps you bring GPU-acceleration to your projects by providing a flexible, easy-to-use interface to the CUDA GPU computing toolkit. RustaCUDA makes it easy to manage GPU memory, transfer data to and from the GPU, and load and launch compute kernels written in any language.

Please add crate to category: GPU Computing

Pro-tip: a quick PR to _data/crates.yaml often gets merged quicker.

https://github.com/bheisler/RustaCUDA

Hello there!

I am working on a few machine learning related crates. Now feels like a good time to add them to are-we-learning-yet :)
Unfortunately, I am not sure which categories some of them should go into, or if they should be added at all. That's why I did not simply create a pull request.
The alleged ML crates are:

• forester: Random Forests and Decision Trees. This one is easy; the "Decision Trees" category feels like a good match.
• openml: An interface to OpenML. It handles data set downloads and running user defined ML algorithms. There does not seem to be a well-matching category¹. "Data Processing" seems like the best fit, but it is more about pre-processing than data set loading and model verification. Any ideas?
• arff deals with the ARFF file format. I'm not sure if this crate should be added at all because it's just a (de)serialization library. Granted, the file format is machine-learning specific (only used by Weka and OpenML as far I know), but does this make it suitable for are-we-learning-yet?

¹_{Maybe it would be appropriate to add a new category, like "Data Sets" or "Evaluation"?}