Git Product home page Git Product logo

manta-rs's Introduction

github-banner

manta-rs

Workflow Status Latest Release

Rust Crates for the Manta Network Ecosystem


About

The manta-rs libraries represent the core logic of the Manta Network protocols shared across all of its products. This project is built as a monorepo for the foundational Rust codebase of Manta. See the following repositories for projects by Manta Network that build off of manta-rs:

  • Manta: Core blockchain node implementations built on substrate
  • manta-signer: Desktop ZKP prover and wallet implementation
  • manta-front-end: Web dApp for interacting with Manta networks
  • sdk: Software Development Kit for building wallets and dApps on top of Manta

The protocols implemented in manta-rs are described by the Manta Specifications. See those documents for more information.

Contributing

To contribute to manta-rs please read the CONTRIBUTING.md file for more details.

manta-rs's People

Contributors

alexparshh avatar bhgomes avatar boyuanfeng avatar dengjianping avatar dependabot[bot] avatar ghostofgauss avatar ghzlatarev avatar stechu avatar supremough avatar tsunrise avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

manta-rs's Issues

Scalar Multiplication Algorithms Tracking Issue

These are the classic scalar multiplication algorithms:

  • Double-and-add: #24
  • Windowed Double-and-add
  • Sliding-window Double-and-add
  • NAF
  • wNAF
  • Vector Chains
  • Montgomery Ladder

We should implement (optimized versions of) all the above and provide a uniform interface for applying a specific technique (perhaps a trait which each can implement). Also, we should ensure that each implementation is compiler-independent so we can use it in-circuit.

Improve Simulation Pipeline in CI

Right now we run a barrage of simulations and a lot of them fail because they run too long. We want to run shorter simulations and/or allow them to exit early by emitting correctness checkpoints where we know that the network state is correct.

Manta Possible Vulerabilities

This issue logs the possible manta vulnerabilities (need to be revisit when mainnet ready):

  • Public parameters for LeafHash, TwoToOneHash, and Commitment
    More specifically, we are using a pre-0.3.0 Hash parameters for pedersen, essentially the LeafHash and TwoToOneHash uses the same parameters, this could be a security vulnerability. The way we currently hardcode both parameters for the hash functions and the parameters for the commitment make this hard to change. See comment.
  • Public parameters for ZKP

Add Nova Proof System Backend

We should explore the Nova Proof System as an option for a backend. How to integrate this experimental code into the manta-pay module:

  1. Add an optional nova dependency to manta-pay/Cargo.toml
  2. Add a submodule of the crypto/constraint module gated by the nova feature (see crypto/constraint/mod.rs for examples of how to set this up)
  3. Implement the ProofSystem trait for Nova including (and leverage the existing arkworks curves we already use in manta-pay if possible)

We can use this as a tracking issue and a place to hold discussions about this proof system.

Check Zero-Transfer Edge Cases

We don't want to pay for transfers of zero (and less than existential deposit in some cases) so we need to check these edge cases. Most likely most of these checks can belong to pallet-manta-pay, but we need to investigate them in general.

Add Configurations for More Curves

Our protocol depends on the choice of curve, and right now we are using BLS12-381. For other deployments, other curves might make sense so we should add a configuration for at least these curves:

  • BLS12-377
  • BN-254

Setup `dependabot`

Make sure that dependabot is set up properly. It should automatically add a pull-request whenever a rust dependency is out of date.

Setup Poseidon as a Permutation

To integrate Poseidon in the duplex-sponge encryption circuits, we want to add it as an instance of PseudorandomPermutation (see #81 for definition). To do this we need at least the following interfaces:

impl manta_crypto::permutation::PseudorandomPermutation for Poseidon {}
impl manta_crypto::permutation::sponge::Absorb for Input {}
impl manta_crypto::permutation::sponge::Squeeze for Output {}

where Input and Output are constrained to be ARITY-long vectors (not WIDTH-long since the extra state field element is the "capacity" of the sponge and we cannot read/write it). Then, for Poseidon as a hash function, we should simply implement ArrayHashFunction as usual which will execute the PseudorandomPermutation impl and then project out the 0-th component.

Poseidon Parameter Generation: Separate LFSR from Sampling Distribution/Algorithm

To improve the modularity and usefulness of the Poseidon Parameter Setup code, we want to separate the LFSR implementation from the field element sampling algorithm and the generation of the round constants and MDS matrix.

Old Design

Right now when generating the Poseidon parameters we have something like the following interfaces:

impl LFSR {
    /// Generates a new `LFSR` random number generator with a seed derived from a Poseidon specification.
    fn new(field_size: u32, width: u32, full_rounds: u32, partial_rounds: u32) -> Self { ... }

    /// Samples a random field point using rejection sampling.
    fn sample_field_point(&mut self) -> F { ... }
}

/// Generates the Poseidon hasher from its specification.
fn generate_hasher(field_size: u32, width: u32, full_rounds: u32, partial_rounds: u32) -> Hasher {
    // Generates a new LFSR from the Poseidon specification.
    let lfsr = LFSR::new(field_size, width, full_rounds, partial_rounds);

    // Generates the round constants using LFSR as the random field point sampler.
    let round_constants = generate_round_constants(lfsr);

    // Generates the MDS matrix using a Cauchy matrix over `(0..t) x (t..2t)`.
    let mds = generate_mds(width);
    
    // Initializes the Hasher parameters checking that they match the correct sizes.
    Self::new(round_constants, mds)
}

The issue with this design is that we are very restricted in our ability to utilize other sampling methods or PRNGs for constructing these types. We want to be able to leverage the existing random sampling APIs available in Rust and manta-crypto.

New Design

For each type we have a structure that defines its own sampling methods.

Additive Round Constants

/// Additive Round Constants
pub struct AdditiveRoundConstants<F> 
where
    F: Field,
{ ... }

impl<D, F> Sample<D> for AdditiveRoundConstants<F> 
where
    D: Clone,
    F: Field + Sample<D>,
{
    #[inline]
    fn sample<R>(distribution: D, rng: &mut R) -> Self
    where
        R: CryptoRng + RngCore + ?Sized,
    {
        Self::new(rng.sample_iter(core::iter::repeat(distribution)).collect())
    }
}

Maximum Distance Separable Matrix

/// Maximum Distance Separable Matrix
pub struct MaximumDistanceSeparableMatrix<F> 
where
    F: Field,
{ ... }

impl<F, X, Y> Sample<CauchyMatrixDistribution<X, Y>> for MaximumDistanceSeparableMatrix<F> 
where
    F: Field,
    X: IntoIterator,
    Y: IntoIterator,
{
    #[inline]
    fn sample<R>(distribution: CauchyMatrixDistribution, rng: &mut R) -> Self
    where
        R: CryptoRng + RngCore + ?Sized,
    {
        Self::new(distribution.to_matrix(|x, y| F::sub(x, y).inverse()))
    }
}

/// Cauchy Matrix Generator
#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq)]
pub struct CauchyMatrixGenerator<X, Y> {
    /// X<sub>i</sub> Injective Sequence
    pub xs: X,
    /// Y<sub>j</sub> Injective Sequence
    pub ys: Y,
}

impl<X, Y> CauchyMatrixGenerator<X, Y> {
    /// Generates the Cauchy matrix arising from the pairwise inverse
    /// differences of `self.xs` and `self.ys` using `inv_diff` to compute the
    /// inverse difference.
    #[inline]
    pub fn as_matrix<'s, T, F>(&'s self, mut inv_diff: F) -> Option<Vec<Vec<T>>>
    where
        &'s X: IntoIterator,
        &'s Y: IntoIterator,
        F: for<'t> FnMut(
            &'t <&'s X as IntoIterator>::Item,
            &'t <&'s Y as IntoIterator>::Item,
        ) -> Option<T>,
    {
        let ys = self.ys.into_iter().collect::<Vec<_>>();
        self.xs
            .into_iter()
            .map(|x| ys.iter().map(|y| inv_diff(&x, y)).collect())
            .collect()
    }

    /// Generates the Cauchy matrix arising from the pairwise inverse
    /// differences of `self.xs` and `self.ys` using `inv_diff` to compute the
    /// inverse difference.
    #[inline]
    pub fn to_matrix<T, F>(self, mut inv_diff: F) -> Option<Vec<Vec<T>>>
    where
        X: IntoIterator,
        Y: IntoIterator,
        F: for<'t> FnMut(&'t X::Item, &'t Y::Item) -> Option<T>,
    {
        let ys = self.ys.into_iter().collect::<Vec<_>>();
        self.xs
            .into_iter()
            .map(|x| ys.iter().map(|y| inv_diff(&x, y)).collect())
            .collect()
    }
}

Poseidon Hasher

/// Poseidon Hasher Sampling Distribution
#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq)]
pub struct HasherDistribution<A, M> {
    /// Additive Round Constants Distribution
    pub additive_round_constants: A,
    /// Maximum Distance Separable Matrix Distribution
    pub maximum_distance_separable_matrix: M,
}

impl<A, M, S> Sample<HasherDistribution<A,M>> for Hasher<S> 
where
    S: Specification,
    AdditiveRoundConstants<S::ParameterField>: Sample<A>,
    MaximumDistanceSeparableMatrix<S::ParameterField>: Sample<M>,
{
    #[inline]
    fn sample<R>(distribution: HasherDistribution<A, M>, rng: &mut R) -> Self
    where
        R: CryptoRng + RngCore + ?Sized,
    {
        Self::new(
            rng.sample(distribution.additive_round_constants),
            rng.sample(distribution.maximum_distance_separable_matrix)
        )
    }
}

Linear Feedback Shift Register PRNG

/// Linear Feedback Shift Register Pseudo-Random Number Generator
pub struct LinearFeedbackShiftRegister { ... }

impl RngCore for LinearFeedbackShiftRegister { ... }

impl SeedableRng for LinearFeedbackShiftRegister { ... }

and in the Poseidon module:

/// Generates the canonical LFSR for Poseidon constructed for the given [`Specification`].
#[inline]
pub fn canonical_sampling_rng<S>() -> LinearFeedbackShiftRegister 
where
    S: Specification,
    S::ParamterField: BinarySize,
{
    todo!("generate the canonical LFSR based on the specification")
}

where BinarySize is a trait that tells us how many bits the field is made of (trait name not final).

Rejection Sampling Utilities

NB: The following must be defined in manta_crypto::rand.

/// Returns the first value from `sample` applied to `rng` that returns `Some`.
#[inline]
pub fn find_sample<R, T, F>(rng: &mut R, mut sample: F) -> T
where
    F: FnMut(&mut R) -> Option<T>,
{
    loop {
        if let Some(value) = sample(rng) {
            return value;
        }
    }
}

/// Rejection Sampling Distribution
/// 
/// This distribution uses [`find_sample`] to convert a [`TrySample`] sampling algorithm into
/// a [`Sample`] one by performing rejection sampling.
#[derive(Clone, Copy, Debug, Default, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub struct RejectionSampling<D = ()>(pub D);

impl<D, T> Sample<RejectionSampling<D>> for T 
where
    D: Clone,
    T: TrySample<D>,
{
    #[inline]
    fn sample<R>(distribution: RejectionSampling<D>, rng: &mut R) -> Self {
        find_sample(rng, |rng| Self::try_sample(distribution.0.clone(), rng).ok())
    }
}

To plug these into the Poseidon parameter generation trait we just need to add the TrySample<()> implementation to the arkworks::Fp field object then just call the standard parameter generation as:

let hasher = canonical_sampling_rng::<S>().sample(
    HasherDistribution {
        additive_round_constants: RejectionSampling,
        maximum_distance_separable_matrix: CauchyMatrixGenerator { xs: (0..S::WIDTH), ys: (S::WIDTH..2*S::WIDTH) }
    }
);

Fix Arkworks Derive Semantics

For a lot of the arkworks types in manta-pay, the default derive implementations have the wrong semantics with respect to generic parameters. Like this:

#[derive(Clone)]
struct ArkworksType<T> 
where
    T: Trait,
{
    element: T::Element,
}

In this case, the derive macro adds a T: Clone assumption which is incorrect. Even sometimes the derivative::Derivative macro doesn't save you from this so we have to manually write adapters for some arkworks types.

Add circuit measurements to `measure` binary

In manta-pay we have a measure binary that prints out sizes of individual circuit components. We also should add the entire circuits to this measure binary to get a full profile.

Implement Merkle Tree Pruning

The Partial and ForkedTree implementations can implement leftward proof removal to reduce storage costs for Merkle Trees (Full and SinglePath cannot implement this by their nature). See the existing notes and TODOs in the manta_crypto::merkle_tree module for more. We should also implement a better testing suite for the Merkle Tree library to improve future development of this module.

  • Implement remove_path for Partial
    #[inline]
    fn remove_path(&mut self, index: usize) -> bool {
    // TODO: Implement this optimization.
    let _ = index;
    false
    }
  • Implement push and push_provable for Partial
    #[inline]
    fn maybe_push_provable_digest<F>(
    &mut self,
    parameters: &Parameters<C>,
    leaf_digest: F,
    ) -> Option<bool>
    where
    F: FnOnce() -> Option<LeafDigest<C>>,
    {
    self.maybe_push_digest(parameters, leaf_digest)
    }
    #[inline]
    fn maybe_push_digest<F>(&mut self, parameters: &Parameters<C>, leaf_digest: F) -> Option<bool>
    where
    F: FnOnce() -> Option<LeafDigest<C>>,
    {
    self.maybe_push_digest(parameters, leaf_digest)
    }

Use Pre-computed Powers in Scalar Multiplication

Right now we use a slow kind of scalar multiplication in-circuit where we compute powers of the base while multiplying. We can pre-compute these powers before entering the circuit and save on a lot of constraints. We can leverage the PrecomputedScalarMul trait and the other traits in manta_crypto::ecc to do this.

Since this is a parameter change, we need to make sure to upgrade the SDK whenever we add this in.

CI: Support Wasm backend and `nostd` targets

Add CI for the following two targets:

  • aarch64-unknown-none (or other equivalent) to check if nostd is properly supported
  • wasm32-unknown-unknown (or other equivalent) to check if wasm is properly supported

Fix HTTP Semantics

Right now we are adding a request body to the sign method on the signer HTTP clients/servers, but the sign method is marked as GET which shouldn't have a semantically meaningful body. Instead, we should be using query strings or mark sign as POST (unclear if this is semantically correct).

Migration Plan

To migrate over to manta-rs we will need to reorganize the dependency structure of many repositories. We also want to use this migration to clean up stale issues and resurface important ones to their relevant projects where they can be resolved properly. The following are a list of repositories and issues which are relevant for this migration:

Repositories

Deprecated (should also archive, private archive, or delete these):

Update Dependencies:

Discuss the Status of these Repositories:

Issues

Intertwine with Specification

We should find ways to link to the spec inside the code, especially for the manta-accounting library where the spec-related definitions exist.

Remove `manta_util::codec` in favor of Serde

We had to use this as a temporary measure for implementing uniform serialization across all the data structures and encoding formats in the library. Right now we have to deal with the following three kinds of serialization:

We would like to merge all three into just serde by writing adapters to replace the manta_util::codec adapters featured in manta-pay. We can begin by addressing the serialization of circuit parameters and find a way to make that work with just serde and ark-serialize (since SCALE compatibility is not needed for this). For the SCALE-based encodings, we can try and leverage the serde-scale crate.

Because this may require writing some serde Serializer and Deserializer implementations, we should find a way to make a more general version of the implementation in manta_accounting::fs::serde where we can have optional encryption and optional file-IO. If we implement that, we should move it into manta_util and just use a specific form of it in manta_accounting::fs.

Add PLONK Proof System Backend

We need the PLONK backend for mainnet. To finish the backend we need at least the following:

  • Add ConstraintSystem trait implementation for the ZK-Garage PLONK StandardComposer
  • Add ProofSystem implementation which implements the proving and verifying algorithms from ZK-Garage PLONK
  • Add ZCash trusted setup parameters to https://github.com/Manta-Network/sdk (see internal tracking for the progress on this)

Add more operation implementations on constraint systems

Add more + implementations so we don't need to clone or project the 0th component. We can also implement conditional_add, conditional_sub, etc methods on ConstraintSystem:

fn conditional_add<'v, V>(&mut self, bit: &Self::Bool, lhs: &'v V, rhs: &'v V) -> V 
where
    V: ConditionalSelect<Self>,
    &'v V: Add<Self>,
{
    let sum = self.add(lhs, rhs);
    self.conditional_select(bit, sum, lhs)
}

Originally posted by @bhgomes in #34 (comment)

Parallelize ZKP Generation

We can probably decrease user wait times for batched transactions by a lot if we parallelize proof generation. Probably a waste of time to do this until manta-rs signer is done.

Use Fastest Poseidon Implementation

We have a few different Poseidon implementations in the Manta ecosystem and we want to use the fastest one in production. Here are some goals for merging the implementation from Plonk-Prototype:

  • Upgrade posiedon file to directory in manta-pay crate
  • Keep the same interface that is defined in the current poseidon module, just upgrade the implementation (should not need to change any other files in manta-pay after the upgrade)
  • Add tests for consistency with known Poseidon implementations
  • Add in the parameter generation logic into the posiedon module and upgrade the parameters in the bin/generate_parameters.rs file (also remove the Sample implementation since it is not correct)
  • Add a PR to manta-sdk for new parameters

NB: For this PR we only care about the R1CS implementation, not PLONK.

Define Development Garbage Collection Cycle

To ensure healthy and clean development, we should schedule garbage collection of the repository for the code itself and for development tools like documentation, issues, pull-requests, CI pipelines, etc. These cycles should occur with some frequency, not too long, or else it becomes ineffective or difficult to perform garbage collection, but also not too short or else we slow down development and increase the noise-to-signal ratio. Whatever cycle length we choose could also be hinted/enforced by some GitHub action that polls developers to do the garbage collection.

Build Uniform Interface for WebSockets and GET/POST HTTP

Right now we have a less-than-ideal interface for HTTP clients where we assume that all communication should use the POST command so that we have access to the body of the request when sending arbitrary data. WebSockets is friendlier since doesn't have a GET/POST distinction. We want to have an interface that treats the communication channels that we use in manta-rs more in the style of WebSockets since we aren't creating a web service that should respect full HTTP semantics. In either case, we want a uniform interface of this form:

trait Channel {
    type Error;

    fn request<T, R>(
        &mut self,
        method: Method,
        command: &'static str,
        request: &T
    ) -> Result<R, Self::Error>
    where
        T: Serialize,
        R: DeserializeOwned;
}

where Method is GET or POST (or something similar) which does the right thing in HTTP and WebSockets.

Fast Recovery

After reviewing the design offline, we can use something like the following schema:

struct Note {
    ephemeral_public_key: PublicKey,
    ciphertext: Ciphertext,
    tag: u8,
}

fn create_note(ephemeral_secret_key: SecretKey, public_view_key: &PublicKey, asset: Asset) -> Note {
    let ephemeral_public_key = derive(&ephemeral_secret_key);
    Note {
        ciphertext: encrypt(agree(&ephemeral_secret_key, public_view_key), asset),
        tag: HASH(public_view_key, &ephemeral_public_key),
        ephemeral_public_key,
    }
}

fn try_decrypt(note: &Note, secret_view_key: &SecretKey, public_view_key: &PublicKey) -> Option<Asset> {
    if note.tag == HASH(public_view_key, &note.ephemeral_public_key) {
        decrypt(agree(secret_view_key, &note.ephemeral_public_key), &note.ciphertext)
    } else {
        None
    }
}

where HASH : PublicKey -> PublicKey -> u8 is some hash function that is faster than decryption.

Originally posted by @bhgomes in Manta-Network/spec#4 (comment)

Enforce Stricter Actions Ownership Policy

Right now the audit action is owned by a non-verified creator, so we can’t use a stricter filter in the Actions Settings. We should vendor our own auditing action, or if it’s simple enough, just write it inline in the .github directory.

Trusted Setup Tracking Issue

We want to migrate https://github.com/Manta-Network/trusted-setup into manta-crypto and eventually OpenZL, but for now, since it's still in experimental stages, we should only add it to manta-pay. To run ceremonies, we need to give access to the ConstraintSynthesizer (we're only supporting an arkworks backend for now) for phase2 parameters. We should also build the ceremony infrastructure to integrate into manta-signer and our CLIs.

Issues

Module Documentation Tracking Issue

We need to add documentation for each of these modules. To check off one of these boxes, add a link to the PR that fills in the documentation. If one of these boxes should be checked off without a PR that fills it in, leave a comment below explaining why.

  • manta-util
    • array
    • bytes
    • cache
    • codec
    • convert
    • iter
    • num
    • persistence
    • pointer
    • sealed
  • manta-crypto
    • accumulator
    • commitment
    • constraint
    • ecc
    • encryption
    • hash
    • key
    • merkle_tree
      • forest
      • fork
      • full
      • inner_tree
      • node
      • partial
      • path
      • single_path
      • test
      • tree
    • rand
  • manta-accounting
    • asset
    • fs
      • serde
    • key
    • transfer
      • batch
      • canonical
      • test
    • wallet
      • ledger
      • signer
      • state
      • test
        • sim
  • manta-pay
    • bin
      • generate_parameters
      • simulation
    • config
    • crypto
      • constraint
        • arkworks
          • codec
          • groth16
          • pairing
        • plonk
      • ecc
        • arkworks
      • encryption
      • hash
        • poseidon
      • key
    • key
    • signer
      • base
      • client
        • http
        • websocket
    • simulation
    • test
      • transfer

Investigate Parallel APIs for Cryptographic Primitives

In the future, we will need to run certain cryptographic algorithms in a multi-threaded context to get the optimal performance. One issue is the incompatibility of asynchronous and blocking APIs and the fact that multi-threaded algorithms usually make choices about which thread pool and parallelism model they are using. Running multi-threaded code in the browser is challenging and requires custom thread pools, so it would be good to define some APIs that are abstracted over the thread pool or parallelism model in general so that we can run similar code in WebWorkers and in native threads.

Example of JS WebWorkers Thread Pool: https://github.com/rustwasm/wasm-bindgen/blob/main/examples/raytrace-parallel/src/pool.rs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.