Git Product home page Git Product logo

cdiego's Introduction

Implementation of Consensus-DIstributEd Generalized Oja Algorithm (C-DIEGO)

This project contains the implementation of C-DIEGO algorithm in a file algorithms.py.
The algorithm depends on multiple inputs, including designing of weight matrix using Erdős–Rényi approach, generation of Synthetic data etc.

We explain the process of each function below and implementation in pca_data_functions.py.

Population Covariance Matrix Generation

Details of generation of population covariance matrix $\mathbf{\Sigma}$ with custom eigengap $\Lambda$ are as follows:

List of Experiments

1. Performance Effect of different values of consensus rounds.

This experiment is performed by 1.effect_of_diff_Tc_synthetic_data_mp.py by generating synthetic data for three different values of consensus rounds $T_c$.

  1. $T_c = T_{mix}$
  2. $T_c = \log(Nt)$
  3. $T_c = T_{mix} \frac{3}{2} \log(Nt)$. which we define to be optimal value of consensus rounds.

Expected Outcome: The experiment show that for optimal value of $T_c$, there is no gap between the output by DIEGO (which assumes a fully connected network) and that of output by C-DIEGO.
The same experiment is also performed using MNIST dataset is provided in 1a.effect_of_diff_Tc_mnist_data_mp.py
Both experiments using Synthetic and Real dataset are averaged over 50 monte-carlo trials running on multiple cores.

2. Convergence rate effect on different values of $\Lambda$ (eigengap)

This experiment uses three different values of eigengap $\Lambda$ to see the effect on convergence. We generate three covariance matrices with three different eigengaps and run the experiment under FC graph and NFC graph. The topology of NFC graph remains fixed for all three cases of eigengap.
Expected Outcome: The effect of optimal Tc remains independent of eigengap. The larger the eigengap the better convergence.
The experiment is performed by 2.effect_of_diff_eiggap_synthetic_data_mp.py and uses synthetic data.

3. Convergence rate effect on different values of N (the number of nodes)

This experiment uses three different values of $N$ and see the effect on convergence rate under FC network.
Expected Outcome: The higher the value of $N$, the better the convergence to corroborate theoretical results.
The experiment is performed by 3.effect_of_diff_N_synthetic_data_mp.py and uses synthetic data.

4. Convergence rate effect under different topologies

This experiment see the effect of C-DIEGO FC vs NFC in three different topologies:

  1. Ring Graph
  2. Erdos-Reyni (FC) with one dropped node
  3. Line Graph
    We use (topologies_gen.py)[topologies_gen.py] file to generate Line and Ring graph and their weight matrices.
    (Experiment in-progress)

cdiego's People

Contributors

intarian avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.