Git Product home page Git Product logo

svdlibc's Introduction

Doug Rohde's SVD C library

Good news: as of version 1.4, SVDLIBC is explicitly available under a BSD license.

SVDLIBC is a fast implementation of SVD matrix decomposition by Doug Rohde. It works particularly efficiently in the following cases:

  • the matrix is sparse,
  • only a few singular values are needed.

These properties make it particularly well suited for latent semantic analysis, for example.

I ran an experiment on an Amazon EC2 m2.xlarge instance - which might be way overkill - with a 70k ร— 500k matrix containing 8M entries (density: 0.02%). Here is the running time for different values of d (= dimensions = number of singular values):

  • -d 50: 39s wall time
  • -d 300: 4m53s wall time
  • -d 1000: 31m48s wall time

Why this fork ?

The latest official release of the library (version 1.34) dates back from 2005. It has a few quirks, such as:

  • make / make install don't work "as expected"
  • it doesn't compile on Mac OS X out of the box
  • some bugs have been found, e.g. by piskvorky

A caveat

I'm not a release engineer, and have only limited knowledge of the different languages (C, Makefile) and tools (make, gcc) involved. The modifications in this fork are working for me, but nothing guarantees they'll work for you.

If you find a bug and fix it yourself, I'd be happy to get a pull your changes over.

Installation instructions

Easy as pie:

# Download the code. Alternatively you can also download the zip file.
git clone git://github.com/lucasmaystre/svdlibc.git
cd svdlibc

# Just like any other sane program...
make
make install

# You're done. Start using it!
svd -o result -d 10 THE_MATRIX

svdlibc's People

Contributors

lucasmaystre avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

svdlibc's Issues

Sparse matrices bring NaN computation and infinite-loop

The latest, as well as v1.34 calculates NaN values at a time when the input matrices have lesser than 1% density of non-zero values. I generated lots of matrices with size of bigger than 1000x1000 (for a start) and 0.05% density and nearly all of them brought this error. The problem first shows up in the svd_pythag method, where the computation goes into an infinite loop because of the NaN values.

Smaller matrices brought weird values instead of the infinite loop.

The algorithm to generate the test matrices is simple: using a random generator to ensure the given density value and generating a matrix with random [1-9] values where the random lets us.

I also state here that I used the dense matrix format, but even if I format it to sparse, things did not change (maybe because the algorithm already converts it to sparse when it starts?)

Tests were done one a Ubuntu machine with gcc and Intel Pentium Dual Core CPU, as well as on a Windows 7 machine with Visual Studio 2010 and a 6 core AMD Phenom CPU.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.