Git Product home page Git Product logo

lp-sparsemap's Introduction

wheels PyPI version

LP-SparseMAP

Differentiable sparse structured prediction in coarse factor graphs

This repo contains:

  • ad3qp: an updated fork of ad3, supporting the solving of SparseMAP QPs in arbitrary factor graphs. (C++, LGPL license.)

  • dysparsemap: a library that provides a dynet function using ad3qp for forward and backward pass computation for structured hidden layers. (C++, MIT license.)

  • lpsmap: a python wrapper for ad3qp and some example usage scripts. (cython and python, MIT license.) This repository is a work-in-progress, with the end-goal to drastically simplify the AD3 API.

Reference

Vlad Niculae and Andre F. T. Martins. LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction. https://arxiv.org/abs/2001.04437

lpsmap

Requirements:

  • Cython
  • Eigen (if it's a non-standard directory, set EIGEN_DIR=/path/to/eigen.)

For examples and tests: numpy, pytest.

Installation:

pip install lp-sparsemap    # installs a wheel, if available.

In-place installation from source:

# export MACOS_DEPLOYMENT_TARGET=10.14  # on MacOS
export EIGEN_DIR=/path/to/eigen
python setup.py build_clib  # builds ad3 in-place
pip install -e .            # builds lpsmap and creates a link

Using the Cython API from your own code.

You can add custom factors and other extensions by cimporting the base classes provided. (See an example in this project.) The installed lp-sparsemap package provides a copy of libad3 to statically link against. To get the path to it, use lpsmap.config.get_libdir(). Warning: both lp-sparsemap as well as client libraries linking against it should be compiled with the same standard library implementation. On MacOS you may have issues unless MACOS_DEPLOYMENT_TARGET >= 10.14. If you get undefined symbol errors for AD3 symbols, try compiling your code with the same toolchain as the installed lp-sparsemap. (If in doubt, recompile both locally.)

dysparsemap

Requires this patch to dynet in order to make dynet export cmake targets to link against. (sorry, I'm new to cmake and haven't managed to test it and make a PR yet.)

Once the patched dynet is installed, do

cd cbuild
cmake ..
make

Then you can try the dynet gradient check tests that get compiled.

lp-sparsemap's People

Contributors

goncalomcorreia avatar tsvm avatar vene avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

lp-sparsemap's Issues

compatibility with Eigen 3.4

Eigen's default version is in C++14, hence installing LP-SparseMAP with it raises compiler errors.

Suggested change: line 16 of 'setup.py' to '-std=c++14',

Trouble installing custom version of DyNet with Boost

Hi Vlad, thank you for the exciting resource. Looking forward to trying it out!

I am trying to install all of the dependencies. I am having trouble getting DyNet installed using the DyNet fork.

I have followed the instructions to install DyNet for C++ here: https://dynet.readthedocs.io/en/latest/install.html#building

Where I have diverged from the official instructions is instead of cloning the official DyNet repo:
git clone https://github.com/clab/dynet.git

I am cloning your custom version:
git clone https://github.com/vene/dynet.git.

I am then switching to the py3 branch: git checkout py3.

I am then running the cmake command as normal:

(base) jbarry@jbarry-desktop:~/dynet-base3/dynet/build$ cmake .. -DEIGEN3_INCLUDE_DIR=/home/jbarry/dynet-base3/eigen -DENABLE_CPP_EXAMPLES=ON
-- Boost dir is Boost_INCLUDE_DIR-NOTFOUND
CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find Boost (missing: Boost_INCLUDE_DIR program_options regex
  serialization)
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.16/Modules/FindBoost.cmake:2179 (find_package_handle_standard_args)
  CMakeLists.txt:103 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/jbarry/dynet-base3/dynet/build/CMakeFiles/CMakeOutput.log".

I then tried downloading Boost v 1.62 which seemed to be the most recent version of Boost at the time of the Dynet fork around November 2016 and tried again using the instructions for compiling with Boost:

(lp-sparsemap-pt-dy-custom) jbarry@jbarry-desktop:~/dynet-base3/dynet/build$ cmake .. -DEIGEN3_INCLUDE_DIR=/home/jbarry/dynet-base3/eigen -DENABLE_CPP_EXAMPLES=ON -DENABLE_BOOST=ON
-- Boost dir is /home/jbarry/dynet-base3/boost
CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find Boost (missing: program_options regex serialization) (found
  version "1.62.0")
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.16/Modules/FindBoost.cmake:2179 (find_package_handle_standard_args)
  CMakeLists.txt:103 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/jbarry/dynet-base3/dynet/build/CMakeFiles/CMakeOutput.log".

I am probably doing something completely wrong when compiling with Boost. Apologies as I am quite inexperienced with using cmake commands and editing CMake options. Here is how my DyNet base directory looks:

(lp-sparsemap-pt-dy-custom) jbarry@jbarry-desktop:~/dynet-base3$ ls
boost dynet  eigen

ls dynet/
build  cmake  CMakeLists.txt  config.h.cmake  doc  dynet  examples  external  LICENSE  MEM.notes  pyexamples  python  python_tests  README.md  rnnlm  tests  tutorial

I'm just wondering if you have any pointers to install this correctly? I was able to successfully install the official DyNet code in a separate environment without Boost. Attached is the CMakeFiles/CMakeOutput.log if it helps. Many thanks!
CMakeOutput.log

No module named 'lpsmap.ad3ext.parse'

Dear Vlad,

Thanks for your amazing work.

I am trying to install lpsmap and having trouble with an error when I use it.

from lpsmap.ad3ext.parse import Parse
ModuleNotFoundError: No module named 'lpsmap.ad3ext.parse'

I first download the eigen as indicated in https://gitlab.com/libeigen/eigen and then perform the following scripts
export EIGEN_DIR=/path/to/eigen
python setup.py build_clib # builds ad3 in-place
pip install . # builds lpsmap and installs

My env is:
Unbuntu16.04
Python: 3.6
Pytorch: 1.6.0

I am not sure if the eigen should be compiled firstly.

Would you please give some suggestions?

Thanks a lot!

Unable to install lp-sparsemap

I am facing an issue while trying to install the lp-sparsemap package. The installation process is failing with the following error:

I have attempted to upgrade the pip package using the command pip install --upgrade pip, but the issue persists.

Screenshot 2024-02-29 012147
Screenshot 2024-02-29 012222

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.