Git Product home page Git Product logo

numpile's Introduction

Numpile

A tiny 1000 line LLVM-based numeric specializer for scientific Python code.

You really shouldn't use this for anything serious, it's just to demonstrate how you might build one of these things from scratch. There's a lot of untapped potential and low hanging fruit around selective embedded JIT specialization for array expression languages in the SciPython space.

Installing

Numpile requires numpy and llvmlite (the latter includes needed LLVM libraries). You can either try to install them using your OS package manager, or alternatively, using pip:

$ pip install llvmlite
$ pip install numpy

Usage

import numpy as np
from numpile import autojit


@autojit
def dot(a, b):
    c = 0
    n = a.shape[0]
    for i in range(n):
        c += a[i] * b[i]
    return c


a = np.arange(100, 200, dtype='int32')
b = np.arange(300, 400, dtype='int32')
result = dot(a, b) 
print(result)

License

Released under the MIT License. Copyright (c) 2015, Stephen Diehl

numpile's People

Contributors

cdeil avatar mattclarke avatar pfalcon avatar sdiehl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

numpile's Issues

Calls to llvm.core `alloca` incorrectly assume second positional argument is `name`.

In [2]: from numpile import autojit

In [3]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:@autojit
:def dot(a, b):
:    c = 0
:    n = a.shape[0]
:    for i in range(n):
:       c += a[i]*b[i]
:    return c
:--

In [4]: %cpaste
Pasting code; enter '--' alone on the line to stop or use Ctrl-D.
:a = np.array(range(1000,2000), dtype='int32')
:b = np.array(range(3000,4000), dtype='int32')
:
:print dot(a,b)
:--
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-c2f171281c61> in <module>()
      2 b = np.array(range(3000,4000), dtype='int32')
      3 
----> 4 print dot(a,b)

/Users/ely_spears/scratch/python/numpile/numpile.py in _wrapper(*args)
    956                 return function_cache[key](*args)
    957             else:
--> 958                 llfunc = codegen(ast, specializer, retty, argtys)
    959                 pyfunc = wrap_module(argtys, llfunc)
    960                 function_cache[key] = pyfunc

/Users/ely_spears/scratch/python/numpile/numpile.py in codegen(ast, specializer, retty, argtys)
    976 def codegen(ast, specializer, retty, argtys):
    977     cgen = LLVMEmitter(specializer, retty, argtys)
--> 978     mod = cgen.visit(ast)
    979     cgen.function.verify()
    980 

/Users/ely_spears/scratch/python/numpile/numpile.py in visit(self, node)
    795         name = "visit_%s" % type(node).__name__
    796         if hasattr(self, name):
--> 797             return getattr(self, name)(node)
    798         else:
    799             return self.generic_visit(node)

/Users/ely_spears/scratch/python/numpile/numpile.py in visit_Fun(self, node)
    686         # Setup the register for return type.
    687         if rettype is not void_type:
--> 688             self.locals['retval'] = self.builder.alloca(rettype, "retval")
    689 
    690         map(self.visit, node.body)

/Users/ely_spears/anaconda/envs/basic27/lib/python2.7/site-packages/llvm/core.pyc in alloca(self, ty, size, name)
   2323 
   2324     def alloca(self, ty, size=None, name=""):
-> 2325         sizeptr = size._ptr if size else None
   2326         return _make_value(self._ptr.CreateAlloca(ty._ptr, sizeptr, name))
   2327 

AttributeError: 'str' object has no attribute '_ptr'

Here is the conda info dump for this environment:

(basic27)ely_spears@macbook:~/scratch/python/numpile$ conda list -e
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-64
astroid=1.2.1=py27_0
dateutil=2.1=py27_2
flask=0.10.1=py27_1
freetype=2.4.10=1
ipython=2.0.0=py27_0
itsdangerous=0.24=py27_0
jinja2=2.7.3=py27_1
libpng=1.5.13=1
llvm=3.3=0
llvmpy=0.12.7=py27_0
logilab-common=0.62.1=py27_1
markupsafe=0.23=py27_0
matplotlib=1.4.2=np19py27_0
mock=1.0.1=py27_0
nose=1.3.4=py27_0
numpy=1.9.1=py27_0
openssl=1.0.1k=0
pandas=0.14.1=np19py27_0
pip=1.5.6=py27_0
pylint=1.3.1=py27_0
pyodbc=3.0.7=py27_0
pyparsing=2.0.1=py27_0
python=2.7.9=1
python.app=1.2=py27_3
pytz=2014.9=py27_0
readline=6.2=2
scikit-learn=0.15.2=np19py27_0
scipy=0.14.0=np19py27_0
seaborn=0.5.0=np19py27_0
setuptools=7.0=py27_0
six=1.8.0=py27_0
sqlite=3.8.4.1=0
tk=8.5.15=0
werkzeug=0.9.6=py27_1
zlib=1.2.8=0

Out of date

I'm super super fascinated to explore this, and try out the @autojit decorator, but llvmpy has been deprecated in favour of llvmlite.

I'm installing on OS X but via brew+pip instead of Anaconda just because I have been building the rest of my software on this machine against those versions and I'm in too deep to switch to Anaconda just to experiment.

Here's how I installed python:

$ brew install python

# add the suggested line to ~/.bash_profile
---------------------------------------------------------------
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
----------------------------------------------------------------

Trying to install llvmpy fails for me with

ip-192-168-2-145:src kousu$ pip install llvmpy
Collecting llvmpy
  Using cached https://files.pythonhosted.org/packages/97/0f/98b78dc8a5ec032e05179fc406622e550d5c571f036beca8f06a4752f648/llvmpy-0.12.7.tar.gz
    Complete output from command python setup.py egg_info:
    LLVM version = '6.0.1'
    LLVM targets =
    Generate intrinsic IDs
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/sy/qyp_73v51_757c_h339_h9xw0000gn/T/pip-install-_1kua0qb/llvmpy/setup.py", line 83, in <module>
        auto_intrinsic_gen(incdir)
      File "/private/var/folders/sy/qyp_73v51_757c_h339_h9xw0000gn/T/pip-install-_1kua0qb/llvmpy/setup.py", line 76, in auto_intrinsic_gen
        intrgen.gen(path, fout)
      File "/private/var/folders/sy/qyp_73v51_757c_h339_h9xw0000gn/T/pip-install-_1kua0qb/llvmpy/tools/intrgen.py", line 13, in gen
        for line in open(f):
    FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/Cellar/llvm@6/6.0.1/include/llvm/Intrinsics.gen'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/sy/qyp_73v51_757c_h339_h9xw0000gn/T/pip-install-_1kua0qb/llvmpy/

I'll try to patch numpile to run against llvmlite. First, for that, I need to do:

$ brew install llvm@6  # llvmlite is only up to date with last-year's llvm
  ~/.bash_profile
----------------------------------------------------------------
export LDFLAGS="-L/usr/local/opt/llvm@6/lib -Wl,-rpath,/usr/local/opt/llvm@6/lib"
export PATH="/usr/local/opt/llvm@6/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/llvm@6/lib"
export CPPFLAGS="-I/usr/local/opt/llvm@6/include"
----------------------------------------------------------------

$ source ~/.bash_profile

$ pip --version
pip 18.0 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)
$ pip install numpy
$ pip install llvmlite

... TODO ...

List of numpile forks

For information of people who may be interested in doing further work on numpile, already known forks (in the order of appearance):

Both of these forks are problematic from copyright/licensing point of view - they are clearly based on numpile, and include substantial portions of it, but violate the following clause from numpile's license:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.