libdynd / dynd-python Goto Github PK

View Code? Open in Web Editor NEW

118.0 39.0 23.0 5.5 MB

Python exposure of dynd

Home Page: http://libdynd.org

License: Other

Shell 0.96% CMake 7.39% Python 31.13% C++ 45.80% Makefile 0.02% Batchfile 0.65% Cython 14.04%

dynd-python's Introduction

DyND-Python

TravisCI: AppVeyor:

DyND-Python, a component of the Blaze project, is the Python exposure of the DyND dynamic multi-dimensional array library.

To discuss the development of this library, subscribe to the LibDyND Development List.

Python versions 2.7, 3.4 and 3.5 are supported.

http://libdynd.org

https://github.com/libdynd/libdynd

https://github.com/libdynd/dynd-python

Trying Out DyND

The easiest way to try it out is through the Anaconda Python distribution. The latest release of Anaconda includes a version of DyND.

http://continuum.io/downloads

For trying the latest updates, there is also an automated build configured which tracks the latest git master. When all the tests pass, it uploads conda packages to the anaconda.org channel "dynd/channel/dev". To get these versions, you can run the following command.

conda install dynd-python --channel dynd/channel/dev

It may work best to install development versions of DyND into an environment instead of the main Anaconda directory. You can do this with a command like:

C:\>conda create -n dynd-env python=3.3 dynd-python --channel dynd/channel/dev

Developing DyND

See the build and install instructions for details on building the software for environments not supported by Anaconda, or if you would like to modify or contribute to the project.

dynd-python's People

Contributors

Stargazers

Watchers

dynd-python's Issues

Some nd.array construction inconsistencies

In [3]: nd.array(5, '{x: string, y: int32}')
Out[3]: nd.array(["5", 5], type="{x : string, y : int32}")

In [4]: nd.array(5, '2 * int32')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-ea5c9514e819> in <module>()
----> 1 nd.array(5, '2 * int32')

C:\Users\mwiebe\Anaconda\lib\site-packages\dynd\_pydynd.pyd in _pydynd.w_array.__cinit__ (_pydynd.cxx:6510)()

TypeError: 'int' object is not iterable

data_size in struct types should provide the actual size, not zero

This shows the problem:

In [7]: t = ndt.make_struct([ndt.int32, ndt.float64], ['x', 'y'])

In [8]: t
Out[8]: ndt.type('struct<int32 x, float64 y>')

In [9]: t.data_size
Out[9]: 0

Support datashapes containing types int, float, ...

Does datashape accept types without a bitsize? If so then DyND is inconsistent here.

Types like int and float instead of int32, float64 are really convenient for non-expert users.

In [4]: nd.array(1, dtype='int')
RuntimeError: Error parsing datashape at line 1, column 1

In [5]: nd.array(1, dtype='int32')
Out[5]: nd.array(1, type="int32")

Addsupport for ndt.float16

I know that this is a weird dtype that needs additional logic, but it is mainly a reminder for supporting it in the future.

Unable to submodule update

The repository does not appear to exist.

[~/Projects]λ git clone https://github.com/ContinuumIO/blazeprototype.git
Cloning into 'blazeprototype'...
remote: Counting objects: 483, done.
remote: Compressing objects: 100% (302/302), done.
remote: Total 483 (delta 283), reused 325 (delta 140)
Receiving objects: 100% (483/483), 134.86 KiB, done.
Resolving deltas: 100% (283/283), done.
[~/P/blazeprototype]λ git submodule init
Submodule 'libraries/dynamicndarray' ([email protected]:ContinuumIO/dynamicndarray.git) registered for path 'libraries/dynamicndarray'
[~/P/blazeprototype]λ git submodule update
Cloning into 'libraries/dynamicndarray'...
ERROR: Repository not found.
fatal: The remote end hung up unexpectedly
Clone of '[email protected]:ContinuumIO/dynamicndarray.git' into submodule path 'libraries/dynamicndarray' failed

nd.as_numpy should work over strings too

This is an example of the problem:

In [301]: a = nd.array([('k1', 'v1'), ('k2', 'v2')], ndt.type('{key: string; val: string}'))

In [302]: a.key
Out[302]: nd.array(["k1", "k2"], strided_dim<string>)

In [303]: nd.as_numpy(a.key)  # does not work for fields
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-303-582bb138de29> in <module>()
----> 1 nd.as_numpy(a.key)

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.as_numpy (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:9374)()

RuntimeError: dynd as_numpy could not convert dynd type string to a numpy dtype

In [304]: nd.as_numpy(a)  # and neither for the struct array
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-304-a702f58c9e1d> in <module>()
----> 1 nd.as_numpy(a)

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.as_numpy (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:9374)()

RuntimeError: dynd as_numpy could not convert dynd type string to a numpy dtype

I think dynd should choose the 'U4' dtype for doing this kind of transformation.

A tolist() would be handy for JSON serialization

JSON allows only a small set of data containers to be serialized, and lists are specially handy for serializing arrays. It would be nice if dynd could support the same functionality as numpy:

In [33]: np.int32(3).tolist()
Out[33]: 3

In [34]: np.array(3).tolist()
Out[34]: 3

In [35]: np.array([3]).tolist()
Out[35]: [3]

In [36]: np.array([3,3]).tolist()
Out[36]: [3, 3]

In [38]: np.array([[3,3], [3,3]]).tolist()
Out[38]: [[3, 3], [3, 3]]

bytes and str should be recognized as a type in constructors

For example, int and float are supported:

In []: ndt.make_struct([int, float, float], ['x', 'y', 'z'])
Out[]: ndt.type('struct<int32 x, float64 y, float64 z>')

but not str or bytes:

In []: ndt.make_struct([int, str, bytes], ['x', 'y', 'z'])
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-101-d33c3c9952aa> in <module>()
----> 1 ndt.make_struct([int, float, bytes], ['x', 'y', 'z'])

/Users/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.make_struct (/private/var/jenkins/workspace/DyND-Python/PLATFORM/osxbuild/PYTHON_VERSION/2.7/build/_pydynd.cxx:5533)()

RuntimeError: could not convert the given Python TypeObject into a dynd type

I think both str and bytes should be nice additions.

nd.zeros() does not work properly with struct types

This shows the problem:

In [17]: nd.zeros(10, '{x : float32; y : float64}')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-17-3f0fe7b78824> in <module>()
----> 1 nd.zeros(10, '{x : float32; y : float64}')

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.zeros (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:9569)()

RuntimeError: Cannot assign from int32 to cstruct<float32 x, float64 y>

Not being able to convert into a numpy dtype should be raised as a TypeError, not RuntimeError

For example:

In []: t3
Out[]: ndt.string

In []: t3.as_numpy()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-51-72a869b30454> in <module>()
----> 1 t3.as_numpy()

/Users/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_type.as_numpy (/private/var/jenkins/workspace/DyND-Python/PLATFORM/osxbuild/PYTHON_VERSION/2.7/build/_pydynd.cxx:2981)()

RuntimeError: cannot convert dynd type string into a Numpy dtype

I think a TypeError is more significant here.

Cannot create a dynd array from an empty list and a dtype

This shows the issue:

In []: nd.array([], dtype='float64')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-22-2e3a724f4378> in <module>()
----> 1 nd.array([], dtype='float64')

/Users/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__cinit__ (/private/var/jenkins/workspace/DyND-Python/PLATFORM/osxbuild/PYTHON_VERSION/2.7/build/_pydynd.cxx:6907)()

TypeError: Cannot get the leading dimension size of dynd array with type float64

although this works just fine:

In []: nd.array([1], dtype='float64')
Out[]: nd.array([1], type="strided * float64")

and this too:

In []: nd.array([], type='strided * float64')
Out[]: nd.array([], type="strided * float64")

lowlevel access to important metadata should follow numpy conventions

Now (dynd 0.4.2), for getting the pointer of the data area of a dynd array with Cython, one must do:

    data = <char *><Py_uintptr_t>_lowlevel.data_address_of(array)

where as with numpy:

  data = array.ctypes.data

which is far easier. The same goes for shape and strides.

Also, adopting numpy convention for this low level access will ease a lot the migration of existing numpy extensions to dynd.

Implement a way to easiliy get the shape of a fixed type

A fixed dynd type type should have an easier way to get the shape out of it. I know that dshape exists:

In [407]: a = nd.empty('2, 2, int8')

In [408]: ft = nd.type_of(a)

In [409]: ft.dshape
Out[409]: '2, 2, int8'

but I would prefer something like ft.shape that returns the (2,2) tuple in this example.

asarray does not accept structured scalars

This shows the problem:

In [8]: r = np.array([(10, 11, 12)], dtype='i4,i8,f8')

In [9]: nd.asarray(r)
Out[9]: nd.array([[10, 11, 12]], strided_dim<{f0 : int32; f1 : unaligned(int64); f2 : unaligned(float64)}>)

In [10]: nd.asarray(r[0])
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-10-7388e9be24d6> in <module>()
----> 1 nd.asarray(r[0])

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.asarray (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:8877)()

RuntimeError: could not create a dynd array from the numpy scalar object

Segfault when assigning numpy zero-shaped arrays to dynd scalars

dynd should allow ingesting numpy zero-shaped arrays. This shows the problem:

In [4]: a = nd.empty(ndt.float64)

In [5]: a[()] = 3

In [6]: a
Out[6]: nd.array(3, float64)

In [9]: a[()] = np.array(3)
Violació de segment

copy() should be implemented

It should be a handy way to do a copy in dynd without recurring to some black magic like:

 x = x.cast(ndt.make_fixed_dim(count, c_dtype)).eval()

something like:

x = x.copy()

nd.empty() does not accept an `access` parameter

I know that in empty() this makes little sense but, just for consistency, the next should work:

In []: nd.zeros("3 * int32", access='rw')
Out[]: nd.array([0, 0, 0], type="3 * int32")

In []: nd.ones("3 * int32", access='rw')
Out[]: nd.array([1, 1, 1], type="3 * int32")

In []: nd.empty("3 * int32", access='rw')  # this does not work...
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-67f7dfc5109e> in <module>()
----> 1 nd.empty("3 * int32", access='rw')

TypeError: empty() got an unexpected keyword argument 'access'

Out of range slicing should be allowed

In many situations, allowing out-of-range indexing in slices is useful, and actually numpy does have support for this:

In [465]: npa = np.array([1])

In [466]: npa[1:]
Out[466]: array([], dtype=int64)

but dynd does not:

In [467]: nda = nd.array([1])

In [468]: nda[1:]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-468-d57b4ea9c636> in <module>()
----> 1 nda[1:]

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__getitem__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:7660)()

IndexError: index range [1:] is out of bounds for axis 0 in shape (var)

Improve consistency in the order of parameters for type constructors

ndt.make_fixed_dim() and ndt.make_strided_dim() have params that are in different order:

>>> ndt.make_fixed_dim(5, ndt.int32)
ndt.type('fixed_dim<5, int32>')

>>> ndt.make_strided_dim(ndt.int32, 3)
ndt.type('strided_dim<strided_dim<strided_dim<int32>>>')

As ndt.make_var_dim() also takes element_tp as an argument, I propose to change the order to ndt.make_fixed_dim() so that element_tp would be the first argument.

The string version of nd.array objects is broken

The next describes the issue:

In [445]: repr(nd.array([1,2,3]))
Out[445]: 'nd.array([1, 2, 3], strided_dim<int32>)'

In [446]: str(nd.array([1,2,3]))
---------------------------------------------------------------------------
BroadcastError                            Traceback (most recent call last)
<ipython-input-446-604acb86c515> in <module>()
----> 1 str(nd.array([1,2,3]))

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__str__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:7260)()

BroadcastError: cannot broadcast input datashape '3, int32' into datashape 'string'

len() for a scalar array should raise a more specific error

Right now the error is:

In [16]: sd = nd.array(1)

In [17]: len(sd)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-17-05d458eb1a9e> in <module>()
----> 1 len(sd)

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__len__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:9166)()

TypeError: Cannot get the leading dimension size of dynd array with type int32

I would rather throw something like Blaze does right now:

IndexError: Scalar dynd arrays have no length

although a ValueError could be even better?

Zero strides should be supported

A nice trick in NumPy to support arrays with constant values is to use a value of a stride 0, which saves a lot of space. It would be nice if DyND could get this support too.

Also currently this does not work:

In [110]: zs = np.ndarray(10, dtype="i2", buffer=np.array([4]), strides=(0,))

In [111]: zs
Out[111]: array([4, 4, 4, 4, 4, 4, 4, 4, 4, 4], dtype=int16)

In [112]: nd.array(zs)
Out[112]: nd.array([4, 4, 4, 4, 4, 4, 4, 4, 4, 4], strided_dim<int16>)

In [113]: zsnd = nd.array(zs)

In [114]: zsnd.strides
Out[114]: (2,)

In [115]: zs.strides
Out[115]: (0,)

Better way to specify shape and type in constructors

While coding using dynd, it will probably be pretty common having users wanting to build an array by specifying the shape (as a tuple) and the type at the same time, but the way dynd does this currently is a bit involved IMO:

Case 1:

>>> nd.empty('2, 2, int8')

in this case you need to create the format manually from shape and type

Case 2:

>>> nd.empty((2, 2), 'M, N, int16')

in this case, you can use the shape as a tuple, but still you need to add the place holders 'M, N,' manually in front of the type, which is again convenient.

I would suggest supporting something like:

>>> nd.empty((2, 2), 'int16')

i.e. being able to use the shape and type straight in the constructor.

provide a cython header for more efficient access from cython

For using it from cython, both for convenience and for speed, maybe extracting the function pointer from the lowlevel.dataaddress_of ctypes function into a cython C function pointer, and calling that instead of going through the python interface as this way does would work well.

Provide an equivalent of numpy.squeeze

Providing the functionality of numpy.squeeze() would be handy.

Implement the equivalent of assert_array_equal and assert_array_almost_equal in numpy

These utilities would be very handy for building test units.

Compound types should be indexable by field name, not only by position

It would be nice if this could work:

In []: t = ndt.type('{key: string; val: int32}')

In []: t['key']
Out[]: ndt.string

In []: t['val']
Out[]: ndt.int32

right now, one is required to do:

In []: t = ndt.type('{key: string; val: int32}')

In []: t[nd.as_py(t.field_names).index('key')]
Out[]: ndt.string

In []: t[nd.as_py(t.field_names).index('val')]
Out[]: ndt.int32

which is far more verbose (and non-trivial to figure out)

DynD Datetime can't convert to Numpy Datetime

In [62]: from datetime import date, datetime

In [63]: b = nd.array(datetime(2000, 1, 1,0,0,52))

In [64]: b
Out[64]: nd.array(2000-01-01T00:00:52, type="datetime")

In [65]: nd.as_numpy(b,allow_copy=True)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-65-f403ac0c6d10> in <module>()
----> 1 nd.as_numpy(b,allow_copy=True)

/Users/quasiben/Research/ContinuumDev/GitRepo/dynd-python/build/dynd/_pydynd.so in _pydynd.as_numpy (/Users/quasiben/Research/ContinuumDev/GitRepo/dynd-python/build/_pydynd.cxx:10951)()

TypeError: dynd as_numpy could not convert dynd type datetime to a numpy dtype

sum() does not work with other types than 'int32'

the sum() function does not work with dynd arrays that are not of type 'int32':

In [536]: sum(nd.array([1,2], 'int32'))
Out[536]: nd.array(3, expr<int32, op0=int32, op1=int32, expr=addition(op0, op1)>)

however:

In [537]: sum(nd.array([1,2], 'float64'))
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-537-5ab37d087c9e> in <module>()
----> 1 sum(nd.array([1,2], 'float64'))

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__add__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:7913)()

RuntimeError: A dynd pointer type's target cannot be the expression type convert<to=float64, from=int32>

In [538]: sum(nd.array([1,2], 'int8'))
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-538-e8f1db63fdbb> in <module>()
----> 1 sum(nd.array([1,2], 'int8'))

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__add__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:7913)()

RuntimeError: A dynd pointer type's target cannot be the expression type convert<to=int32, from=int8>

Strides becomes 0 for single-element arrays

In [460]: nd.array([1.,2.]).strides
Out[460]: (8L,)

In [461]: np.array([1.,2.]).strides
Out[461]: (8,)

In [462]: np.array([1]).strides
Out[462]: (8,)

# until here so far so good, but now:
In [463]: nd.array([1]).strides
Out[463]: (0L,)    # doh!

Scalars cannot be broadcasted into struct arrays

This does not work in dynd:

In [17]: nd.zeros(10, '{x : float32; y : float64}')
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-17-3f0fe7b78824> in <module>()
----> 1 nd.zeros(10, '{x : float32; y : float64}')

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.zeros (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:9569)()

RuntimeError: Cannot assign from int32 to cstruct<float32 x, float64 y>

but it works in numpy:

In [23]: npa = np.empty(10, 'f4,f8')

In [24]: npa[:] = 0

In [25]: npa
Out[25]: 
array([(0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0),
       (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)], 
      dtype=[('f0', '<f4'), ('f1', '<f8')])

it would be nice if dynd could do the same too.

scalars should evaluate immediately

Currently dynd does not evaluate scalars immediately, and that creates some problems like:

In [204]: a = nd.array([0,1,2])

In [205]: a[a[0]]    # works!
Out[205]: nd.array(0, int32)

In [206]: a[a[0]+1]   # doesn't!
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-206-f935c978338b> in <module>()
----> 1 a[a[0]+1]

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__getitem__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:7660)()

TypeError: dynd array must have kind 'int' or 'uint' to be used as an index

This is only one example, but one could find many more I guess.

repr() of a string dtype raises an error when doing a round trip

This shows the problem:

In []: nd.array(["v1", "v2"], ndt.string)
Out[]: nd.array(["v1", "v2"], strided_dim<string>)

In []: nd.array(["v1", "v2"], strided_dim<string>)
  File "<ipython-input-82-b141f8994dd2>", line 1
    nd.array(["v1", "v2"], strided_dim<string>)
                                              ^
SyntaxError: invalid syntax

Creating an array from iterator does not properly recognize the types

This works as expected:

In []: nd.array(np.arange(10))
Out[]: nd.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], strided_dim<int64>)

However, when trying to use an iterator, the inferred type is a float64 and not an int64:

In []: nd.array(iter(np.arange(10)))
Out[]: nd.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], var_dim<float64>)

Implement reshape for arrays

I am trying to do some reshaping in dynd arrays, but I cannot see a reshape() method, nor either nd.array(), nd.asarray() or nd.view() seems to fill the bill (but I might be blind :)

Unable to build: git submodule problem

Tried:

$ git submodule init
$ git submodule update
Cloning into libraries/dynamicndarray...
remote: Counting objects: 5145, done.
remote: Compressing objects: 100% (854/854), done.
remote: Total 5145 (delta 3767), reused 5126 (delta 3748)
Receiving objects: 100% (5145/5145), 1.10 MiB | 260 KiB/s, done.
Resolving deltas: 100% (3767/3767), done.
fatal: reference is not a tree: 71ae6345bf18a1c72cb51d9f92daff71a0612a32
Unable to checkout '71ae6345bf18a1c72cb51d9f92daff71a0612a32' in submodule path 'libraries/dynamicndarray'

What I am doing wrong?

Btw. note that the git submodule init and update should be done BEFORE first invocation of cmake, if not, then:

fatal: destination path 'libraries/dynamicndarray' already exists and is not an empty directory.

it will contain CMakeFiles dir

Update Cython version requirement in the BUILD_INSTALL file

Hi,

Compiling the Cython part of dynd, I've got an Cython syntax error about a const char * declaration.

I read that Since version 0.18, Cython has support for the const modifier in the language (from http://docs.cython.org/src/tutorial/strings.html#dealing-with-const) whereas the Cython version specified in the BUILD_INSTALL.md file is >=0.16.

Thanks,
Damien G.

Segfault when indexing empty nd.array objects

This shows the behavior:

In [1]: from dynd import nd, ndt

In [2]: c = nd.array(dtype='2,2, int32')

In [3]: c
Out[3]: nd.array()

In [4]: c[0]
Violació de segment

This is using dynd 0.4.2.post12.ga4dd27e

Add an equivalent of np.int_ for dynd

In some situations it is handy to use an np.int_ as the underlying int. dynd already support this:

In [314]: nd.array([1,2,3], dtype=np.int_)
Out[314]: nd.array([1, 2, 3], strided_dim<int64>)

However, there not an equivalent to it straight in dynd:

In [317]: ndt.int*?
ndt.int16
ndt.int32
ndt.int64
ndt.int8

CMake FindPythonLibNew does not cover Debian Python install

Hi,

I'm on a Debian testing. My Python lib is installed at /usr/lib/python2.7/config-x86_64-linux-gnu. The find_library does not work without setting the variable PYTHON_LIBRARY explicitly. I was surprised before understanding that my Python lib was not in a standard lib path such as /usr/lib.

Maybe you can use the Debian script python-config (see the option --configdir) to add an other path to the _PYTHON_LIBS_SEARCH variable in order to find the lib. It's not obvious for a new user that this CMake script does not find a such library.

Thanks,
Damien G.

nd.zeros and nd.ones should be implemented

numpy.zeros() and numpy.ones() are very handy when one wants to build large arrays with predefined values. They should be present in dynd too.

Also, perhaps a nd.fill(value, shape, element_tp) would be handy to create arrays with general values.

documentation: make a table of numpy idioms and corresponding dynd idioms

e.g. numpy arr.copy() => dynd nd.array(arr)

numpy cannot ingest dynd arrays with a stride

This describes the problem:

In [279]: a = nd.array(np.arange(10))

In [280]: np.array(a[:])
Out[280]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [281]: np.array(a[1:])
Out[281]: array([1, 2, 3, 4, 5, 6, 7, 8, 9])

In [282]: np.array(a[1:8])
Out[282]: array([1, 2, 3, 4, 5, 6, 7])

In [283]: np.array(a[1:8:3])    # this does not work...
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-283-7f1c10783b7e> in <module>()
----> 1 np.array(a[1:8:3])

ValueError: setting an array element with a sequence.

Add an efficient sum() mechanism to dynd

An efficient sum() mechanism is pretty useful for array-based code. Note that usually it is unavoidable to have a parallel machinery than the offered by Python itself. For example, notice how different the performance is for numpy:

In [24]: time x.sum()
CPU times: user 12 ms, sys: 0 ns, total: 12 ms                                                                                                           
Wall time: 11.2 ms                                                                                                                                       
Out[24]: 49999995000000.0                                                                                                                                

In [25]: time sum(x)
CPU times: user 2.6 s, sys: 0 ns, total: 2.6 s
Wall time: 2.6 s
Out[25]: 49999995000000.0

str/repr on large arrays should cap values

Currently, str/repr dump all the values of the array, but when this is large, it is really inconvenient (e.g. NumPy does offer an algorithm that caps the dump of the array when it is large).

Making a type out of a numpy struct type loops forever

This shows the problem:

In [53]: ndt.type(np.dtype('f4,f8'))    # loops forever

equality does not work among values in arrays

This describes the problem:

In [236]: a = nd.empty('4, int8')

In [237]: a                                                                                                          
Out[237]: nd.array([64, 0, 0, 0], fixed_dim<4, int8>)                                                                

In [238]: a[1]
Out[238]: nd.array(0, int8)

In [239]: a[1] == a[1]
Out[239]: False

Also:

In [243]: a[1] > a[1]
Out[243]: False

In [244]: a[0] > a[1]
Out[244]: False

An nd.fromiter() would be desirable, for a count= parameter

I know that the dynd array constructor does accept iterators, but sometimes it is important to accept just a number of elements out of them, and currently I cannot see a way to do that other than:

        #return np.fromiter(self.where(key), dtype=self._dtype, count=count)
        return nd.array(self.where(key), dtype=self._dtype)[:count]

but this case is not optimal because the constructor builds the complete array and then a view is returned.

str() of a struct array does not work

This illustrates the issue:

In [282]: t = ndt.type('{key: string; val: string}')

In [283]: a = nd.array([('k1', 'v1'), ('k2', 'v2')], t)

In [284]: print repr(a)   # this works!
nd.array([["k1", "v1"], ["k2", "v2"]], strided_dim<{key : string; val : string}>)

In [285]: print str(a)   # this does not
---------------------------------------------------------------------------
BroadcastError                            Traceback (most recent call last)
<ipython-input-285-9d7b17ad5387> in <module>()
----> 1 print a

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__str__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:7567)()

BroadcastError: cannot broadcast input datashape '2, {key: string; val: string}' into datashape 'string'

str() for arrays is broken

This shows the problem:

In [7]: a = np.arange(10)

In [8]: d = nd.array(a)

In [9]: d   # repr does work
Out[9]: nd.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], type="strided * int64")

IIn [10]: print(d)  # str does not
---------------------------------------------------------------------------
BroadcastError                            Traceback (most recent call last)
<ipython-input-11-9909575fff82> in <module>()
----> 1 print(d)

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__str__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:8687)()

BroadcastError: cannot broadcast input datashape '10 * int64' into datashape 'string'

/home/faltet/anaconda/lib/python2.7/site-packages/dynd/_pydynd.so in _pydynd.w_array.__str__ (/var/jenkins/workspace/DyND-Python/PLATFORM/CentOS5x64b/PYTHON_VERSION/2.7/build/_pydynd.cxx:8687)()

BroadcastError: cannot broadcast input datashape '10 * int64' into datashape 'string'

libdynd / dynd-python Goto Github PK

dynd-python's Introduction

DyND-Python

Trying Out DyND

Developing DyND

dynd-python's People

Contributors

Stargazers

Watchers

Forkers

dynd-python's Issues

Recommend Projects

Recommend Topics

Recommend Org