Git Product home page Git Product logo

spacepy / spacepy Goto Github PK

View Code? Open in Web Editor NEW
223.0 223.0 68.0 90.23 MB

Space Science library for Python - contains superposed epoch classes, drift shell tracing, access to magnetic field models, streamline tracing, bootstrap confidence limits, time and coordinate conversions, etc.

Home Page: https://spacepy.github.io

License: Other

Python 34.73% C 0.67% Fortran 60.47% HTML 3.88% Batchfile 0.07% Shell 0.17%
batsrus cdf coordinates python python3 space swmf

spacepy's Introduction

DOI Build Status

SpacePy

SpacePy is a package for Python, targeted at the space sciences, that aims to make basic data analysis, modeling and visualization easier. It builds on the capabilities of the well-known NumPy and MatPlotLib packages. Publication quality output direct from analyses is emphasized among other goals:

  • Quickly obtain data
  • Read (and write) data from (and to) data formats like NASA CDF and HDF5
  • Create publications quality plots
  • Perform complicated analysis easily
  • Run common empirical models
  • Change coordinates and time systems effortlessly
  • Harness the power of Python

The SpacePy project seeks to promote accurate and open research standards by providing an open environment for code development. In the space physics community there has long been a significant reliance on proprietary languages that restrict free transfer of data and reproducibility of results. By providing a comprehensive, open-source library of widely-used analysis and visualization tools in a free, modern and intuitive language, we hope that this reliance will be diminished.

To help foster an open and welcoming environment, we have adopted a code of conduct that we encourage members of the SpacePy community to read and follow.

Getting SpacePy

Our latest release version is available through PyPI and can be installed using

pip install spacepy --user

This will also automatically install most dependencies.

The latest "bleeding-edge" source code is available from our github repository at https://github.com/spacepy/spacepy.

Further installation documentation, including building from source and OS-specific information, can be found here. Full documentation is at https://spacepy.github.io.

SpacePy supports Python 3.6 and later.

Dependencies

SpacePy has a number of well-maintained dependencies which are automatically installed by pip. These include:

  • numpy (>=1.15.1)
  • dateutil (>=2.5)
  • scipy (>=1.0)
  • matplotlib (>=3.1)
  • h5py (>=2.10)

Attribution

When publishing research which used SpacePy, please provide appropriate credit to the SpacePy team via citation or acknowledgement.

To cite SpacePy in publications, please cite both the code (DOI: 10.5281/zenodo.3252523) and the papers describing the package (BibTeX code):

@article{niehof2022spacepy,
  title={The SpacePy space science package at 12 years},
  author={Niehof, Jonathan T and Morley, Steven K and Welling, Daniel T and Larsen, Brian A},
  journal={Frontiers in Astronomy and Space Sciences},
  volume={9},
  year={2022},
  doi={10.3389/fspas.2022.1023612},
  publisher={Frontiers}
}

and/or

@INPROCEEDINGS{spacepy11,
author = {{Morley}, S.~K. and {Koller}, J. and {Welling}, D.~T. and {Larsen}, B.~A. and {Henderson}, M.~G. and {Niehof}, J.~T.},
title = "{Spacepy - A Python-based library of tools for the space sciences}",
booktitle = "{Proceedings of the 9th Python in science conference (SciPy 2010)}",
year = 2011,
address = {Austin, TX}
}

For additional information, see the CITATION.cff file. Certain modules may provide additional citations in the __citation__ attribute. Contact a module's author before publication or public presentation of analysis performed by that module. This allows the author to validate the analysis and receive appropriate credit for his or her work.

For acknowledging SpacePy, please provide the URL to our github repository: github.com/spacepy/spacepy.

Changes

Changes in the released version of SpacePy are provided in the release notes. For changes since the latest release, see the repository version.

spacepy's People

Contributors

aaronhendry avatar antoinebrunet avatar asher-m avatar asreimer avatar astricklan avatar ayshih avatar balarsen avatar dpeterk avatar drsteve avatar jamesw732 avatar jgieseler avatar jhaiduce avatar jnlanl avatar jtniehof avatar lrastaet avatar mghenderson64 avatar pheosics avatar scivision avatar spacecataz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spacepy's Issues

dmarray breaks on masking [Migrated from SF; Bugs #65 and #71]

Combining two trackers from SourceForge:
#65 dmarrays do not properly convert to masked arrays

#71 dmarray array behavior difference in np.median

Basic summary of these two bug trackers is that dmarray is a subclass of numpy ndarray. A straightforward subclass of ndarray (e.g. following their RealisticInfoArray example) is that ufuncs like max, min, etc. behave inconsistently between numpy arrays and subclasses of arrays. The fix that was committed for this pushed the error further into hiding and manifested as a breakage on conversion to a MaskedArray. One major impact was that dmarrays needed to be converted to ndarrays before being passed to matplotlib's pcolormesh, but the issue was basically anywhere that masking was needed.

pip install fails to resolve dependencies under python 3 (PyPI) or when installing from git

As discussed in #38, installing in a clean environment using pip only works under Python 2, installing from PyPI. It fails under Python 3, and under both 2 and 3 when pointing to a git repo, e.g.,

pip install git+https://github.com/spacepy/spacepy

To reproduce

~/.local/bin/virtualenv -p python3 py3-clean
cd py3-clean/
source bin/activate
pip3 install spacepy

or

~/.local/bin/virtualenv py2-clean
cd py2-clean/
source bin/activate
pip install git+https://github.com/spacepy/spacepy

This should yield

Collecting git+https://github.com/spacepy/spacepy
  Cloning https://github.com/spacepy/spacepy to /tmp/pip-req-build-FI27ko
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-FI27ko/setup.py", line 24, in <module>
        from numpy.distutils.core import setup
    ImportError: No module named numpy.distutils.core
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-FI27ko/

It seems likely that the issue is that we import numpy at the top of the setup.py which means that an ImportError is raised before we even get to running the setup function. It's unclear why it works under pip2 with PyPI as the source, but it does...

Test failure: test_irbempy.IRBEMBigTests.test_getLstar_T05

Error on 64-bit Windows, python 2.7, in getUTC, line 1162 of spacepy time (so this may be a time bug); ultimately "cannot convert float NaN to integer".

Works on 32-bit Windows, python 2.7; 32-bit Windows, Python 3.6; and 64-bit Windows, Python 3.6.

List capabilities/routines

For the python in heliophysics community, we should have a list of our capabilities (finer-grained than just the module list, not necessarily a list of every function.) That might also help users find stuff.

Document required packages and versions

We need to document the requirements better and capture the versions of each that are required. I propose that the best way to do this is a document page describing the flow and creation of requirements.txt with minimum versions for each package.

README.md needs to be generated

The repo should have a README.md that can give folks a softer landing when they hit the github page.

What should go in it?

  • Short summary of what SpacePy provides
  • Basic install instructions (and where the harder stuff is described)
  • Link to documentation
  • ?

Transition offical repo from sourceforge to github

@spacepy/core Team, @spacecataz

Please weigh in with your comments on this issue tracker.

I'm looking to change the status of this repo from our unofficial mirror to the official repo.

Pros

  • We can set up continuous integration with Travis (on multiple versions of Python) so we actually see the test failures
  • GitHub has better visibility and I'd like the official repo to be the one people find
  • Zenodo has good github integration and our releases could be auto-archived and assigned DOIs so they are citeable
  • We can use github pages to provide a new homepage, since the old one is defunct
  • It looks like PyPI actually tracks stars and forks of github repos when the project URL points to GitHub, and I like that feature!

Cons

  • One or more of the core developers would need greater permissions for this group[1]
  • We'd want all the devs to switch to using the github.
  • We'd need to work out what to do with the repo over at SourceForge[2]
  • Are there really more cons?

I'd also like to then enable a 2 person review for each pull request and have everyone fork the repo and send pull requests when they have updates. That way test failures happen outside the master branch.

Finally, for this thread anyway, I'd like people to start thinking about what needs to go into issue trackers here. We have all sorts of gaps in documentation, there are outstanding items that probably don't have issue trackers, ... even if you're working on something, it probably warrants a tracker. We should also start tagging some of these as "beginner friendly" to encourage user participation. If it looks like we can sensibly change to using github as our primary then I'd like to do it ASAP. There's a paper on Python in Heliophysics in preparation and SpacePy is one of the packages under discussion. I'd like to be able to at least tag a current version and archive it at Zenodo before submission (estimated in less than 2 weeks) so the code can be properly cited.

Steve

[1]: @graingert Can you make all of the core team owners on the group? That'd be really helpful.
[2]: SourceForge isn't the pit of malware it used to be - it got bought out a couple years back and is legit again. So we probably can actually delete the project if we want to, or we can look at automated syncing.

Nightly builds of CI

#31 aside, we should also catch when an updated version of a dependency breaks stuff. So can we have pinned versions on the pull request builds but also a nightly build that runs against master and the latest dependencies (and maybe even automatically opens a issue?)

datetime64 support in pycdf

It came up in the helio python conference that some people are trying to use datetime64 ndarrays with pycdf and it's of course failing horribly. We should support that (at least on write, read will be trickier). Leapseconds are probably going to be a bit of an issue.

pip install stalls for python 2.7

I was using spacepy back in July with no problem and installed it via pip. I recently ran into a problem with my anaconda install and decided to do a fresh install. Now installing with pip to a python 2.7 distribution doesn't work (it does work for python 3.6). It gets to the point:
Building wheels for collected packages: spacepy
Running setup.py bdist_wheel for spacepy ...
And then just sits there. Anyone have suggestions on this?

Jon

Set up notebook repository

We talked at one point about integrating Jupyter notebooks into the repo/docs somehow. We should do that :) Figure out where to host/how to reference, convert existing case studies and get them into the repo and linked from the docs (and have the case studies also set to link back to the Sphinx.)

spatialaudio/nbsphinx#89 has some info on cross-referencing between Jupyter and Sphinx.

Related: do we want to stick with pythonhosted as our main docs repo or set up with github pages? Github pages isn't hard to set up and would get us one-stop shopping (no reason to eliminate the pythonhosted mirror, of course.)

When using spacepy.coordinates.Coords, getting 'ImportError: cannot import name irbempylib'

I have installed spacepy 0.1.5 using pip install spacepy and then done from spacepy import time as spt, coordinates as spc. I attempt to create a Coords object by writing my_gse_coords = spc.Coords(my_coords, 'GSE', 'car'), where my_coords is a numpy array of shape (n, 3). Python fails on this line and I get an error: ImportError: cannot import name irbempylib. The full traceback is below.

~/anaconda/lib/python2.7/site-packages/spacepy/coordinates.pyc in __init__(self, data, dtype, carsph, units, ticks)
     77     def __init__(self, data, dtype, carsph, units=None, ticks=None):
     78 
---> 79         from . import irbempy as op
     80         from spacepy.irbempy import SYSAXES_TYPES as typedict
     81 

~/anaconda/lib/python2.7/site-packages/spacepy/irbempy/__init__.py in <module>()
     68 
     69 
---> 70 from .irbempy import *
     71 

~/anaconda/lib/python2.7/site-packages/spacepy/irbempy/irbempy.py in <module>()
     21 import spacepy.coordinates as spc
     22 import spacepy.datamodel as dm
---> 23 from . import irbempylib as oplib
     24 import spacepy.toolbox as tb
     25 

ImportError: cannot import name irbempylib

Am I doing something wrong, or is this just a bug in spacepy?

CI support for Windows

AppVeyor will do github integration, supports Windows, and is free for OSS, so would be good to tie in CI to that.

pycdf reading empty NRV scalars

Consider a variable dimensioned (5, 3). Assume no data has been written. Everywhere in here, assume "read" means "index [...]", i.e. read everything. Reading explicit indices will obviously raise an IndexError.

If it's record-varying, there's an implicit first dimension. Reading it returns an array of shape (0, 5, 3) of the appropriate type.

If it's non-record-varying, there's no implicit first dimension. Reading it returns an array of shape (0,0) so the dimensionality is maintained even if the shape isn't. (Although this may not be the best idea!)

Now assume it's scalar.

RV is again easy, implicit first dimension, shape (0,).

NRV though is impossible. If I do numpy.empty(shape=()), I get a numpy scalar, which makes sense, but it has a value. (Often 0. as the default constructor, but not always, depends on exactly the way it's made.)

Ideally these reads should return something that acts as much as possible like the desired object but communicates "emptiness" somehow.

It doesn't help that I don't actually have a means of communicating whether an NRV has been written or not....the shape doesn't have a record dimension. I could re-add that dimension and then NRV behave just like RV but you're always dragging around this first dimension that might be 0 or might be 1 and never anything else. And it would be a MAJOR API change.

I could also say that reading NRV that has no data is such an abnormal conditoin that it should raise an error, or return None, or something. But sometimes you really are okay just getting back an empty array and processing it like everything else.

So, what should reading a scalar NRV with no data return?

Relevant to #16 but this is more of a refinement on it.

pybats testReadZip failure

Windows, Python2, 32-bit. Fails in readascii, line 191, gzip.open...the resulting mode is rtb, so ValueError: InvalidMode ('rtb')

fortran build fails on arm

python setup.py build --fcompiler=gnu95 yields error message "gfortran: error: unrecognized command line option โ€˜-m32โ€™" on raspbian. uname: Linux midori 4.14.69-v7+ #1141 SMP Mon Sep 10 15:26:29 BST 2018 armv7l GNU/Linux. This is the default compiler (and gnu doesn't work.)

Exception: Cannot load CDF C library from C:\CDF_Distribution\cdf37_0-dist\lib\dllcdf.dll.Try os.environ["CDF_LIB"] = library_directory before import.

(JTN copied from SF#103 posted by Shreya Singh)

This is the code i have written:

import os
os.environ['CDF_LIB'] = 'C:\CDF_Distribution\cdf37_0-dist\lib'
from spacepy import pycdf

I am using notebook python 3.6 on windows 10

This is the error i am getting:

Exception Traceback (most recent call last)
<ipython-input-23-1606178e5c8f> in <module>()
1 import os
2 os.environ['CDF_LIB'] = 'C:\CDF_Distribution\cdf37_0-dist\lib'
----> 3 from spacepy import pycdf</module></ipython-input-23-1606178e5c8f>

C:\Users\abc\AppData\Local\Continuum\Anaconda33\lib\site-packages\spacepy\pycdf__init__.py in <module>()
1035 
1036 try:
-> 1037 _libpath, _library = Library._find_lib()
1038 except:
1039 if 'sphinx' in sys.argv[0]:</module>

C:\Users\abc\AppData\Local\Continuum\Anaconda33\lib\site-packages\spacepy\pycdf__init__.py in _find_lib()
417 raise Exception(('Cannot load CDF C library from {0}.'
418 'Try os.environ["CDF_LIB"] = library_directory '
--> 419 'before import.').format(', '.join(failed)))
420 
421 @staticmethod

Exception: Cannot load CDF C library from C:\CDF_Distribution\cdf37_0-dist\lib\dllcdf.dll.Try os.environ["CDF_LIB"] = library_directory before import.

and under sub-directory lib i have these items:
i have attached screenshot

capture

PSD data URL is broken

The PSD data is still downloading from the old LANL website...we should figure out where that needs to go and put it there permanently (gh-pages?)

updating or making the repository official

Hello there, I've just met two of the spacepy developers, and they would like to get this repo either official, or update it - so the community is up to date if using this repository.
If it's ok with you, you could add some of us to the team and we do the updating.

Thanks

License.md is required

Currently the license is in the source code, and can be queried through spacepy.license, but the license text should be in a LICENSE.md.

Issue with pycdf.CDF.from_data, NRV?

This is the second issue, that after speaking with Jon, appears to be separate from the other issue I opened.

In [5]: pycdf.lib.set_backward(False)

In [8]: pycdf.CDF.from_data('test_cdf.cdf', d)
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-8-916e03bc5ee8> in <module>()
----> 1 pycdf.CDF.from_data('test_cdf.cdf', d)

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in from_data(cls, filename, sd)
   1767         with cls(filename, '') as cdffile:
   1768             for k in sd:
-> 1769                 cdffile[k] = sd[k]
   1770             cdffile.attrs.clone(sd.attrs)
   1771

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in __setitem__(self, name, data)
   1597                 self[name].attrs.clone(data.attrs)
   1598         else:
-> 1599             self.new(name, data)
   1600
   1601     def __iter__(self, current = 0):

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in new(self, name, data, type, recVary, dimVarys, dims, n_elements, compress, compress_param)
   2090                 n_elements = 1
   2091         else:
-> 2092             (guess_dims, guess_types, guess_elements) = _Hyperslice.types(data)
   2093             if dims is None:
   2094                 if recVary:

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in types(data, backward)
   3589         types = []
   3590
-> 3591         _Hyperslice.check_well_formed(d)
   3592         if d.dtype.kind in ('S', 'U'): #it's a string
   3593             types = [const.CDF_CHAR, const.CDF_UCHAR]

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in check_well_formed(data)
   3523         if d.dtype == numpy.object: #this is probably going to be bad
   3524             try:
-> 3525                 len(d.flat[0])
   3526             except TypeError: #at least it's not a list
   3527                 pass

IndexError: index 0 is out of bounds for axis 0 with size 0

Jon mentioned this likely has to do with confusing RV and NRV things.

spacepy plot style deprecated in matplotlib 2.2.3

from spacepy import plot as spp

Bad key "axes.color_cycle" on line 17 in
/Users/balarsen/Library/Python/3.6/lib/python/site-packages/spacepy/data/spacepy.mplstyle.
You probably need to get an updated matplotlibrc file from
http://github.com/matplotlib/matplotlib/blob/master/matplotlibrc.template
or from the matplotlib source distribution

So the rc has changed with matplotlib.

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated

New warning that appeared in python 3.7.0

This one is in time.py

/Users/balarsen/.local/lib/python3.7/site-packages/spacepy/time.py:228: DeprecationWarning: 
Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, 
and in 3.8 it will stop working
  if not isinstance(dtype, collections.Callable):

The offending code is collections.Callable. This should instead be:

import collections.abc
collections.abc.Callable

Possible that pycdf will have the issue as well. There are a lot of collections.MutableSequence, collections.MutableMapping, etc in there.

Python 2.7 CI

CI hangs on 2.7 with @drsteve 's latest trials. This might be related to #24 since both seem to be failures in fortan build. But we should ideally have travis running on 2.7, at least for now.

Updating gcc 7 to gcc 8 with homebrew breaks coordinates module

I get an error when trying to use a Coords object in Mojave, having just updated. Running the following two blocks of code:

from spacepy import coordinates
coordinates.Coords(np.random.random((100,3)), np.random.random(100), 'car')

Gives the following traceback:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-8-f437b8d7fe1e> in <module>()
----> 1 coordinates.Coords(np.random.random((100,3)), np.random.random(100), 'car')

/anaconda2/envs/python3/lib/python3.6/site-packages/spacepy/coordinates.py in __init__(self, data, dtype, carsph, units, ticks)
     75     def __init__(self, data, dtype, carsph, units=None, ticks=None):
     76 
---> 77         from . import irbempy as op
     78         from spacepy.irbempy import SYSAXES_TYPES as typedict
     79 

/anaconda2/envs/python3/lib/python3.6/site-packages/spacepy/irbempy/__init__.py in <module>()
     69 
     70 
---> 71 from .irbempy import *
     72 

/anaconda2/envs/python3/lib/python3.6/site-packages/spacepy/irbempy/irbempy.py in <module>()
     26 import spacepy.datamodel as dm
     27 try:
---> 28     from . import irbempylib as oplib
     29 except ImportError:
     30     if 'sphinx' in sys.argv[0]:

ImportError: dlopen(/anaconda2/envs/python3/lib/python3.6/site-packages/spacepy/irbempy/irbempylib.cpython-36m-darwin.so, 2): Library not loaded: /usr/local/opt/gcc/lib/gcc/7/libgfortran.4.dylib
  Referenced from: /anaconda2/envs/python3/lib/python3.6/site-packages/spacepy/irbempy/irbempylib.cpython-36m-darwin.so
  Reason: image not found

I tried brew install gcc as this looks like a gfortran issue to me, and I got:

(python3) home-one:~ john$ brew install gcc
Warning: gcc 8.2.0 is already installed and up-to-date
To reinstall 8.2.0, run `brew reinstall gcc`

I tried pip uninstall spacepy and pip install spacepy and that did not help either.

Does anyone have any ideas?

Can't control metadata order in JSON headed ASCII

It would be nice to be able to order the JSON metadata when writing an ASCII file. You can specify the order of the data in toJSONheadedASCII, but writeJSONmetadata uses the same "order" of vars, and the metadata aren't vars so there's no way I can find to order the header.

CI needed, python 2 and python 3

We should get CI setup, all the cool kids are doing it.

What versions?
I use a bit of python2.7 and a lot of 3.6 (3.7 is also just out).

Are we still supporting 2.6 I forget?

Need requirements.txt for pip

We need a requirements.txt file for pip. We currently cannot take a bare python install and do a pip install spacepy even can't do pip install https://github.com/spacepy/spacepy

This file is a good way to document version requirements and could ever interface with the CI having several requirements.txt files in the repo and calling them in the CI rather than a hard code.

Comments and defaults in spacepy.rc

Right now, if spacepy.rc doesn't exist, we populate it with the default values, but leave them commented out, with a further comment that they're the default:

#foo = bar #default value in spacepy 0.0.0

Unfortunately configparser doesn't (by default) support comments at the end of line, only on a line by themselves. So if a user uncomments:

foo = bar #default value in spacepy 0.0.0

then config option "foo" will have the value "bar #default value in spacepy 0.0.0". Which really makes a mess with, for instance, the URLs for downloading leapsecond data.

I'm thinking the best way to address this is:

#Default from spacepy 0.0.0
#foo = bar

although that does double the length of the rc. Note that we can't just put one "defaults from spacepy 0.0.0" at the top, since if spacepy.rc exists but a particular value doesn't, that line will be added.

Problems with plot.utils.EventClicker

I'm using SpacePy with Python 3.6.4 (Anaconda 64-bit). When attempting to use EventClicker from plot.utils, I receive the following error:

File "/Users/jwoodroffe/anaconda/lib/python3.6/site-packages/spacepy/plot/utils.py", line 358, in _add_event_phase
ls=self._styles[self._curr_phase / len(self._colors) % len(self._styles)])
TypeError: list indices must be integers or slices, not float

This is due to a change in how division is handled between Python 2.x and 3.x. When fixed by changing "/" to "//", another error occurs nearby:

File "/Users/jwoodroffe/anaconda/lib/python3.6/site-packages/spacepy/plot/utils.py", line 359, in _add_event_phase
if not self._xydata is None:
AttributeError: 'EventClicker' object has no attribute '_xydata'

If I fix this by initializing _xydata to None in init, I get:

File "/Users/jwoodroffe/anaconda/lib/python3.6/site-packages/spacepy/plot/utils.py", line 394, in _add_event_phase
self._data_events.resize((self._data_events.shape[0] + 1,
AttributeError: 'NoneType' object has no attribute 'resize'

And so on. This particular code may be in need of some TLC.

BAD_ENTRY_NUM when reading CDF file

New bug report filed on the old Sourceforge tracker.
https://sourceforge.net/p/spacepy/bugs/102/

Submitted text (by Baptiste Cecconi) follows:
I'm running a pipeline that produces a series of CDF files, and then read them again one after the other to extract global attributes and write archive labels for each file. Every file is open and closed one after the other in a for loop. Once in a while, I get the following error:

File "/Users/baptiste/Development/pyvenv/py35/lib/python3.5/site-packages/spacepy/pycdf/init.py", line 517, in check_status
raise CDFError(status)
spacepy.pycdf.CDFError: BAD_ENTRY_NUM: Illegal attribute entry number specified.

Usually I just re-run the processing again and it works, but this not very satisfactory. I see that this bug seems to be already detected and solved:

Commit [d043ae]
pycdf: fix occasional BAD_ENTRY_NUM when looking up zEntry type

Any idea of why this re-appeared ?

Crash in test_pycdf

Python 2.7, 32-bit Windows, in the middle of test_pycdf crashes with a popup "python.exe has stopped working". python test_pycdf.py NoCDF fails. Running with any one test (e.g. python test_pycdf.py NoCDF.testTypeGuessing) does not fail. Running python test_pycdf.py --verbose NoCDF and it ends with:

testTypeGuessing (__main__.NoCDF)
Guess CDF types based on input data ...

so looks like the crash is in there but is preconditioned by some other test. They appear to be running alphabetically and TT2000ToEpoch16 is the one right before.

Pinned dependencies on CI

Continuous integration builds on pull requests shouldn't fail just because a dependency has been updated, so we should probably pin versions there.

Repeated "not supported" message when running test_irbem on Windows

When I'm running from master on Windows, I get this repeated "not supported" message, as if spacepy were being reloaded or something. This might actually be a reload or there might be something odd going on in the import machinery. I'll look into it but wanted to log as an issue in case I forget...haven't even verified if it happens on other OSs.

conda build

Ultimately we want people to be able to do "conda install spacepy" and have it just work. Notes/comments here towards that.

datetime_to_tt2000 fails on ARM raspberry Pi

From Rodrigue Piberne:

Round-tripping a datetime through datetime_to_tt2000 and tt2000_to_datetime results in an invalid time; this has been tracked down to:
pycdf._library.CDF_TT2000_from_UTC_parts(2018.0,9.0,3.0,15.0,58.0,7.0,992.0,823.0,0.0)
returning an invalid TT2000.

I suspect some sort of bug with data types not matching (maybe a ctypes issue) and have sent over some code to test.

Document pull request workflow

We've been running this a little informally (which is a lot more formal than before!) but we should probably document how we're handling PRs. Stuff like CI build should work, merge should be done by somebody other than the person submitting the PR, and I suggest we leave PRs open for 24 hours minimum so people can comment.

I don't know if we want to start this in a wiki or stick to Sphinx.

Issue with pycdf.CDF.from_data

After trying to copy a CDF using CDF.copy and CDF.from_data, I get:

In [1]: from spacepy import pycdf
This unreleased version of SpacePy is not supported by the SpacePy team.

In [2]: c = pycdf.CDF('psp_isois-epihi_l1-49f_00000000_v0.0.0.cdf')

In [3]: d = c.copy()

In [4]: pycdf.CDF.from_data('test_cdf.cdf', d)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-916e03bc5ee8> in <module>()
----> 1 pycdf.CDF.from_data('test_cdf.cdf', d)

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in from_data(cls, filename, sd)
   1767         with cls(filename, '') as cdffile:
   1768             for k in sd:
-> 1769                 cdffile[k] = sd[k]
   1770             cdffile.attrs.clone(sd.attrs)
   1771

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in __setitem__(self, name, data)
   1597                 self[name].attrs.clone(data.attrs)
   1598         else:
-> 1599             self.new(name, data)
   1600
   1601     def __iter__(self, current = 0):

/usr/local/lib/python2.7/dist-packages/spacepy/pycdf/__init__.pyc in new(self, name, data, type, recVary, dimVarys, dims, n_elements, compress, compress_param)
   2119                     const.CDF_TIME_TT2000.value) \
   2120                 and self.backward:
-> 2121             raise ValueError('Data requires EPOCH16, INT8, or TIME_TT2000; '
   2122                              'incompatible with backward-compatible CDF')
   2123         new_var = Var(self, name, type, n_elements, dims, recVary, dimVarys)

ValueError: Data requires EPOCH16, INT8, or TIME_TT2000; incompatible with backward-compatible CDF

After retrying with pycdf.lib.set_backward(False), a different error is returned, though this appears to be a separate issue.

Jon asked me to throw this bug in here, but I'm not entirely certain what details to include. I can elaborate if/when necessary.

Remove Python 2 support

(Before anybody panics: the title of this issue is to indicate we'll close it when Python 2 support is removed, not that this is an immediate goal!)

Python 2.7 will be unsupported sometime in 2020. All our dependencies have pledged to EOL Python 2 support no later than that ( https://python3statement.org/ ). I suggest a gradual phase-out:

  1. No prebuilt Python 2.7 packages: We should deliver one for 0.1.7. But potentially not as soon as the next release (presumably 0.1.8). June 2019, perhaps?
  2. Allow new features/functions that don't support Python 2.7 as long as they don't break existing functionality. Beginning of 2020?
  3. No workarounds for versions of dependencies that don't support Python 3; I'd lump the date in with 2.
  4. No fixes for bugs that cannot be reproduced on Python 3. I would take this as the latter of June 2020 and when numpy, scipy, matplotlib have all released versions that do not support Python 2, with a caveat of no later than fall AGU of 2020.
  5. Actively remove existing Python 2 code from the codebase: January 1, 2021.

This would mean a committment to full support of Python 2.7 through June 2019 (or whatever date is chosen for 1), followed by a phaseout of 12-18 months, while still falling within the 2020 pledge.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.