Test instability On Windows builds about spiceypy HOT 4 CLOSED

ok now I saw this in travis:

piceypy/tests/test_spiceerrors.py::test_emptyKernelPoolException
  /Users/travis/.pyenv/versions/3.4.0/lib/python3.4/site-packages/numpy/ctypeslib.py:435: RuntimeWarning: Item size computed from the PEP 3118 buffer format string does not match the actual item size.
    return array(obj, copy=False)
-- Docs: http://doc.pytest.org/en/latest/warnings.html

EDIT:
this is apparently a known bug in python?
https://stackoverflow.com/questions/4964101/pep-3118-warning-when-using-ctypes-array-as-numpy-array

from spiceypy.

not a python bug; bad input.

from spk14b.f:

  INTEGER               NSTATE
  PARAMETER           ( NSTATE = 6 )

  NCOEFF = CHBDEG + 1
  PKTSIZ = NSTATE * NCOEFF + 2

In our test case, the Chebyshev Polynomial degree is

  CHBDEG = 2

i.e. the last argument to spice.spk14b

  spice.spk14b(handle, "SAMPLE_SPK_TYPE_14_SEGMENT", 399, 0, "J2000", 100.0, 400.0, 2)

then

  NCOEFF = (2+1) = 3

and

  PKTSIZ = (6 * 3) + 2 = 20

And since we are passing in ncsets=4 epochs, I am pretty sure that means the coeffs array should have eighty (80) values, but it only has thirty-six (36).

from spiceypy.

@drbitboy ok this is a bit confusing as the tspice (naif test code) confirms your point, but I am wondering about why:

1. the tests don't catch this allways
2. the naif docs for spk14a say coeffs should be a 1d array, despite the test code passing in a 2d array. Maybe this is ok in c-land, I am a bit rusty but will try code something to check this

from spiceypy.

Not confusing if we think like a CPU (or a compiler).

1. spice.spk14a_c starts reading from wherever it is told by the ctypes-generated coeffs pointer, but it reads past the end of the coeffs array we provide because ncsets and chbdeg tell spk14a_c that the array is bigger than the coeffs we gave to ctypes. If the memory after the ctypes-allocated coeffs array is readable, no matter what non-coefficient garbage is there, then spk14a_c reads it and there is no memory error. If that memory is not readable, then spk14b_c throws the memory exception. What ends up in that memory after the passed coeffs array is a function of whatever time, other memory allocations, garbage collection, operating system state, etc., are current at the time of the call, so it makes sense that the result is inconsistent between runs.

1.1) Put another way: we told ctypes that coeffs is 36 doubles (288 contiguous bytes); we told spk14b_c/spk14a_c that coeffs is 80 doubles (640 contiguous bytes). For all we know, ctypes/python/C allocates memory in 1024-byte chunks, so even if spk14a_c reads past byte 288, it may only read the otherwise unused memory that was allocated along, and contiguous, with the 288 bytes ctypes knows about.

2. The ctypes-generated coeffs pointer is the only memory address passed, whether the memory after that is set up as a 2d or 1d array in the compiler does not matter: it's a block of memory and only the [ncsets] (spk14a_c) and [2+(chbdeg+1)*6] (spk14b_c) values determine how that memory is interpreted. I can all but guarantee that, for the cases where we are passing the short array but it works anyway by reading past the coeffs memory, if we did not remove the test14.bsp file, the I could parse that file and find all of those values that we did pass, plus the garbage values, just at the wrong locations after the first record.

The C language: all the power of assembler, with all the convenience of assembler.

from spiceypy.