shapelets / khiva Goto Github PK

Describe the bug

Hi, I'm seeing a segfault and error on Mac with OpenCL and different array data types.

• f32 - success
• f64 - segfault
• s32 - error (stack trace below)

  File "/usr/local/lib/python3.9/site-packages/khiva/matrix.py", line 231, in stomp_self_join
    raise Exception(str(error_message.value.decode()))
Exception: stomp_self_join: ArrayFire Exception (Internal error:998):
In function cl::Program opencl::buildProgram(const vector<std::__1::string> &, const vector<std::__1::string> &)
In file src/backend/opencl/compile_module.cpp:128
OpenCL Device: Intel(R) Iris(TM) 

To Reproduce

It's easiest to reproduce with the Python library (but think it's probably related to the C++ code).

from khiva.array import Array, dtype
from khiva.library import get_backend_info, set_backend, KHIVABackend
from khiva.matrix import stomp_self_join

# everything works with the CPU backend
# set_backend(KHIVABackend.KHIVA_BACKEND_CPU)

print(get_backend_info())

# success
a = Array.from_list([1, 2, 3, 4, 5], dtype.f32)
stomp_self_join(a, 3)

# segfault
a = Array.from_list([1, 2, 3, 4, 5], dtype.f64)
stomp_self_join(a, 3)

# error
a = Array.from_list([1, 2, 3, 4, 5], dtype.s32)
stomp_self_join(a, 3)

Expected behavior

No segfault or error, like with the CPU backend.

Environment information:

• OS: Mac OS 11.1
• Khiva Version: 0.5.0
• Khiva dependencies versions: ArrayFire 3.7.3, Boost 1.74.0

Here's the output of get_backend_info():

ArrayFire v3.7.3 (OpenCL, 64-bit Mac OSX, build default)
[0] APPLE: Intel(R) Iris(TM) Plus Graphics, 1536 MB

Additional context

Let me know if there's any I can do to help debug.

Not sure if this is on my end or in one of my shared libraries causing the problem.

Here's the error during the build.
https://gist.github.com/zpzim/a7added1e63587e162f871383e33b1dc

It might be related to Intel TBB. I get this error even when I comment out the entire test function

Describe the bug
Linking errors raise when building the project and linking against the libraries installed by conan.
Errors sample:

CMakeFiles/features.dir/featuresTest.cpp.o: In function `testing::AssertionResult testing::internal::CmpHelperEQFailure<long long, int>(char const*, char const*, long long const&, int const&)':
/home/asem/.conan/data/gtest/1.8.1/bincrafters/stable/package/4a8c5b4cd3b4d45b83fff85d53160ea02ae5fa2d/include/gtest/gtest.h:1435: undefined reference to `testing::internal::EqFailure(char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)'

The linking issue is fixed when linking the khiva tests against the same gtest version 1.8.0 that is built from its main repository.

Sample from CMakeCache.txt before fixing

//Path to a file.
GTEST_INCLUDE_DIRS:PATH=/home/asem/.conan/data/gtest/1.8.1/bincrafters/stable/package/4a8c5b4cd3b4d45b83fff85d53160ea02ae5fa2d/include

//Path to a library.
GTEST_LIBRARIES:FILEPATH=/home/asem/.conan/data/gtest/1.8.1/bincrafters/stable/package/4a8c5b4cd3b4d45b83fff85d53160ea02ae5fa2d/lib/libgtest.a

//Path to a library.
GTEST_MAIN_LIBRARIES:FILEPATH=/home/asem/.conan/data/gtest/1.8.1/bincrafters/stable/package/4a8c5b4cd3b4d45b83fff85d53160ea02ae5fa2d/lib/libgtest_main.a

After fixing

//Path to a file.
GTEST_INCLUDE_DIRS:PATH=/opt/gtest/include

//Path to a library.
GTEST_LIBRARIES:FILEPATH=/opt/gtest/lib/libgtest.a

//Path to a library.
GTEST_MAIN_LIBRARIES:FILEPATH=/opt/gtest/lib/libgtest_main.a

I am just reporting this case since it costed me a lot of time! Couldn't expect that conan might be the reason.

Describe the bug
I can’t install khiva, when I run make –j8 I get the following error after 32% :

CMakCe EMrraork aet /EAprprliocr ataiton s//CMAakpe.palppi/Ccoantteintosn/ssh/aCrMe/ackmeak.ea-3p.p14//CMoodnultees/nGtoosg/lesThesatrAded/Tcemsatkse.-c3m.a1k4e/:M4o0d (message):
u l eEsr/rGooro grluenTneisntgA dtdeTsets tesx.eccmuatkaeb:l4e0.
(
m e s s aPgateh): :'
/U s eErrsr/omra rriuan/nPiyncgha rtmePsrto jeexcetcsu/tkahbilvea.N
e
w / b u iPladt/bhi:n /'p/oUlsyenrosm/imaalr'i
a / P y cRheasrumlPtr:o jPercoctess/sk htievramNienwa/tbeudi lddu/eb itno/ vteirmseioount'

   O utRpuets:

u l t : P
r
o
cess termin
ated due to timeout
Output:

make[2]: *** [bin/polynomial] Error 1
make[2]: *** Deleting file bin/polynomial' make[2]: *** [bin/version] Error 1 make[2]: *** Deleting file bin/version'
make[1]: *** [test/CMakeFiles/polynomial.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [test/CMakeFiles/version.dir/all] Error 2
[ 32%] Built target statistics
[ 32%] Built target regularization
[ 32%] Built target normalization
[ 32%] Built target clustering
[ 32%] Built target matrix
fatal: not a git repository (or any of the parent directories): .git

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
File "/Users/maria/anaconda3/lib/python3.7/site-packages/sphinx/config.py", line 368, in eval_config_file
execfile_(filename, namespace)
File "/Users/maria/anaconda3/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 150, in execfile_
exec_(code, _globals)
File "/Users/maria/PycharmProjects/khivaNew/doc/sphinx/source/conf.py", line 44, in
["git", "describe"]).strip().decode("utf-8").split('-')[0]
File "/Users/maria/anaconda3/lib/python3.7/subprocess.py", line 389, in check_output
**kwargs).stdout
File "/Users/maria/anaconda3/lib/python3.7/subprocess.py", line 481, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'describe']' returned non-zero exit status 128.

make[2]: *** [doc/CMakeFiles/documentation] Error 2
make[1]: *** [doc/CMakeFiles/documentation.dir/all] Error 2
make: *** [all] Error 2

To Reproduce
after running:
make -j8

Environment information:

• OS: 10.14.3
• Python 3.5
• ArrayFire 3.6.2
• Conan version 1.13.2

The binnedEntropy function produces nan values with the following input parameters:

auto tss = {1..20, 21.. 40}
result = binnedEntropy(tss, 5)
af_print(result) => [nan nan]

Describe the bug
The documentation of the mass function is not correct.

Expected behavior
The brief part of the documentation should state Mueen's Algorithm for Similarity Search

Describe the bug
In the documentation it appears as tss_l and tss_n as parameters of the znorm functions and they are not parameters

Expected behavior
/**
* @brief Calculates a new set of times series with zero mean and standard deviation one.
*
* @param tss Time series concatenated in a single row.
* @param epsilon Minimum standard deviation to consider. It acts as a gatekeeper for
* those time series that may be constant or near constant.
* @param result Array with the same dimensions as tss where the time series have been
* adjusted for zero mean and one as standard deviation.
*/
[DllImport(DLLLibrary.khivaPath, CallingConvention = CallingConvention.Cdecl)]
public extern static void znorm([In] ref IntPtr tss, [In] ref double epsilon, [Out] out IntPtr result);

Describe the bug
That function produces a OSError tested in Python:
Traceback (most recent call last):
File "", line 1, in
File "C:\Python37\lib\site-packages\khiva\features.py", line 607, in local_maximals
ctypes.pointer(b))
OSError: [WinError -529697949] Windows Error 0xe06d7363

To Reproduce
import khiva as kv
import numpy as np
local_maximals_result = kv.local_maximals(kv.Array(data=[0.0, 4.0, 3.0, 5.0, 4.0, 1.0, 0.0, 4.0])).to_numpy()

Expected behavior
numpy array of [0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
Environment information:

• OS: Windows
• Version 10
• Khiva dependencies versions [e.g. ArrayFire 3.6.2, Boost 1.66.0]

Hi, also wanted to let you know there are now Ruby bindings for Khiva. It's modeled after the Python bindings (with some minor changes to make it more Ruby-like). The C API was really nice to use and easy to follow.

If you have any feedback, feel free to let me know here or create an issue in the project. Thanks!

Describe the bug
In the documentation of find_best_n_discords function there is a @param that does not include the number of the param

Expected behavior
/**
* @brief Primitive of the findBestNDiscords function.
*
* @param profile The matrix profile containing the minimum distance of each
* subsequence
* @param index The matrix profile index containing the index of the most similar
* subsequence
* @param length_profile Length of the matrix profile
* @param m Subsequence length value used to calculate the input matrix profile.
* @param n Number of discords to extract
* @param discord_distances The distance of the best N discords
* @param discord_indices The indices of the best N discords
* @param subsequence_indices The indices of the query sequences that produced
* the "N" bigger discords.
* @param self_join Indicates whether the input profile comes from a self join operation or not. It determines
* whether the mirror similar region is included in the output or not.
*/

Is your feature request related to a problem? Please describe.
Khiva already contains a function that computes the maximum point within a time series.
However, khiva is missing a function that computes the maximum between several time series. It is getting the maximum element across all time series for each dimension.

t1 -> [0,1,2,3]
t2 -> [1,0,4,1]
t3 -> [2,0,1,4]
Res -> [2,1,4,4]

Describe the bug
The actual @brief of energy_ratio_by_chunks in khiva_c is
@brief Calculates the sum of squares of chunk i out of N chunks expressed as a ratio.

• with the sum of squares over the whole series. segmentFocus should be lower
• than the number of segments

Expected behavior
The @brief should be:
@brief Calculates the sum of squares of chunk i out of N chunks expressed as a ratio

• with the sum of squares over the whole series. segment_focus should be lower
• than the number of segments

great code thanks
may you clarify :
will it work for multivariate time series
1
where all values are continues values
2
or even will it work for multivariate time series where values are mixture of continues and categorical values
for example 2 dimensions have continues values and 3 dimensions are categorical values

color        weight     gender  height  age  

1 black 56 m 160 34
2 white 77 f 170 54
3 yellow 87 m 167 43
4 white 55 m 198 72
5 white 88 f 176 32

Describe the bug
Visvalingam requires a input array with two columns, where the fist one is the x-axis and the second one the y-values. We need to describe that requirement in the description of the method.

Describe the bug
This function returns 4 arrays containing 4 different coefficients for all input sequences.
It doesn't match the style of all other khiva functions. Thus, we need to return one array with for 4 rows and a number of columns equal to the number of input sequences.

Describe the bug
Before run command cmake .. is necessary run: git submodule update --init

Environment information:

• OS: macOS Mojave
• Version 10.14.6
• Khiva dependencies versions ArrayFire 3.6.2, etc.

Hi, thanks for this great library! Just wanted to let you know it's now available on Homebrew, so Mac users can install it with:

brew install khiva

Homebrew installs all dependencies and precompiles the library for each version of macOS, so installation is really quick and easy. 🎉

Ref: Homebrew/homebrew-core#67078

Khiva already contains a number of functions with a diverse purpose i.e. clustering, distances, dimensionality, feature extraction, normalization, etc...
However, khiva is lacking some functions to perform predictions as GARCH model, ARIMA or other more complex ones.

It would be nice to develop a module for forecasting purposes.

Issue:
ApproximateEntropy function is returning time series in a row fashion instead of columns.

Code example:
auto a = approximateEntropy(tss);
af_array(a);

Describe the bug
Following the installation instructions in the README, and issuing conan remote add conan-mpusz https://api.bintray.com/conan/mpusz/conan-mpusz

The following error message is raised:

google-benchmark/1.3.0@mpusz/stable: Not found in local cache, looking in remotes...
google-benchmark/1.3.0@mpusz/stable: Trying with 'conan-center'...
google-benchmark/1.3.0@mpusz/stable: Trying with 'conan-mpusz'...
ERROR: Failed requirement 'google-benchmark/1.3.0@mpusz/stable' from 'PROJECT'
ERROR: Unable to find 'google-benchmark/1.3.0@mpusz/stable' in remotes

To Reproduce

cd build
conan install .. -s build_type=Debug

Environment information:

• OS: Ubuntu
• Version 18.04
• Khiva dependencies versions Conan version 1.9.2

Is your feature request related to a problem? Please describe.
We need to include a shapelet extraction algorithm in Khiva. This kind of algorithms let extract important segments of a time series that are meaningful to characterise the time series under observation.

Examples of this type of algorithms are given in the following works [1], [2] [3] and [4]

Describe the solution you'd like
It would be nice to have some algorithms that let us extract shapelets from time series to perform more complex analysis.

Describe the bug
Function features::localMaximals(tss) is not tested, we need to add its test.

Error Message:

libc++abi.dylib: terminating with uncaught exception of type af::exception: ArrayFire Exception (Invalid input size:203):
In function dim_t af::calcDim(const af_seq &, const dim_t &)
In file src/backend/dim4.cpp:205
Invalid dimension for argument 1
Expected: seq.begin >= -DBL_MIN && seq.begin < parentDim

In function af::array af::array::array_proxy::operator array()
In file src/api/cpp/array.cpp:719
Abort trap: 6

Is your feature request related to a problem? Please describe.
It would be nice to have a function that let us reverse a time series. Like the following example:

a = [1,2,3,4,5]
a_reversed = [5,4,3,2,1]