Git Product home page Git Product logo

ariadne's People

Contributors

evelinag avatar sergey-tihon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ariadne's Issues

Overflow in GaussianProcess.Predict() when Using Large Dataset

I am working with a rather large sub-set of data (922854 location-position pairs). When attempting to do GuassianProcess.Predict() an integer overflow occurs, which results in an index out of bounds error. (Specifically, the index is being set to -2146233066).

I have narrowed in this issue starting somewhere between 45,000 and 50,000 pairs, with the library running and producing results without error when in the valid range.

A trace of the error is listed below:

   at MathNet.Numerics.LinearAlgebra.Storage.DenseColumnMajorMatrixStorage`1.OfInit(Int32 rows, Int32 columns, Func`3 init)
   at MathNet.Numerics.LinearAlgebra.MatrixBuilder`1.Dense(Int32 rows, Int32 columns, Func`3 init)
   at Ariadne.GaussianProcess.covarianceMatrix[T](FSharpFunc`2 kernel, T[] input1, T[] input2)
   at Ariadne.GaussianProcess.GaussianProcess`1.PosteriorGaussianProcess(IEnumerable`1 data, T[] newLocations)
   at Ariadne.GaussianProcess.GaussianProcess`1.Predict(IEnumerable`1 data, T[] newLocations)
   at package.Program.function(params) in C:\Users\squirvel\package\Program.cs:line xxx

I am calling the library from some C# code, so I am unsure if that would affect anything. I do not think that that is the case, but for the sake of completion, here is what my code looks like:

// Generate gaussian process
// start with covariance function. Define params first
float lengthscale = 0.5F;
float signalVariance = 0.3F;
float noiseVariance = 0.5F;

// Setup kernel fuction
Ariadne.Kernels.SquaredExp.SquaredExp kernelFunction = new Ariadne.Kernels.SquaredExp.SquaredExp(lengthscale, signalVariance, noiseVariance);
// Use kernel function to spawn process
Ariadne.GaussianProcess.GaussianProcess<double> gaussianProcess = kernelFunction.GaussianProcess();

// Points is an object I am unpacking values into these items from
double[] pointTimes = new double[points.Count];
double[] xPoints = new double[points.Count];
// unpacking of values into above variables occurs here

// Generate Observation sets
int SIZE = 45000;  // Testing variable to help isolate around when the bug occurs
Ariadne.GaussianProcess.Observation<double> xObservations = new Ariadne.GaussianProcess.Observation<double>(pointTimes[0..SIZE], xPoints[0..SIZE]);
Ariadne.GaussianProcess.Observation<double> yObservations = new Ariadne.GaussianProcess.Observation<double>(pointTimes[0..SIZE], yPoints[0..SIZE]);

// Turn into enumeratble objects (required for function signature)
List<Ariadne.GaussianProcess.Observation<double>> xEnumerable = new List<Ariadne.GaussianProcess.Observation<double>>
{
     xObservations
};

// Attempt predictions
// TIMESTAMPS is pre-defined, and is the set of locations I am attempting to get predictions for, and is 10 locations
Tuple<double[], double[]> xResults = gaussianProcess.Predict(xEnumerable, TIMESTAMPS);  // IndexOutOfBounds error occurs here

As it can be seen in the trace, this is not specifically occurring in Ariadne, but rather the underlying MathNet.Numerics library. Specifically in LinearAlgebra.MatrixBuilder

My current speculation is that:
The list contained in xEnumerable is getting casted to an array. Because the MathNet library would be pre-compiled, likely without gcAllowVeryLargeObjects being enabled, we are literally running out of space, and overflowing. This could be avoided if we can work directly with lists rather than Int32 arrays.

However I am not sure if that is the case, and if were, I have not been able to find code that hints at such a casting occurring.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.