I need to find a way to deal with linearly dependent inputs.

Hi,

I find it hard to install this package in windows.
I have anaconda ,numpy installed and cygwin installed.
the installations stalls with "c:....\gcc.exe failed with exit status 1"

thanks
Madhu

The _utilreorderxby function is a bottleneck for large sample sizes. Using BLAS for the row copies and swaps might speed it up. Or, is there a faster algorithm?

Currently, the entire data set is reordered for each variable at each iteration. This process is currently the biggest performance bottleneck by far. It would be better if all variable orders were determined upfront and stored, then passed into valid_knots and best_knot. This requires modification to those two methods. It will probably make both methods slower, but hopefully not as slow as reordering the data all the time.

The ability to read and write models in PMML would do a lot to improve portability.

Dear Jason,

I just tried a couple of fitting examples on which to apply pyearth.

This is the code:
import numpy
from pyearth import Earth
from matplotlib import pyplot

Create some fake data

numpy.random.seed(seed = 0)

N = 100
X = numpy.linspace(-10, 10, N)
numpy.random.shuffle(X)
y = numpy.zeros(X.shape)
sigma = 1
for i in range(y.size):
r = numpy.random.normal(1, sigma)
y[i] = numpy.sin(X[i]) + r

Fit an Earth model

model = Earth()
print 'fitting'
model.fit(X,y)
print 'done'

print "\n*** SUMMARY ***"

print model.summary()

Plot the model

y_hat = model.predict(X)
pyplot.figure()
pyplot.plot(X,y,'r.')
pyplot.plot(X,y_hat,'b.')
pyplot.show()

It generates a noisy sin function and tries to fit it. There an issue I'd like to point out. If the number of samples is not large enough, e.g. N = 100, the algorithm may crash, and raise a segmentation fault. Its success depends on the data. If I do not set the random seed, then sometimes it crashes and sometimes it does not.

I hope this information may help you!

Best,

Marco

py-Earth fails when fitting a multi column [m x n] label matrix with the two-dimensional training data. This can be considered as an enhancement

model.fit(X_train, y_train)
File "/Library/Python/2.7/site-packages/pyearth/earth.py", line 331, in fit
X, y, sample_weight = self._scrub(X, y, sample_weight)
File "/Library/Python/2.7/site-packages/pyearth/earth.py", line 262, in _scrub
y = y.reshape(y.shape[0])
ValueError: total size of new array must be unchanged

See the other Friedman, 1991, entitled "Estimating functions of mixed ordinal and categorical variables using adaptive splines".

See Friedman, 1993

All the publicly visible methods should have docstrings.

y <- abs(x6 - 4) + E

Gives this result:

Forward Pass

iter parent var knot mse terms gcv rsq grsq

0 - - - 150.123858 1 150.425 0.000 0.000
1 0 6 431 0.911382 3 0.928 0.994 0.994
2 0 9 981 0.903521 5 0.935 0.994 0.994
Stopping Condition: 2

Pruning Pass

iter bf terms mse gcv rsq grsq

0 - 5 0.90 0.935 0.994 0.994
1 4 4 0.91 0.933 0.994 0.994
2 3 3 0.91 0.928 0.994 0.994
3 2 2 140.46 141.875 0.064 0.057
4 1 1 150.12 150.425 0.000 0.000
Selected iteration: 2

Earth Model

(Intercept) -0.022567565678
h(x6-4.0982) -0.0736813881288
h(4.0982-x6) -0.0328571125806
h(x9+38.6904) -0.0066944613208

Specifically, changing minspan_alpha seems to have no effect.

This can be accomplished by premultiplying basis functions by the Cholesky square root of the weight matrix (a generalization of how diagonal weight matrices are currently handled). See equation (7) in [1].

[1] Green, P. J. (2011). Iteratively Reweighted Least Squares for Maximum Likelihood Estimation, and some Robust and Resistant Alternatives. Journal of The Royal Statistical Society, Series B (Methodological), 46(2), 149–192.

Both of the usage examples result in the same error for me..

import numpy
import cProfile
from pyearth import Earth
from matplotlib import pyplot

numpy.random.seed(2)
m = 1000
n = 10
X = 80_numpy.random.uniform(size=(m,n)) - 40
y = numpy.abs(X[:,6] - 4.0) + 1_numpy.random.normal(size=m)
model = Earth(max_degree = 1)
model.fit(X,y)
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\pyearth\earth.py", line 312, in fit
self.forward_pass(X, y)
File "C:\Python27\lib\site-packages\pyearth\earth.py", line 383, in forward_pass
forward_passer = ForwardPasser(X, y, **args)
File "_forward.pyx", line 67, in pyearth._forward.ForwardPasser.init (pyearth/_forward.c:3146)
File "_forward.pyx", line 96, in pyearth._forward.ForwardPasser.init_linear_variables (pyearth/_forward.c:3698)
ValueError: Buffer dtype mismatch, expected 'INT_t' but got 'long long'

Probably the best way is to not store the ForwardPasser or PruningPasser objects, but rathe just store their records.

This is a very basic feature. It should be easy for users to get the parameters of the fitted model, such as basis function coefficients, and statistics like R^2 or GCV.

I need to figure out what the correct formula should be.

Currently the ForwardPasser automatically decides whether a variable should enter linearly. There should also be a way for users to specify that a variable should enter linearly.

I saw this issue on Ubuntu 12.04 (LTS Kernel) with a source install of sklearn (version 0.16-git). The setup process completes correctly, but I see the following traceback when importing pyearth:

>>> import pyearth
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pyearth/__init__.py", line 8, in <module>
    from .earth import Earth
  File "/usr/local/lib/python2.7/dist-packages/pyearth/earth.py", line 5, in <module>
    from sklearn.utils.validation import assert_all_finite, safe_asarray
ImportError: cannot import name safe_asarray

It looks like sklearn.utils.validation.safe_asarray is an internal tool to sklearn, and as mentioned on their website, is not guaranteed to be stable between releases.

There needs to be documentation.

This pertains to the smoothed representation of Friedman, 1991, section 3.7. Should there be a larger penalty in the cost complexity function? Should smoothing happen before or after pruning?

Such as the transform and scale methods

Specifically, user defined endspans are not used.

Unit tests are currently not very scikit-learn friendly

Have some kind of string or message instead of just a number.

It produces nans. I'm guessing this has to do with adjacent knot candidates being exactly equal, which generally only happens with non-continuous distributions, but might also happen with zero-padded data.

The xlabels argument is currently treated as a hyperparameter. It should probably be treated differently. Issues:

1. In the Pandas case, the labels come with the data and should therefore be set in fit.
2. In the Pandas case, column order may be less relevant than column name.

At the very least, I should make it easy to set xlabels in the fit method.

Specifically, there is a bug in the way new basis functions are added at the end of next_pair after the new parent, variable, and knot location have been selected.

Currently there is no checking on input data. If you put in data in the wrong format, the first time you know about it might be when you get a segfault or, worse, an incorrect model.

If neither numpy nor pyearth are installed, and we install both of them by:

$ pip install -r requirements.txt

where requirements.txt:

numpy
scipy
-e git+https://github.com/jcrudy/py-earth.git#egg=py-earth

Throws ImportError: no module named numpy when installing pyearth. Refer to this issue and this issue to see how pip works in this case.

I might be missing something but setup.py includes the examples/vFunctionExample.py which is not in the repository.

'scripts':['examples/vFunctionExample.py'],

Add a cythonize option to setup.py, but it should not be the default option. By default just compile the .c files.

Is there a way to dump the model formula ?

Currently pickling raises a TypeError