Interaction between two categorical covariates sometimes switches order, causing error about formulaic HOT 3 CLOSED

Hi Martin! Thanks for taking the time to report this bug.

This ought not to be possible because terms are ordered based on the formula, and the algorithm should in principle be deterministic after this. And I don't seem to be able to reproduce it. The following works fine for me:

import formulaic
import numpy
import pandas

for n in (10, 100, 1000):
    df = pandas.DataFrame({
        "frog_type": numpy.random.choice(["stiff", "movable"], n),
        "is_cargo": numpy.random.choice([True, False], n),
    })
    for i in range(1000):
        formulaic.model_matrix("C(frog_type):C(is_cargo)", df).model_spec.get_model_matrix(df)

Do you have an example you can share of when this breaks?

from formulaic.

Thanks Matthew for replying so quickly, I really appreciate it!

I'll have to work on an example; it's slightly tricky because I won't be able to share the actual data I'm working with.

One question: the error actually appeared originally using formulaic 0.5.2. I save a pickle file, restore it, and then do these computations. The error also persisted with the lastest formulaic version and this pickle file, but I am now wondering whether maybe it's because the pickle file was saved with the old version. Basically, my question is: do you think it could be that this was possible in 0.5.2 but has been fixed since?

from formulaic.

Ah... yes. Version 0.6.0 changed the ordering of terms, as described in the release notes, so if you are using older model specs that will be problematic.

Also note that (as described in the docs here), reusing model specs from older versions of formulaic is not supported. In most cases, it will work fine, but there will be occasional issues which we do not plan to support at this time.

If things consistently fail for the same formula/model spec, that is expected here. If it stochastically fails, then this is more concerning, since both 0.5.x and 0.6.x have stable (but different) term ordering strategies.

I'll close this one out for now, but feel free to reopen if you think there's any remaining issues not dealt with above!

And thanks again!

from formulaic.