Multi-expression objects and fixed functional forms about symbolicregression.jl HOT 4 OPEN

I have done something like that in the past.
You have a good catch of mutating a single tree at the time: as the search space is larger with multi-trees, of course less likely to get multiple lucky simultaneous mutations.

Makes sense to assume all solutions have same number of trees if you know this a priori, but that needs not always need to be the case. E.g., you might want to do SR for interpretable dimensionality reduction from R^d (d original features) to R^k (k latent features), where you want k to be small to have fewer trees to interpret, but large enough to get good reconstruction accuracy.

from symbolicregression.jl.

This could also be a nice way to get vector expressions:

SymbolicRegression.@form F = [f1(x1, x2, x3), f2(x1, x2, x3), f3(x1, x2, x3)]

from symbolicregression.jl.

I have been thinking more about how this could work. Programmatically I think the easiest way forward is as follows.

1. Create a way to "freeze" a set of nodes in a tree. Those nodes cannot be mutated or used in crossovers.
   • Option 1: a new custom tree type FrozenNode <: AbstractNode (SymbolicML/DynamicExpressions.jl#53)
   • Option 2: maybe just storing custom metadata in Node{T,schema}?
2. You would use a regular Node type to store all expression information. For two subexpressions f and g, you would have root.l storing f, and root.r storing g. Since root would be labeled as frozen, it would not be reduced.
   • However, you could still crossover the contents of f with the contents of g. Just you couldn't crossover root itself.
3. Rewrite functions like eval_tree_array, string_tree, node_to_symbolic to work with a custom functional form.

It will still take a bit of work to get this to work. But I think this is the easiest way to embed it in SymbolicRegression.jl without breaking anything for existing functionality.

from symbolicregression.jl.

Found this issue after submitting a quite similar feature request for composite PySR regressor in MilesCranmer/PySR#514

For the evolution: at each mutation, perhaps one random tree of the multi-tree would be mutated or crossed-over. There would be an assumption that all individuals have the same multi-tree structure.

I wonder when different trees are degenerate, would correlated changes (mutating all sub-trees together) actually help to optimize faster, even though with a larger search space?

1. Create a way to "freeze" a set of nodes in a tree.

Very interesting approach. If I understand correctly, would it be easy for this to enforce, say the final sub-tree is at most a bivariate function (i.e., $k\leq2$ in @marcovirgolin 's comment), with one variable being a constrained expression of the other sub-trees ($f+g/f$ in an above example)?

from symbolicregression.jl.