Add support for multi-stage formulas. about formulaic HOT 5 OPEN

What is the intention of the first formula? What is exogenous and what is endogenous? Clearly the Z are instruments.
.

from formulaic.

Returning to this after several years 😓 .

Multi-part formulas are already implemented as of v0.3.0: y ~ a | b | c does the right thing.

@bashtage : If I were to take this further, I'd look to implement something like: y ~ 1 + x1 + x2 + x3 + [ x4 + x5 ~ z1 + z2 + z3], exactly as you have done here. The results would be made available on the Structured instance as something like:

.lhs
    y
.rhs
    1 + x1 + x2 + x3 + IV[x4] + IV[x5]
    .iv_x4:
        z1 + z2 + z3
    .iv_x5:
        z1 + z2 + z3

This is within reach of the parser now, but I'd love your take on this (given that you have much more experience in this space).

from formulaic.

An advanced syntax would be great. I have a few current uses.

1. IV like you have above.
2. Absorbing regression where high dimensional fixed effects are absorbed. Something like y ~ x + [eff1 + eff2 + eff3] where eff# are categorical variables usually that are then encoded to sparse arrays.
3. Systems equations. I currently use a dictionary. These models have multiple equations, something like y1 ~ x + z, y2 ~ x + w. Not sure if something like this woudl make sense to have as a syntax.

from formulaic.

Nice. I don't yet know how much it makes sense to always have these advanced operators in place (versus having a family of parsers that extend some common set), but I'll definitely be working toward making the parser capable of generating formulae for these kinds of situations.

For further clarity:
On 2. Absorbing regression is just your usual fixed-effects regression, right? Where you demean the data based on a set of covariates prior to modelling, perhaps using another regression? What would you want output in that case? Something like:

.lhs
    y_residuals
    .fixed_effects
         eff1 + eff2 + eff3
.rhs
    x

On 3. Would a Structured instance of a tuple of formulas work? That could be implemented trivially today (either in formulaic or downstream by adding the , operator):

[0]
    .rhs
        y1
    .lhs
        x + z
[1]
    ...

from formulaic.

I haven't really through about it. I could imagine that formulas could be nested. For example

y ~ 1 + x + [w ~ z]

could be something like

.lhs
   y
.rhs
   1 + x + [w ~ z]

and when you access .rhs it would be [1{Term}, x{Term}, [w ~ z]{Formula}] so that one could handle nested formulas with some recusions, e.g.

for term_or_fmla in formula.rhs.terms:
    if isinstance(term_or_fmla , Term):
        """Do something"""
    else:
        """Handle nested formula probably using a recursion"""

Maybe too complicted.

from formulaic.