Settle syntax about soss.jl HOT 15 CLOSED

Hmm good point. Maybe m(y=[1,2,3]) (the more common case) for observations in the usual sense, and foo(m, :(y=[1,2,3])) for inlining. for some value of foo [Note to self: don't actually call it foo].

from soss.jl.

Another great question. And again it's really helpful, because it's getting me to think through things I've been taking for granted.

I think both map and broadcast make sense here. A model is really a generalization of a distribution. Soss is already set up to be able to write things like

m = @model foo
   x ~ foo
end

where foo can be either a distribution or another model (for details see this slide.

Having function application syntax makes this even better, since now we should be able to write foo as a function call in both places.

Oh!! Does this mean we should adopt function notation overall? As in, be able to call a model just based on the order of arguments? Should a model allow kwargs with default values?!

from soss.jl.

I don't really care much for the data = . . . syntax. It's a little clunky and it ultimately just adds another layer to the signature that doesn't really need to be there.

kwargs looks like a good route to me.

from soss.jl.

kwargs looks like a good route to me.

I like that too. My concern was if we have other things to pass. Some inference algorithms have values that can be tuned, like number of samples or proposal acceptance rate. My biggest concern with passing data as kwargs is how to keep from getting it confounded with these tuning parameters. I guess a simple solution would be to insist on a form for tuning parameters, like having them lead with a _. But "stringly-typed" is kind of smelly, and it's not clear to me how to get around it

from soss.jl.

At one stage I thought your models supported creating derived models by adding observations? In that case perhaps you can use

model = @model ...

derived_model = model(a=2, b=[2,3,4]) # or whatever your current syntax is for adding observations

nuts(derived_model; nuts_config_kwargs...)

# More succinctly
nuts(m(a=2, b=[2,3,4]); nuts_config...)

This feels like it composes better than trying to cram two sets of different keyword args into the same function call.

from soss.jl.

At one stage I thought your models supported creating derived models by adding observations?

They do, but in two different ways. In the current implementation, m(a=2, b=[2,3,4]) inlines those values statically into the model. This is great for small values, but not really usable for most real data.

But it's an interesting suggestion, and I do like the syntax. Most data is "boxed" instead of inlined, so maybe we could do something like

• m(:(a=2)) inlines the expression in the code
• m(a=2) adds the data a=2 to the model. "adding the data" could be in terms of an additional field for the Model type, and (IIUC) this could also have the advantage of helping type inference, since it would need to resolve before calling the inference method.

Currently a Model looks like this:

struct Model <: AbstractModel
    args :: Vector{Symbol}
    body :: Vector{Statement}
end

Maybe a Model should also be parameterized by the type of data it takes? Then we could add a data field matching this parameter?

@thautwarm is looking at the types to see if there's a way to pin things down enough to use generated functions, seems like this could help that along

from soss.jl.

Most data is "boxed" instead of inlined, so maybe we could do something like ...

Yes I like this distinction between inlining vs carrying the data along separately. It seems quite excessive / unnecessary to create a completely new model type just to add some observations.

I'm not sure about the syntax m(:(a=2)); if the case of inlining some extra AST is actually uncommon you could just have a new / different method for that. If you view models as (distributions over) named tuples, Base.merge might have the right semantics?

merge(m, (a=2,))

from soss.jl.

Base.merge might have the right semantics?

I like this idea if it were just this, but currently we can also do things like m(N=:(length(x))) for a kind of code editing. If we use your idea for the usual case of observing data, this would conflict, but we could still dispatch on m(::Expr). So for example,

m(y=[1,2,3])                     # "boxed" observation - typical PPL style
m(:(y = 3*x))                    # code editing
m(:(y ~ Poisson(2.0) |> iid(3))) # code editing
m(:(y = [1,2,3]))                # "inlined data", but really also more code editing

From the Soss perspective, there are only two cases here: the first, and the rest. This also opens the door for programmatic high-level model rewriting :)

from soss.jl.

Yes merge probably isn't quite right. The underlying idea is that a given generic function should have well defined meaning. So if code editing is semantically different from adding observations, perhaps they should be separate generic functions. I know this somewhat goes against the desire to use the most compact syntax possible :-)

from soss.jl.

To throw another spanner in the design works (sorry!) — what should it mean to treat a model as a function? It's convenient syntax for adding data, but could there be a more natural meaning? map and broadcast are built on top of call syntax; is there a sensible meaning for map(model, things) and model.(things)?

from soss.jl.

Does this mean we should adopt function notation overall? As in, be able to call a model just based on the order of arguments? Should a model allow kwargs with default values?!

I don't know! But that sounds like a great question to be asking. (Side note - keyword arguments don't participate in broadcasting syntax so that might be a reason to use positional arguments for some things. But wanting that to work seems like a corner case and maybe not worth worrying about?)

from soss.jl.

I kind of like the idea of allowing users to decide this, just as Julia does for functions. But this would mean either

@model a,b=2 ...

or

@model a; b=2 ...

The latter is more like "normal" Julia kwargs. But there seems to be some parsing weirdness here. For example,

julia> :(a,b,c=1; d=2)
:(($(Expr(:parameters, :(d=2))), a, b, c = 1))

@c42f any insights/suggestions here?

from soss.jl.

Expr(:parameters) is the way that keyword parameters in function definitions are represented separately from positional parameters with defaults. For example

julia> Meta.@dump f(a,b, c=1,d=2)
Expr
  head: Symbol call
  args: Array{Any}((5,))
    1: Symbol f
    2: Symbol a
    3: Symbol b
    4: Expr
      head: Symbol kw
      args: Array{Any}((2,))
        1: Symbol c
        2: Int64 1
    5: Expr
      head: Symbol kw
      args: Array{Any}((2,))
        1: Symbol d
        2: Int64 2

julia> Meta.@dump f(a,b; c=1,d=2)
Expr
  head: Symbol call
  args: Array{Any}((4,))
    1: Symbol f
    2: Expr
      head: Symbol parameters
      args: Array{Any}((2,))
        1: Expr
          head: Symbol kw
          args: Array{Any}((2,))
            1: Symbol c
            2: Int64 1
        2: Expr
          head: Symbol kw
          args: Array{Any}((2,))
            1: Symbol d
            2: Int64 2
    3: Symbol a
    4: Symbol b

from soss.jl.

btw this distinction between is largely unnecessary for function calls (except when splatting lists of keywords) but it's required for function definitions. The parser uses the same AST representation regardless of function call vs function definition.

from soss.jl.

This is now settled:

• A model makes no reference to which data are observed. It's a stochastic function-like object.
• Models are parameterized by the names of their arguments and a type-level representation of their body
• "Applying" a model to a NamedTuple matching its arguments results in a BoundModel (might rename this later)
• Most inference methods require specification of data. This is also passed as a named tuple.

A common interface is currently something like

myInference(myModel(arg1=val1, arg2=val2), dataNamedTuple)

from soss.jl.