VLSIR has its own primitives-library. It's largely that of SPICE: ideal elements.
Hdl21 also has its PHYSICAL primitives, e.g. a capacitor specified in physical dimensions (length, width, and layers).

We should probably expose at least the latter as VLSIR literals, similar to how VLSIR exposes its own primitives-library.

@aviralpandey reported hdl21.to_proto failing when run in a Jupyter notebook.
And it does. The proto-exporter is always looking for a qualified (Python) path to name modules.
So that if you have a pile of files like this, each defining a module named Flop:

• folder1/sub1/stuff1.py::Flop
• folder2/.../my_favorite_module.py::Flop
• my_modules.py::Flop

The intent is we can uniquely name each of those.
(Now, I’m not sure this actually carries through to netlisting - but that’s another story.)
The core bit of logic which does this is here:

def _qualname(mod: Union[Module, ExternalModule]) -> Optional[str]:
    """# Qualified Name
    Helper for exporting. Returns a module or import path-qualified name."""

    if mod.name is None:
        return None
    return mod._defpath() + "." + mod.name

def _defpath(mod: Union[Module, ExternalModule]) -> str:
    """# Definition Path
    Helper for exporting.
    Returns a string representing "where" this module was defined.
    This is generally one of a few things:
    * If "normally" defined via Python code, it's the Python module path
    * If *imported*, it's the path inferred during import"""

    if mod._importpath is not None:
        # Imported. Return the period-separated import path.
        return ".".join(mod._importpath)

    # Defined the old fashioned way. Use the Python module name.
    if mod._source_info.pymodule is None:
        raise RuntimeError(f"{mod} is not defined in Python")
    return mod._source_info.pymodule.__name__

Which I think can pretty quickly be tweaked to:

• If a module has no name, its _qualname is also None (already there)
• ADD: If a module has no _source_info.pymodule - as is the case in the notebooks - its _qualname is its name
• If a module is defined in a Python module (i.e. most of the ones we’ve made to date), it has both a name and a pymodule, and goes through that last line of defpath . (Also already there)

Support for the NoConn type is not yet implemented in elaboration and netlisting.
The issue: implement it.

• This will likely be part of the ImplicitSignals elaboration step.
• NoConns with the optional name field should honor this name.
• (Likely more common) un-named NoConns should be named per their port connections
• NoConns which are connected to more than one thing, presumably via port-to-port connections, should probably generate an error.

Example of the latter case:

@h.module 
class Inner:
  p = h.Port()

@h.module
class Bad:
  # Create two instances of `Inner`
  i1 = Inner() 
  i2 = Inner()

  # Connect the first to a `NoConn`
  i1.p = h.NoConn()
  # And then connect the second to the first. 
  # Herein lies the "problem"
  i2.p = i1.p 

I like requiring that we use it, so files aren't getting reformatted as we pass them around.
I less like the CI failures due to forgetting to run it.

Their recommend pre-commit setup is detailed here:
https://github.com/psf/black/blob/main/docs/integrations/source_version_control.md

Once upon a time, I wrote this long and detailed comment on some of the core choices for the Prefixed type:

"""
# Note on Numeric Types 

`Prefixed` supports a sole type for its `number` field: the standard library's `Decimal`. 
One might wonder why it does not include `int` and `float` as well, or perhaps 
some other numeric-like types. 

This boils down to limitations in validation. Previous versions of `Prefixed` have 
used a `Number` union-type along these lines:
`
Number = Union[Decimal, float, int]
`
This is nominally handy, but results in many values being converted among these types, 
often where one may not expect. 
The pydantic docs are clear about their limitations in this respect: 
https://pydantic-docs.helpmanual.io/usage/types/#unions

These limitations include the fact that despite being declared in list-like 
"ordered" syntax, union-types do not have orders in the Python standard library. 
So even interpreting `Union[Decimal, float, int]` as "prefer Decimal, use float 
and then int if that (for whatever reason) doesn't work" fails, 
since the `Union` syntax can be reordered arbitrarily by the language. 

The other clear alternative is not doing runtime type-validation of `Prefixed`, 
or of classes which instantiate them. Prior versions also tried this, 
to fairly confounding results. Incorporating standard-library `dataclasses` 
as members of larger `pydantic.dataclasses` seems to *work* - i.e. it does not 
produce `ValidationError`s or similar - but ultimately with enough of them, 
triggers some highly inscrutable errors in the standard-library methods. 

So: all `Decimal`, all `pydantic.dataclasses`.
"""

Turns out we failed to apply that to many of the Primitive parameters.
In particular they use a union-type called ScalarParam, which is:

# Type alias for many scalar parameters
ScalarParam = Union[Prefixed, int, float, str]

Which, as that comment indicates, turns to int lots of times that you don't want.
@aviralpandey found this in his ADC work. As with most of these subtle translation bugs, it appeared a pain to track down.

@vighneshiyer pointed out (some time ago now) that Poetry now supports the feature we really wanted it to, when we ditched it: editable dev-mode installs.

I think it's worth bringing back, when we have the chance.

Originally noted by @aviralpandey in the context of Primitive exports.
Simple test case:

def test_rountrip_external_module():
    """ Test round-tripping `ExternalModule`s between Hdl21 and VLSIR Proto """

    @h.paramclass 
    class P: # Our ExternalModule's parameter-type
        a = h.Param(dtype=int, desc="a", default=1)
        b = h.Param(dtype=str, desc="b", default="two")

    E = h.ExternalModule(name="E", port_list=[], paramtype=P)

    @h.module 
    class HasE:
        e = E(P())()

    exported = h.to_proto(HasE)
    imported = h.from_proto(exported)
    h.to_proto(imported.hdl21.tests.test_exports.HasE)

Fails with:

        if not is_dataclass(params):
            msg = f"Invalid Parameter Call-Argument {params} - must be dataclasses"
>           raise TypeError(msg)
E           TypeError: Invalid Parameter Call-Argument {'a': 1, 'b': 'two'} - must be dataclasses

hdl21/proto/to_proto.py:304: TypeError
=============================== 1 failed, 90 passed, 3 xfailed in 5.80s ===============================

(While in this case that’s “1.5 round trips”, if we had the VLSIR proto first, importing and exporting would cause it in “1 round trip”.)

Hdl21 requires each port on each instance be connected.
For intentionally unused ports, Hdl21 then has a NoConn special-Signal type which indicates "don't worry about it".

Currently this is only supported for scalar, width-one Signals.
We should make it work for a broader range of cases, including:

• Non unit width Signals
• Bundle-valued ports

Example use case:

    @h.module
    class Inner():
        # Inner module with a handful of port-types
        s = h.Port()
        b = h.Port(width=11)
        d = h.Diff(port=True)

    @h.module 
    class Outer:
        i = Inner(s=h.NoConn(), b=h.NoConn(), d=h.NoConn())
    
    h.elaborate(Outer)

• Marker more specific issues we wanna cover before releasing.
• Track actually releasing.

"Policy" questions on what the no-connect Signal-type should and should not allow.

The initial implementation from #7 has:

• (a) "Strict" single-port, no other connection to anything requirements
• (b) Connects to Signals, not Bundles

Questions and proposals:

• Should they be allowed to connect non-unity-width ports?
  • Propose yes. This is part of #33.
• Should they be allowed to connect to Bundle valued ports?
  • Propose yes. This is part of #33.
• Should they be allowed in Concat?
  • Propose yes
  • I suppose the weird part would be the inference of widths, given the answers above. The most clear, I guess, would be inferring each NoConn as unit-width(?).
  • This may be a "no for now", just make a Signal instead, especially for non-unit widths.
• Should they be allowed in AnonymousBundle?
  • Propose yes
• Should they be Slice-able?
  • Propose no
• Should stuff like the test in #7 work, or fail:

@h.module 
class Inner:
  p = h.Port()

@h.module
class Bad:
  # Create two instances of `Inner`
  i1 = Inner() 
  i2 = Inner()

  # Connect the first to a `NoConn`
  i1.p = h.NoConn()
  # And then connect the second to the first. 
  # Herein lies the "problem"
  i2.p = i1.p 

Currently this fails.
The rationale: NoConn as compared to an otherwise unconnected Signal is really just a marker, saying "I intend for this to connect nowhere". Hdl21's port-to-port connection semantics can then override this intent.

I think this should continue to fail.

The issue: they don't work, yet.

Example test:

def test_anon_with_portref():
    """ Test an AnonymousBundle referring to a PortRef"""

    @h.bundle
    class B:
        s = h.Signal()

    @h.module
    class HasBPort:
        b = B(port=True)

    @h.module
    class HasSignalPort:
        s = h.Port()

    @h.module
    class Outer:
        hass = HasSignalPort()
        hasb = HasBPort(b=h.AnonymousBundle(s=hass.s))

    h.elaborate(Outer)

Task: add an elaboration pass which checks for the directional validity of port-connections.

Each port has a four-value enumerated PortDir attribute direction:

class PortDir(Enum):
    """ Port-Direction Enumeration """

    INPUT = 0
    OUTPUT = 1
    INOUT = 2
    NONE = 3

References on the specific rules abound for Verilog, VHDL, Virtuoso, etc.
Hdl21 rules may require some combination of these (and some further debate) as:

• (a) Hdl21 targets (as a, if not primary goal) exporting to Verilog and similar signal-flow HDLs, but
• (b) Hdl21 modules target "analog-ier" circuits, which often have different (and looser) definitions of which directions "work together"

Example: current summing, as in a DAC. Specifying each unit-cell's output as PortDir.OUTPUT seems the highest designer-clarity. But wiring an array of such outputs together generally requires special handling for signal-flow languages such as Verilog.

There will likely be plenty of other similar trade-offs to be found.

There's probably a broader set of arithmetic operations we should survey and make sure the Prefixed type does.
One that's come up in design work: they multiply against built-in numeric types, but not against themselves.

We have a unit test test_spice_netlister, which checks text netlist output to look like this:

        .SUBCKT DUT 
        + a_4 a_3 a_2 a_1 a_0 b_4 b_3 b_2 b_1 b_0 
        + * No parameters

        rres 
        + a_0 b_0 
        + 10000.0 
        + * No parameters


        ccap 
        + a_1 b_1 
        + 1e-11 
        + * No parameters


        lind 
        + a_2 b_2 
        + 1e-08 
        + * No parameters


        .ENDS

It changed to this:

.SUBCKT DUT 
+ a_4 a_3 a_2 a_1 a_0 b_4 b_3 b_2 b_1 b_0 
* No parameters

rres 
+ a_0 b_0 
+ 10000.0 
* No parameters


ccap 
+ a_1 b_1 
+ 1e-11 
* No parameters


lind 
+ a_2 b_2 
+ 1e-08 
* No parameters


.ENDS

That change may be hard to spot - it's to these lines:

        + * No parameters

Particularly, the + continuation got dropped.
Now there happens to be nothing after them on the instance declaration, and they only write comments. So the "semantic netlist" remains the same. But I can't really see why this is happening.

Report that this error:

RuntimeError: Invalid generator call signature for create_module: [<Parameter "amp_specs">, <Parameter "gen_params">, <Parameter "amplifier_op">, <Parameter "tail_op">]

Was not sufficiently clear.
And I agree. It should at least say what the right call-signature would be. E.g.

RuntimeError: Invalid generator call signature for generator function `create_module`. 
  Generators must be of the form `def GenFunc(params: Params) -> Module`, where `Params` is a `paramclass`
  `create_module` has the arguments: [<Parameter "amp_specs">, <Parameter "gen_params">, <Parameter "amplifier_op">, <Parameter "tail_op">]

Hdl21 elaboration includes an "orphanage" checker that prevents Modules from "stealing" each other's attributes.

For example, from its docs:

    m1 = h.Module(name='m1')
    m1.s = h.Signal() # Signal `s` is now "parented" by `m1`

    m2 = h.Module(name='m2')
    m2.y = m1.s # Now `s` has been "orphaned" (or perhaps "cradle-robbed") by `m2`

This probably doesn't go quite far enough. It inspects everything in the Module namespace, which includes all its Signals, Ports, Bundles, Instances, and the like. But it does not dig into:

• Instance connections
• References to ports or bundles

For example this "orphan" signal survives elaboration:

    m1 = h.Module(name="m1")
    m1.p = h.Port() 

    # The "orphan signal" will not be owned by any module
    the_orphan_signal = h.Signal()
    m2 = h.Module(name="m2")
    m2.i = m1(p=the_orphan_signal) # <= Problem's here

    h.elaborate(m2) # Should fail, doesn't currently. 

This generally fails when we attempt to export or netlist such a Module instead, pretty confusingly.

Reaching into sub-bundles to make instance connections fails elaboration.

Minimal unit test:

def test_sub_bundle_conn():
    """ Test connecting via PortRef to a sub-Bundle """

    @h.bundle
    class B1:
        s = h.Signal()

    @h.bundle
    class B2:
        b1 = B1()

    @h.module
    class HasB1:
        b1 = B1(port=True)

    @h.module
    class HasB2:
        b2 = B2()
        h = HasB1(b1=b2.b1)

    h.elaborate(HasB2)

Generates:

self = <hdl21.bundle.Bundle object at 0x7f637ddb2cd0>, key = 'bundle_ports'

    def __getattr__(self, key):
        ns = self.__getattribute__("namespace")
        if key in ns:
            return ns[key]
>       return object.__getattribute__(self, key)
E       AttributeError: 'Bundle' object has no attribute 'bundle_ports'

hdl21/bundle.py:129: AttributeError

Hdl21 Params have had the commented default_factory attribute here, essentially from day one:

@pydantic.dataclasses.dataclass
class Param:
    """ Parameter Declaration """

    dtype: Any  # Datatype. Required
    desc: str  # Description. Required
    default: Optional[Any] = _default  # Default Value. Optional

    # Default factories are supported in std-lib dataclasses, but "in beta" in `pydantic.dataclasses`.
    # default_factory: Optional[Any] = _default  # Default Call-Factory

This should be fixed in Pydantic by now, and should be enable-able for Hdl21.
Note such updates sit behind #15.

To do: add Primitive elements for the SPICE / VLSIR controlled sources

• VCVS
• VCCS
• CCCS
• CCVS

These should all be very similar to the VLSIR primitives defined here.

It's pretty common to generate errors in hierarchical designs due to failing to set port visibility, or port direction.

Examples:

import hdl21 as h 

@h.module 
class Child:
  x = h.Signal() # Should be a port

@h.module
class Parent:
  x = h.Signal()
  c = Child(x=x) # Fails

This is probably even more less obvious for Bundle valued ports:

import hdl21 as h 

@h.bundle
class SomeBundle:
  ... # Whatever content 

@h.module 
class Child:
  x = SomeBundle() # Should include `port=True`

@h.module
class Parent:
  x = SomeBundle()
  c = Child(x=x) # Fails

The issue: better error reporting here, recommending that there is a thing named x in the Child module namespace, what it is, and what's likely to be wrong.

Similar story applies for typos in port names. We can offer some guidance on what a likely "right answer" would have been.

"Structured Connections" are types which hold a named set of connectable elements, e.g. Signals and other nested structured-connections. Their utility is primarily that such combinations of connections commonly travel together, in groups of several or many. The collected structure can then be used to represent these connections, rather than each element in its set.

This issue is all about their name. There are a few primary pieces of prior art:

• SystemVerilog uses the term interface. So does VHDL (although I have no prior experience with the latter.)
• Chisel uses Bundle

Hdl21 through version 0.1 uses the (Python capitalization-conventions adopted) Interface.

Using interface for this concept probably has more industry familiarity overall. But much of this is from the verification community. And I find that for new users, the term "interface" has the connotations of "IO interface", i.e. something like "a Module's collected pins". (I would also personally frequently use the term "interface" this way.)

Commits starting with c3912ba (thus far in topic branch) move to the Chisel term Bundle.

Minimal test case:

def test_anon_bundle_refs():
    """ Test adding `BundleRef`s to `AnonymousBundle`s. """
    import hdl21 as h 

    @h.bundle
    class Diff:  # Our favorite Bundle: the differential pair
        p, n = h.Signals(2)

    @h.module
    class HasDiff:  # Module with a `Diff` Port
        d = Diff(port=True)

    @h.module
    class HasHasDiff:  # Module instantiating a few `HasDiff`
        d = Diff()
        # Instance connected to the Bundle Instance
        h1 = HasDiff(d=d)

        # THE POINT OF THE TEST: 
        # Instance with an Anon Bundle, "equal to" the Bundle Instance
        h2 = HasDiff(d=h.AnonymousBundle(p=d.p, n=d.n))
        # Instance with (p, n) flipped 
        h3 = HasDiff(d=h.AnonymousBundle(p=d.n, n=d.p))

    h.elaborate(HasHasDiff)

This is a very common and helpful thing to want to do. (In fact it was very much this use-case that led me to the problem.) But, it'll need a few additions to support.

Currently fails with:

E           TypeError: Invalid Bundle attribute for `<hdl21.bundle.AnonymousBundle object at 0x7f40ef9cad90>`: `<hdl21.bundle.BundleRef object at 0x7f40ef9cafa0>`

Should require:

• Add a place to store the BundleRefs on AnonymousBundle. There is an unused field refs which I believe was intended for this.
• Add elaboration handling to resolve the BundleRefs to Signals. The BundleFlattener pass appears the natural place, and does something similar for the existing Signal and BundleInstance members of each AnonymousBundle.

The sim decorator has a bit of documentation that is just, well, wrong.
Particularly here:

    import hdl21 as h
    from hdl21.sim import *

    @sim
    class MySim:
        # ... 

        # Non-`SimAttr`s such as `a_path` below will be dropped from the `Sim` definition,
        # but can be referred to by the following attributes.
        a_path = "/home/models"
        _ = Include(a_path) # <= This guy here is the problem 
        _ = Lib(path=a_path, section="fast") # <= Because of the next line

The intent of this comment is that while some Sim attributes don't seem to need names, they do need to be assigned into the class-body, so that we can store them. Just creating a SimAttr in the class-body is not enough, as in this example:

    @sim
    class MySim:
        Include("/my/path") # This `Include` "goes nowhere", it just gets dropped when this line finishes executing. 

Here the Include "goes nowhere", it just gets dropped when its line finishes executing.

...And that is also kinda what happens in our documented example - just on the following line. Problem is the class-decorator function does not run until after the entire class-body executes. The decorator function effectively gets a dictionary of {name: attribute} pairs as of the end of class-body execution. So in this case, that includes the item {_ : Lib(path=a_path, section="fast"). The Include with key _ is overridden one line after it is created.

Something similar can and does happen if any other name is used instead:

    @sim
    class MySim:
        # ... 
        foobar = Include(a_path) 
        foobar = Lib(path=a_path, section="fast") # Same problem

And really, for any of our class-body-decorator functions, e.g. module:

    @module
    class MyModule:
        whatever = Instance(of=SomeOtherModule) # I just disappear
        whatever = Signal() # Right here

This is part and parcel of using the class-body definition mechanism.

That said, we do make a special case for the conventional "forget me" attribute name _, at least in the sim decorator:

        elif is_simattr(val):
            # Add to the sim-attributes list
            # Special case Python's conventional "ignored" name, the underscore.
            # Leave attributes named "_"'s `name` field set to `None`.
            if key != "_":
                val.name = key
            attrs.append(val)

This essentially says "annotate the SimAttr with the key as its name, unless that key is _".
Many SimAttrs don't have names which don't really matter, including the Include and similar above. I suppose it's possible one would want a means to not set them, and leave them as their default values. This could be done by expanding the test to "annotate anything which does not start with an underscore". E.g. these two attributes would remain nameless:

    @sim
    class MySim:
        # ... 
        _foobar1 = Include(a_path) 
        _foobar2 = Lib(path=a_path, section="fast") 

But at minimum, gotta fix the docs.

I find myself using many "plural" Signal, Port, and similar declarations.
Example from our README:

import hdl21 as h

@h.module
class MyModule:
    a, b = h.Inputs(2)
    c, d, e = h.Outputs(3, width=16)
    f, g, h, i = h.Signals(4)

The one facet I've never loved is having to add the "number of signals" argument, as it doesn't immediately appear clear what those numbers ((2,3,4) here) should mean.

Proposal: make int * Signal multiplication do essentially the same thing.
The same result would be achievable with:

import hdl21 as h

@h.module
class MyModule:
    a, b = 2 * h.Input()
    c, d, e = 3 * h.Output(width=16)
    f, g, h, i = 4 * h.Signal()

I think this may be more clear.
It would work very much like how Hdl21 InstanceArrays are typically generated, via int * Instance.

And given Hdl21 has no ambitions to capture behavioral code in which int * Signal would produce a value of the Signal instead - e.g. during a "procedural block" - I don't think it would be in the way of anything else.

The class-based Sim definitions have a bit different rules from Module and Bundle.
Wondering whether we should move to them.

The Parts in Common

Each of Hdl21's class-decorated types Module, Bundle, and Sim are really organized containers of a small set of types. Module stores signals, instances, etc; Sim stores simulation attributes such as analyses and options. Class attributes of other types - e.g. integers, TabErrors, flask.Apps, anything - are "not allowed". The decorator functions make special exemptions for fields named with the Python language convention for private attributes, the preceding underscore.

The Differences

The differences lie in what happens when encountering any of these unsupported types in the class body.

Module and Bundle immediately throw errors. E.g.:

@h.module
class HasInt:
  i = 5
  s = h.Signal(width=i)

@h.module
class HasMethod:
    def fail(self):
        ...

@h.bundle
class HasTabError:
    tab_error = TabError()

All produce RuntimeErrors.
Note these are all allowed if their attribute names are prepended with underscores, e.g.:

@h.module
class HasInt:
  _i = 5 # Now I'm "private"
  s = h.Signal(width=_i) # And this works

The sim decorator, in contrast, does not immediately throw an error. It allows these values to be created, and referred to by later ones. But it does not store them on the ultimate Sim definition. E.g.

@h.sim
class SimHasInt:
  t = 5e-9 
  tran = Tran(tstop=t) # This works

Here the attribute t can be referred to in the attribute tran. But the Sim object ultimately returned will not include t, only tran. I.e.:

assert isinstance(SimHasInt.tran, Tran) # "Passes" (kinda)*
SimHasInt.t # Fail! AttributeError

Asterisk: Sim doesn't have the getattr-magic lookups of this sort. But the point is that SimHasInt does include a Tran object named tran.

Potential Third Way

The obvious alternative would be to just store everything, of any type, however the user wants.
I don't want to do this.
I want to maintain each of the structured Hdl types as strictly typed containers, in which Hdl21 knows all of the types, their relationships, and where to find all the attributes. We do lots of manipulating these attributes behind the scenes - such might be a description of the entire "elaboration" process. And I don't want to either (a) burden the users with organizing them, or (b) search the entire namespace for each one.

Proposal

I think I like the Sim way best.

@vighneshiyer reported in #10:

I did have some issues running the tests on Python 3.10. We should add a CI target for it. The pytest version needs to bumped and there are a few RecursionErrors that pop up from pydantic.

Give a test run or two from PyPi with the released VLSIR bindings and tools, once they are released.
Required for #53.

Updating to pydantic v1.9 generates a handful of errors along these lines:

_____________________________________ test_slice_resolution _____________________________________

    def test_slice_resolution():
        """ Test resolutions of slice combinations """
    
        from hdl21.elab import _resolve_slice
    
        # Slice of a Signal
        s = h.Signal(width=5)
        assert _resolve_slice(s[0]) == s[0]
    
        # Slice of a Concat
        s1 = h.Signal(name="s1", width=5)
        s2 = h.Signal(name="s2", width=5)
        c = h.Concat(s1, s2)
        assert _resolve_slice(c[0]) == s1[0]
        assert _resolve_slice(c[1]) == s1[1]
        assert _resolve_slice(c[2]) == s1[2]
        assert _resolve_slice(c[3]) == s1[3]
        assert _resolve_slice(c[4]) == s1[4]
        assert _resolve_slice(c[5]) == s2[0]
        assert _resolve_slice(c[6]) == s2[1]
        assert _resolve_slice(c[7]) == s2[2]
        assert _resolve_slice(c[8]) == s2[3]
        assert _resolve_slice(c[9]) == s2[4]
    
        # Slice of a Slice
        sa = h.Signal(name="sa", width=5)
>       assert _resolve_slice(sa[2:5][0]) == sa[2]

hdl21/tests/test_hdl21.py:1777: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
hdl21/signal.py:110: in __getitem__
    return _slice_(parent=self, key=key)
hdl21/signal.py:50: in _slice_
    return Slice(
<string>:9: in __init__
    ???
pydantic/dataclasses.py:98: in pydantic.dataclasses._generate_pydantic_post_init._pydantic_post_init
    ???
pydantic/main.py:1022: in pydantic.main.validate_model
    ???
pydantic/fields.py:854: in pydantic.fields.ModelField.validate
    ???
pydantic/fields.py:1064: in pydantic.fields.ModelField._validate_singleton
    ???
pydantic/fields.py:854: in pydantic.fields.ModelField.validate
    ???
E   RecursionError: maximum recursion depth exceeded while calling a Python object
!!! Recursion detected (same locals & position)
==================================== short test summary info ====================================
FAILED hdl21/tests/test_hdl21.py::test_array_concat_conn - RecursionError: maximum recursion d...
FAILED hdl21/tests/test_hdl21.py::test_slice_resolution - RecursionError: maximum recursion de...

This seems to be part of what was observed in #14.

For now the fix is to change from:

[tool.poetry.dependencies]
pydantic = "^1.8.2"

("Compatible with 1.8.2"), to:

[tool.poetry.dependencies]
pydantic = "1.8.2"

("Exactly 1.8.2")

We could sure use:

• 3 terminal Resistor
• 3 terminal Cap
• 3 terminal Inductor
• Controlled voltage source (VCVS/ CCVS)
• Controlled current source (VCCS/ CCCS)

Generator-created parametric Modules get scalar-string-valued names based on a few rules:

• If each parameter is scalar-valued, and the list is short enough, they use a "list of parameter-values" style.
  • E.g. a Generator Foo with parameters {bar=1, baz=2} would be named something like Foo_bar_1_baz_2
  • This is similar to how parametric Verilog modules are typically named after parameter-flattening.
• For nested and/or longer parameters, the generator parameter-object are JSON-serialized and passed through the standard library's MD5 hashing implementation, with the "security" feature disabled. The combination of the std-lib JSON and "not for security" setting ensures consistent hashing.
  • The same module Foo would then be something like Foo_3f299fec0724c95642e2646e69942115, where the latter-half is the hash-digest of its parameter-values.

... Except it's not quite that. It's generally more like Foo_FooParams_3f299fec0724c95642e2646e69942115__ - where the param-class name FooParams also gets injected.

This issue: get the param-class name outta there. It's just more clutter. It doesn't add info - the Generators and their paramclasses map one-to-one. (And above all I just don't like it.)

The signature for hdl21.pdk.compile is:

def compile(src: vlsir.circuit.Package) -> vlsir.circuit.Package:

Which uses VLSIR's circuit Packages as both input and output types.
While well-intentioned, this has made it harder to use than necessary within Hdl21.

Plus, all existing pdk.compile methods to date just import their content into Hdl21, operate on it, and then export it back. For example from the sample PDK:

def compile(src: vlsir.circuit.Package) -> vlsir.circuit.Package:
    """ Compile proto-Package `src` to the Sample technology """
    ns = h.from_proto(src)
    SamplePdkWalker().visit_namespace(ns)
    return h.to_proto(ns)

This issue/ proposal: just use Hdl21 types instead.

Which types? Generally the things that are "elaboratable" by Hdl21:

• Module and GeneratorCall
  • Maybe(?) ExternalModule, although it's not clear what compilation would do with them, besides return them as-is
  • Again maybe(?) Primitive(Call), returning either mapped ExternalModule(Call) or the input primitive as-is
• Lists (or other sequences) thereof

The proposed pdk.compile would then return objects "of the same shape" as what it was provided. A single Module input returns a single Module, and a sequence thereof would return a sequence of results, in the same order.

Some notes and questions:

• One advantage to using vlsir.circuit.Package is that the hierarchy Walkers know that when traversing a Package, they will see each Module defined exactly once. Repeats are not allowed. Providing a Module, in contrast, will requiring dynamically following its instances and their source Modules. This means the Walkers probably should, or at least would benefit from, caching their results.
• Should this new compile process modify Modules inline, or create new ones? I can see good reasons either way. This is again perhaps a virtue of the existing VLSIR-based method - it produces new Modules and leaves its input unmodifed. Hdl21 elaboration, in contrast, modifes them inline.
  • The latter seems the relevant comp, and generally in line with how Modules are used. Recommend we modify inline.

A feature that has not explicitly made it into the docs -
When you call the Generators/ ExternalModules/ Primitives, you need not explicitly construct the parameter-type. They’ll do it “inline” from keyword args.

So this:

v = Vdc(dc=0 * m, ac=1000 * m)(p=p, n=VSS)

Does the same as:

v = Vdc(Vdc.Params(dc=0 * m, ac=1000 * m))(p=p, n=VSS)

One of the next features on deck will be connecting to individual Instances (and perhaps slices thereof) from InstanceArrays. For example:

@h.module
class M:
  a, b, c = h.Signals(3)
  arr = 5 * OtherM()  # Creates an array 

  # Now connect to its individual Instances
  arr[0].some_port = a
  arr[1].some_port = c
  arr[2].some_port = b
  # Etc

This would frequently be a helpful feature, and one which popular HDLs generally lack.

But it does generate a few policy questions for what constitutes a legal connection. And they're not as easy to concisely state & implement as for (scalar) Instances.

Primarily, what happens when an Instance gets both such a "slice connection" and "array connection". As in:

@h.module
class M:
  w5 = h.Signal(width=5)
  other = h.Signal(width=1)

  arr = 5 * OtherM(some_port=w5) # Creates an array, connects a bit of `w5` to each `some_port`
  arr[0].some_port = other # Then try to connect a scalar Signal to one of its underlying Instances

There are a few policies I can imagine:

• Error Time. Somewhat ambiguous intent such as this generates an error during elaboration.
• "Last-In Wins". Whatever connection-calls are made to whatever layers of array and instance hierarchy, the final one dictates the ultimate connections.
• Tiered Rules. Some prioritization of either individual-instance or array-level connections. Probably prioritizing "more specific" connections, perhaps with a warning message when over-riding less-specific ones.

Last-In

For scalar Instances, "last-in wins" is the effective (and I think, best) policy. For example this works just fine:

@h.module
class M:
  some = h.Signal()
  other = h.Signal()

  inst = OtherModule(some_port=some)  # Initially connect to `some`
  inst(some_port=other)  # Changed our mind, connect to `other` instead

Here the Port some_port on Instance inst is ultimately connected to the Signal other. The short-lived connection to Signal some is effectively ignored.

The implementation of "last-in wins" is both implicit and straightforward: each Instance's connections are kept in a dictionary. Writing new connection key-value pairs just replaces the old ones.

For arrays this wouldn't quite work. Taking the example above:

  arr = 5 * OtherM(some_port=w5) 
  arr[0].some_port = other  

Here arr would be an InstanceArray, and arr[0] would be some other kinda thing yet to be designed. Let's call it InstanceRef. The two would need some means to reconcile "last-in" connection ordering. Presumably the array could keep track of this as a time-ordered list of connections - both to itself at large, and to any constituent members.

Tiering

I could instead imagine a set of rules favoring "more specific" connections rather than "last" ones. This priority would presumably look something like:

1. (Highest Priority) Individual Instance, or single-Instance slice of an array. E.g. OtherM(), arr[0] above.
2. Multi-Instance slices of arrays, e.g. (theoretically) arr[0:1] above.
3. (Lowest Priority) Complete InstanceArrays, e.g. arr above.

I think we will always be able to tell which is which. (I don't think, for example, a Generator could vary whether it connects to an Instance vs a slice across runs.)

Other Language Precedents

The popular HDLs (Verilog, VHDL) do not include the kinds of "procedural connections" under discussion here, and therefore do not offer much insight. Their ethos is generally closest to "Error Time", in that connections must be quite unambiguous to be valid.

I believe Chisel uses the last-in-wins rule in general. I do not know whether it mixes this with anything like the "tiered priority" concept for Instances vs Arrays.

Slice and Concat-valued connections currently hit a NotImplementedError in the generator elaborator.
The issue: implement them.

Hdl21 v1 added the capacity to create generator-parameters "inline", from a set of named keyword args.
For example with the generator M from the test suite:

@h.paramclass
class P:
    a = h.Param(dtype=int, desc="a")
    b = h.Param(dtype=float, desc="b")
    c = h.Param(dtype=str, desc="c")

@h.generator
def M(p: P) -> h.Module:
    return h.Module()

One can of course call M by passing it a P:

m = M(P(a=1, b=2.0, c="3"))

The relatively recent updates add the capacity for Generator to create the parameter-class inline, e.g.:

# Call without constructing a `P`
m = M(a=1, b=2.0, c="3")

Here the P call just adds the three characters P(), and this feature doesn't add much value. In many other cases it's aways more helpful, especially with many elaborately-named parameter-classes in scope.

Clarifying the rules for these calls:

• A single positional argument is allowed, and if provided it must be an instance of the generator's paramclass
• If said positional argument is not provided, Generator will attempt to construct the paramclass from the remaining named keyword arguments. This includes the case in which no arguments are provided, which succeeds if the paramclass is default-constructible.
• Mixing of the positional and named parameters is disallowed, and generates a RuntimeError. Either an instance of the paramclass or a valid set of keyword-args for its construction must be provided - not both.

This has been quite handy for generators.

This issue: add the capacity for our other parameterized Module-like types: ExternalModule and Primitive.

The schema updates on Vlsir/Vlsir#9 have bricked most import and export tests.
Currently:

=== 16 failed, 76 passed, 1 skipped, 4 xfailed in 1.06s ===

Good news is, looks like only import and export tests (and things that use them, like PDKs) were effected.

hdl21/pdk/sample_pdk/test_sample_pdk.py::test_default PASSED             [  1%]
hdl21/pdk/sample_pdk/test_sample_pdk.py::test_compile FAILED             [  2%]
hdl21/pdk/sample_pdk/test_sample_pdk.py::test_netlist FAILED             [  3%]
hdl21/sim/tests/test_sim.py::test_sim1 PASSED                            [  4%]
hdl21/sim/tests/test_sim.py::test_sim2 PASSED                            [  5%]
hdl21/sim/tests/test_sim.py::test_simattrs PASSED                        [  6%]
hdl21/sim/tests/test_sim.py::test_sim_decorator PASSED                   [  7%]
hdl21/sim/tests/test_sim.py::test_tb PASSED                              [  8%]
hdl21/sim/tests/test_sim.py::test_proto1 FAILED                          [  9%]
hdl21/sim/tests/test_sim.py::test_generator_sim FAILED                   [ 10%]
hdl21/sim/tests/test_sim.py::test_delay1 FAILED                          [ 11%]
hdl21/sim/tests/test_sim.py::test_empty_sim1 FAILED                      [ 12%]
hdl21/sim/tests/test_sim.py::test_empty_sim2 SKIPPED                     [ 13%]
hdl21/tests/test_exports.py::test_export_strides XFAIL                   [ 14%]
hdl21/tests/test_exports.py::test_prim_proto1 FAILED                     [ 15%]
hdl21/tests/test_exports.py::test_proto1 PASSED                          [ 16%]
hdl21/tests/test_exports.py::test_proto2 FAILED                          [ 17%]
hdl21/tests/test_exports.py::test_proto3 FAILED                          [ 18%]
hdl21/tests/test_exports.py::test_proto_roundtrip PASSED                 [ 19%]
hdl21/tests/test_exports.py::test_proto_roundtrip2 FAILED                [ 20%]
hdl21/tests/test_exports.py::test_netlist_fmts FAILED                    [ 21%]
hdl21/tests/test_exports.py::test_spice_netlister FAILED                 [ 22%]
hdl21/tests/test_exports.py::test_bad_proto_naming PASSED                [ 23%]
hdl21/tests/test_exports.py::test_generator_recall FAILED                [ 24%]
hdl21/tests/test_exports.py::test_rountrip_external_module FAILED        [ 25%]
hdl21/tests/test_hdl21.py::test_version PASSED                           [ 26%]
hdl21/tests/test_hdl21.py::test_module1 PASSED                           [ 27%]
hdl21/tests/test_hdl21.py::test_module2 PASSED                           [ 28%]
hdl21/tests/test_hdl21.py::test_generator1 PASSED                        [ 29%]
hdl21/tests/test_hdl21.py::test_generator2 PASSED                        [ 30%]
hdl21/tests/test_hdl21.py::test_generator3 PASSED                        [ 31%]
hdl21/tests/test_hdl21.py::test_params1 PASSED                           [ 32%]
hdl21/tests/test_hdl21.py::test_params2 PASSED                           [ 34%]
hdl21/tests/test_hdl21.py::test_params3 PASSED                           [ 35%]
hdl21/tests/test_hdl21.py::test_params4 PASSED                           [ 36%]
hdl21/tests/test_hdl21.py::test_bad_params1 PASSED                       [ 37%]
hdl21/tests/test_hdl21.py::test_array1 PASSED                            [ 38%]
hdl21/tests/test_hdl21.py::test_array2 PASSED                            [ 39%]
hdl21/tests/test_hdl21.py::test_cycle1 PASSED                            [ 40%]
hdl21/tests/test_hdl21.py::test_gen3 PASSED                              [ 41%]
hdl21/tests/test_hdl21.py::test_prim1 PASSED                             [ 42%]
hdl21/tests/test_hdl21.py::test_prim2 PASSED                             [ 43%]
hdl21/tests/test_hdl21.py::test_bundle1 PASSED                           [ 44%]
hdl21/tests/test_hdl21.py::test_bundle2 PASSED                           [ 45%]
hdl21/tests/test_hdl21.py::test_bundle3 PASSED                           [ 46%]
hdl21/tests/test_hdl21.py::test_bundle4 PASSED                           [ 47%]
hdl21/tests/test_hdl21.py::test_bigger_bundles PASSED                    [ 48%]
hdl21/tests/test_hdl21.py::test_signal_slice1 PASSED                     [ 49%]
hdl21/tests/test_hdl21.py::test_signal_slice2 PASSED                     [ 50%]
hdl21/tests/test_hdl21.py::test_bad_slice1 PASSED                        [ 51%]
hdl21/tests/test_hdl21.py::test_signal_concat1 PASSED                    [ 52%]
hdl21/tests/test_hdl21.py::test_slice_module1 PASSED                     [ 53%]
hdl21/tests/test_hdl21.py::test_module_as_param PASSED                   [ 54%]
hdl21/tests/test_hdl21.py::test_mos_generator PASSED                     [ 55%]
hdl21/tests/test_hdl21.py::test_series_parallel_generator PASSED         [ 56%]
hdl21/tests/test_hdl21.py::test_instance_mult PASSED                     [ 57%]
hdl21/tests/test_hdl21.py::test_instance_mult2 PASSED                    [ 58%]
hdl21/tests/test_hdl21.py::test_instance_mult3 PASSED                    [ 59%]
hdl21/tests/test_hdl21.py::test_instance_mult4 PASSED                    [ 60%]
hdl21/tests/test_hdl21.py::test_bad_width_conn PASSED                    [ 61%]
hdl21/tests/test_hdl21.py::test_bad_bundle_conn PASSED                   [ 62%]
hdl21/tests/test_hdl21.py::test_illegal_module_attrs PASSED              [ 63%]
hdl21/tests/test_hdl21.py::test_copy_signal PASSED                       [ 64%]
hdl21/tests/test_hdl21.py::test_copy_bundle_instance PASSED              [ 65%]
hdl21/tests/test_hdl21.py::test_bundle_destructure PASSED                [ 67%]
hdl21/tests/test_hdl21.py::test_orphanage PASSED                         [ 68%]
hdl21/tests/test_hdl21.py::test_orphanage2 PASSED                        [ 69%]
hdl21/tests/test_hdl21.py::test_orphanage3 PASSED                        [ 70%]
hdl21/tests/test_hdl21.py::test_wrong_decorator XFAIL                    [ 71%]
hdl21/tests/test_hdl21.py::test_elab_noconn PASSED                       [ 72%]
hdl21/tests/test_hdl21.py::test_bad_noconn PASSED                        [ 73%]
hdl21/tests/test_hdl21.py::test_array_concat_conn PASSED                 [ 74%]
hdl21/tests/test_hdl21.py::test_slice_resolution PASSED                  [ 75%]
hdl21/tests/test_hdl21.py::test_common_attr_errors PASSED                [ 76%]
hdl21/tests/test_hdl21.py::test_generator_call_by_kwargs PASSED          [ 77%]
hdl21/tests/test_hdl21.py::test_instance_array_portrefs PASSED           [ 78%]
hdl21/tests/test_hdl21.py::test_array_bundle PASSED                      [ 79%]
hdl21/tests/test_hdl21.py::test_sub_bundle_conn XFAIL                    [ 80%]
hdl21/tests/test_hdl21.py::test_anon_bundle_port_conn PASSED             [ 81%]
hdl21/tests/test_hdl21.py::test_multiple_signals_in_port_group PASSED    [ 82%]
hdl21/tests/test_hdl21.py::test_bad_conns PASSED                         [ 83%]
hdl21/tests/test_hdl21.py::test_anon_bundle_refs PASSED                  [ 84%]
hdl21/tests/test_hdl21.py::test_generator_port_slice XFAIL               [ 85%]
hdl21/tests/test_hdl21.py::test_pair1 PASSED                             [ 86%]
hdl21/tests/test_prefix.py::test_decimal PASSED                          [ 87%]
hdl21/tests/test_prefix.py::test_prefix_numbers PASSED                   [ 88%]
hdl21/tests/test_prefix.py::test_prefix_shortname PASSED                 [ 89%]
hdl21/tests/test_prefix.py::test_prefix_from_exp PASSED                  [ 90%]
hdl21/tests/test_prefix.py::test_prefix_mul PASSED                       [ 91%]
hdl21/tests/test_prefix.py::test_e PASSED                                [ 92%]
hdl21/tests/test_prefix.py::test_e_mult PASSED                           [ 93%]
hdl21/tests/test_prefix.py::test_prefix_scaling PASSED                   [ 94%]
pdks/Asap7/asap7/test_asap7.py::test_default PASSED                      [ 95%]
pdks/Sky130/sky130/test_sky130.py::test_default PASSED                   [ 96%]
pdks/Sky130/sky130/test_sky130.py::test_compile FAILED                   [ 97%]
pdks/Sky130/sky130/test_sky130.py::test_netlist FAILED                   [ 98%]
pdks/Sky130/sky130/test_sky130.py::test_module1 PASSED                   [100%]

It's a fairly understandable error for users to misplace similarly-named identifiers, especially those separated solely by a few capitalizations. Module (the class) and module (the decorator-function) serve as prime examples. Currently stuff like this:

@h.Module # <= problem is here 
class M:
  ... 

Generates:

E       TypeError: __init__() takes 1 positional argument but 2 were given

This error could probably be clearer.

Hdl21 / vlsir.circuit's initial netlisting targets have been Verilog and Spectre.
Next on the priority-stack is traditional-format SPICE.
The issue: implement it.

This should all be derivable from the existing content of hdl21/netlist/__init__.py. Tasks would include:

• Create a SpiceNetlister or similar, derived from the base Netlister
• Add it to the enumerated NetlistFormats, and its netlister method
• The primary difference (in addition to overall syntax) will be to Primitives, particularly the "ideal" flavor. Traditional SPICE formats identify their types via instance-name prefixes, e.g. "R1" is a resistor, "M1" is a MOSFET, etc. Existing Spectre-format netlisting has a small table of ideal-primitive module-names. From SpectreNetlister.resolve_reference:

        elif ref.WhichOneof("to") == "external":  # Defined outside package

            # First check the priviledged/ internally-defined domains
            if ref.external.domain == "hdl21.primitives":
                msg = f"Invalid direct-netlisting of physical `hdl21.Primitive` `{ref.external.name}`. "
                msg += "Either compile to a target technology, or replace with an `ExternalModule`. "
                raise RuntimeError(msg)
            if ref.external.domain == "hdl21.ideal":
                # Ideal elements
                name = ref.external.name

                # Sort out the spectre-format name
                if name == "IdealCapacitor":
                    module_name = "capacitor"
                elif name == "IdealResistor":
                    module_name = "resistor"
                elif name == "IdealInductor":
                    module_name = "inductor"
                elif name == "VoltageSource":
                    module_name = "vsource"
                else:
                    raise ValueError(f"Unsupported or Invalid Ideal Primitive {ref}")

Something like this will need to occur in instance-generation (write_instance) as well. A reasonable (if old) reference on the prefix-characters is available on Berkeley's own BWRC server.

Of late we realized problems with many of the Primitive parameters, and swapped them to the Prefixed type. They had previously been something like this Scalar type-union:

Scalar = Union[Prefixed, str, float, int]

Which looks good at first blush, but hits a bunch of problems detailed in #(FIXME: issue number)!.

That change inadvertently killed the ability to refer to simulation parameters, e.g.

    @h.module
    class Tb:
        VSS = h.Port()  # The testbench interface: sole port VSS
        dut = Dut(s=VSS, b=VSS)
        # Problem here: attempting to sweep an instance parameter:
        vd = Vdc(Vdc.Params(dc="vds"))(p=dut.d, n=VSS)
        # ...

    sim = Sim(tb=Tb)
    vds = sim.param(name="vds", val=1800 * m)
    dc = sim.dc(var=vds, ...)

The two things I can think to include as primitive param-types:

• Reinstate the str variant, or
• Add an explicit reference to a sim.Param

I think either would "play nice", in that it would be unambiguous for any constructor value which variant should be used. I think.

Originally came up as part of #45, along with the realization that most Primitive parameters should be of type Prefixed.
It would be nice to be able to "inline" convert to Prefixed from built-in primitive types, especially int. So that stuff like:

@h.paramclass 
class Params:
  num = h.Param(dtype=h.Prefixed, desc="watch")

Params(num = Prefixed(1000, MILLI)) # Works
Params(num = 11 * µ) # Works

Params(num = 1) # <= this here

Would work.

I filed a Pydantic issue at the time:
pydantic/pydantic#4625

And the author's recommendation was to use their custom data types feature:
https://pydantic-docs.helpmanual.io/usage/types/#custom-data-types.

It didn't immediately jump off the page how to fit those together.
This issue: make it work.

How Roles Work Now

• Each Bundle definition (optionally) stores an enumerated set of Roles, by that name.
  • These must be python enums, i.e. instances of EnumMeta.
• Each BundleInstance has an (optional) value from this enumeration named role.
• Signal members of those bundles have source and destination attributes, which use roles to set port directions.

Proposal

• BundleRole becomes a custom type
• Each Bundle definition stores a set of them, which would most commonly be created from a Roles function:

@h.bundle 
class MyBundle: 
  Host, Device = h.Roles(2) 

# Or 

@h.bundle 
class MyBundle: 
  Host, Device = 2 * h.Role() 

• Calling a Role is a shorthand to generate a BundleInstance with that role. E.g.:

@h.module
class HostModule: 
  my_bundle_port = MyBundle.Host()

• The generated BundleInstances would be port-visible by default. An optional visibility argument would override this.

• Sets of roles, tentatively called RoleSets, can be created from Python enums (EnumMeta). This would be the primary means of sharing them between Bundle definitions

class HostDeviceEnum(Enum):
  Host = auto()
  Device = auto()

HostDevice = h.Roles.from_enum(HostDeviceEnum)

@h.bundle
class AnotherBundle:
  roles = HostDevice

@h.bundle
class YetAnotherBundle:
  Roles = HostDevice # Capital R also works

• The naming convention for roles would be that of typical Python classes, i.e. camel-case.

At various points they "look like"

• (a) Constructors for Bundle instances,
• (b) Class attributes,
• (c) Enum variants

Each has a different recommended, idiomatic naming convention. The one that I'd prefer to "stick" (mentally) is probably (a), for creation of bundle instances with each role.

This here lets the test suite fail with Python v3.10:

    continue-on-error: ${{ matrix.dep-installer == 'pypi' || matrix.python-version == '3.10' }}

Probably a holdover from before we really worked with 3.10.
The issue: remove that, make sure everything still works.

To date, this does not work:

import hdl21 as h
from hdl21.primitives import R 

@h.bundle
class Diff:
    p, n = h.Signals(2)

@h.module
class HasDiffRes:
    d = Diff()
    VSS = h.Port()
    # This here gonna be the problem, particularly the `Concat`:
    rs = 2 * R(R.Params(r=1))(p=h.Concat(d.p, d.n), n=VSS)

As of current dev (e17e5d), the error is in checking that the total width of the concatenation is positive:

class Concat:
    """ Signal Concatenation """

    def __init__(self, *parts):
        # ...
        if self.width < 1:
            raise ValueError(f"Invalid Zero-Width Concat of {self.parts}")

And this produces:

AttributeError: 'BundleRef' object has no attribute 'width'

While fixing that may not be so bad, some further outlook:

• It's unlikely this will be the only thing to go wrong.
• While not tested here, PortRefs are almost certain to fail similarly to BundleRefs.
• While also not tested here, Slice is also almost certain to fail similarly to Concat.

The Ref types to Bundle and Port generally "resolve late", during elaboration. Their width is often not known at construction time, and may require, for example, a Generator to run to be determined. Slices and Concats in contrast do at least some of their work upfront: checking for valid widths, checking for out-of range indices.

For slices and concatenations of these Ref types to work, it seems we will need to "late resolve" them too. All of the checking and validation done at Slice and Concat construction-time could be move to an elaboration pass, perhaps the existing SliceResolver.

The signature for hdl21.netlist is:

def netlist(
    pkg: vlsir.circuit.Package, dest: IO, fmt: NetlistFormatSpec = "spectre"
) -> None:

This means in order to netlist, one must first convert an hdl21.Module (or container thereof) to vlsir.Package, generally via hdl21.to_proto. Calls then generally look something like:

h.netlist(h.to_proto(MyModule), dest=sys.stdout)

Where MyModule is an hdl21.Module.

The issue: do this inline. Enable calls like:

h.netlist(MyModule, dest=sys.stdout)

And:

h.netlist([MyModule1, MyModule2], dest=sys.stdout)

To do: re-use elaboration where we can.
This is particularly relevant for "batches" of elaborations, such as many-Module exports and many-corner simulations.

Thus far we tend to elaborate one, forget about it, and then elaborate the next. For example hdl21.sim.run does something like:

def run(inp: Sequence[Sim], ...):
    # ...
    return vsp.sim(inp=[to_proto(s) for s in inp], ...)

That list comprehension is the problem; there is no way for each entry in inp to capitalize on the work done for the ones before it.

In the case of a PVT-variation simulation, the shared content might be 99% of all hardware. E.g. everything being simulated remains the same, and only the top-level testbench or simulation options change.

And, good & bad news is - we've gotten to big enough circuits where Hdl21 elaboration times are starting to matter.

So far we support external modules with parameters, those parameters just have to be key-value only.

I propose we allow external modules to optionally specify their parameters text as they need to be netlisted, and their spice prefix. This will essentially turn all of our primitives into external modules, which could be simpler than our current approach of writing down how to netlist every spice primitive we could ever run into.

So the new external module would look like:

message ExternalModule {
  // Qualified External Module Name
  vlsir.utils.QualifiedName name = 1;
  // Description
  string desc = 2;
  // Port Definitions
  // Ordered as they will be in order-sensitive formats, such as typical netlist formats. 
  repeated Port ports = 3;
  // Signal Definitions, limited to those used by external-facing ports.
  repeated Signal signals = 4;
  // Params
  repeated vlsir.utils.Param parameters = 5;
  // Spice types
  optional string spicetype = 6;
  // Spice Parameter netlist
  optional string paramnetlist = 7;
}

The netlister would then behave the same way it has, but instead of trying to netlist the parameters by mapping them itself, it would simply write the passed paramnetlist.

Structuring and De-Structuring Connections

The "structured connection" type (name-debate in #4, referred to as "bundle" here) is one of Hdl21 and similar libraries' primary productivity-improvement mechanisms for connecting Modules to one another.

For example from the Hdl21 unit tests:

    @h.bundle
    class Jtag:
        roles = HostDevice
        tck, tdi, tms = h.Signals(3, src=roles.HOST, dest=roles.DEVICE)
        tdo = h.Signal(src=roles.DEVICE, dest=roles.HOST)

    @h.bundle
    class Spi:
        roles = HostDevice
        sck, cs = h.Signals(2, src=roles.HOST, dest=roles.DEVICE)
        dq = h.Signal(src=roles.DEVICE, dest=roles.HOST, width=4)


    @h.module
    class Chip:
        spi = Spi(role=HostDevice.HOST, port=True)
        jtag = Jtag(role=HostDevice.DEVICE, port=True)
        ...  # much more internal content

Here Chip.spi and Chip.jtag are bundle-valued Ports, each of which has several underlying (scalar) Signals. Each set of signals commonly travels together, and the bundle-type allows passing them around as a single entity rather than via each of their constituent elements.

Here

This issue is about creating ("structuring") and breaking up ("destructuring") these connection-objects. This will need to happen at both the top and bottom of Hdl21 hierarchies, for a few common reasons:

• At hierarchy-bottom, Modules will eventually need to connect to Primitive elements, which do not include bundle-valued ports. Similarly many ExternalModules created in and instantiated from legacy tools will not include such a concept, and must be resolved to scalar ports.
• At hierarchy-top, exposing an IO interface to said legacy tools (with fully designer-controlled names), while using bundles internally, will require constructing bundles from scalar signals.
• In other cases, different bundles will require "re-wiring" between each other. These may be due to different signal-naming conventions, or to express boundary-changes in functionality (e.g. wiring each tx and rx in a symmetric communications link).

Destructuring

The former is (I think) the easier of the two cases. A dot-access into a Bundle produces a reference to its constituent Signals and/or sub-Bundles. These can then be connected to other instances, like so:

@h.bundle 
class WholeBoardIo:
  spi = Spi()
  jtag = Jtag()

@h.module 
class Board: 
  io = WholeBoardIo(port=True)

  # Connect to a thing called `SpiFlash`, which only uses the `spi` part of `WholeBoardIo`
  flash = SpiFlash(spi=io.spi)
  # ... 

This feature has been supported in Hdl21 essentially since the introduction of the structured-connection (Interface / Bundle) type.

Structuring

How to create bundles from scalar signals is a more interesting policy decision. Considering this case:

@h.bundle
class Diff:
  p = h.Signal()
  n = h.Signal()

@h.module
class Child:
  dport = Diff(port=True)

@h.module 
class Parent:
  p, n = h.Ports(2)
  child = Child() # ... now what? 

Module Parent exposes scalar-valued ports, and instantiates module child with one (or more) bundle-valued ports. (Similar to bullet 2, "top of hierarchy", in section "Here".)

There are a few policies I can imagine:

• Connect a signal at a time
• Create (something) "bundle-valued"
• Connections into Bundle instances
• All of the above

Connecting a Signal at a Time

The "signal at a time" concept would use nested left-hand-side dot-accesses to dictate instance connections. In the case of Parent and Child above, this would look like so:

# In `Parent` class-body 
child = Child()
child.dport.p = p
child.dport.n = n 
# ...

This is very similar to the typical semantics of Chisel connections.

This looks pretty good for the relatively simple case above. Noting however that bundles can be nested, as can bundle-valued ports. So larger cases would look more like:

# A more complicated port-type
child.bundle_port.sub_bundle1.sub_sub_bundle.some_signal = p 

This would then require a "connection rules" policy similar to that discussed in #3. For example, imagine embedding the snippet above in a procedural set of connections, including something like so:

# Several connections to this `child.bundle_port`
child.bundle_port.sub_bundle1   = some_bundle
child.bundle_port.sub_bundle2   = some_other_bundle
child.bundle_port.sub_bundle3   = bundle3      #(1)
child.bundle_port.sub_bundle3.p = some_signal  #(2) Hm, now what? 

Here p is assigned both by commented lines (1) and (2). Our resolution (thus far) to #3 has been "disallow this altogether", i.e. by not enabling the nested LHS assignment-connections.

Create Something Bundle-Valued

The primary alternative is for "instantiators" to create a structured bundle-like object to connect to each bundle-valued port. In the (more elaborate) case above in which child has a port named bundle_port, this would look something like:

child.bundle_port = Something( # Creating the `Something`-bundle-like object 
  sub_bundle1 = some_other_bundle, 
  sub_bundle3 = Something ( # Nesting another `Something`
    p = some_signal, 
    # everything else comes from `bundle3`
  )
)

The generally anonymous Something here is very much like Bundle, but has an anomyous bundle-type. These are essentially BundleInstances of anonymous types created at instantiation time. Instances then have a single connection per-port, including for bundle-valued ports.

Connections into Bundle Instances

SystemVerilog's interface instead exposes select signals as externally-connectable ports, while maintaining other signals as internal and private. Its interfaces are more like "modules without instances" in this sense. An example from https://www.doulos.com/knowhow/systemverilog/systemverilog-tutorials/systemverilog-interfaces-tutorial/:

interface ClockedBus (input Clk);
  logic[7:0] Addr, Data;
  logic RWn;
endinterface

module RAM (ClockedBus Bus);
  always @(posedge Bus.Clk)
    if (Bus.RWn)
      Bus.Data = mem[Bus.Addr];
    else
      mem[Bus.Addr] = Bus.Data;
endmodule

// Using the interface
module Top;
  reg Clock;

  // Instance the interface with an input, using named connection
  ClockedBus TheBus (.Clk(Clock));
  RAM TheRAM (.Bus(TheBus));
  ...
endmodule

Here ClockedBus has an external input-signal Clk and several other internal, private signals. The top instance of ClockedBus in module Top gets (requires?) an external signal tied to Clk. The ClockedBus-valued port Bus of module RAM in contrast does not take such an external connection.

While admitting I don't know these connection-rules completely, I don't love this method.

Status

Commits as of af2c3ce (as of this writing, head of dev) include the "Something" bundle-valued concept, in which this type is named AnonymousBundle. It does not include nested LHS assignments, nor SystemVerilog-style connections into BundleInstances.

I do not believe AnonymousBundle should be the final name. But also don't have a better one immediately at hand. "Bundle" in contrast is used for the bundle-type definitions.

This AnonymousBundle implementation does not support connect-by-assignment, or replacement of connections by assigment. All connections are made at creation-time. This avoids needing support for a "nested connection" policy as outlined here and in #3, while perhaps making some implementations more cumbersome.