waterloo-rocketry / topside Goto Github PK

Steps to reproduce:

1. Load the default procedure and plumbing engine.
2. Advance to step 2 using the procedure controls.

SC_001 is now open (can be tested by looking at DAQ), but the component state displayed in the GUI is still closed. When a redraw is triggered (by resizing the window or dragging the split handles), the state updates immediately, but it should update as soon as it is changed.

This issue has two parts:

1. Add a function to the ProceduresBridge that, when given a step ID, fast-forwards the plumbing engine until the resulting step is reached.

2. Add a control to the UI that invokes this function.

Part 2 is likely to be a bit more complicated - I think we'll want to add a button to each step, or have it as a right-click option, or something like that. For now, let's just focus on implementing Part 1 - for actually controlling it in the UI, let's just add a small text entry field with a button.

For Part 1, you'll want the function to be structured roughly like this:

while (current step != target step) and (termination criteria not met):
    step time
    go to next step if conditions satisfied

For "termination criteria", there are a few heuristics that we could use. For now, let's just start by checking that the procedure isn't over (the current step has at least one successor).

Currently, there's no way to comment a ProcLang document, which means we can't add any rationale for any of the decisions we make. It would be nice if we could do something like this:

main:
    1. set series_fill_valve to closed
    2. [5s] set supply_valve to open
        - [p1 < 600] abort_1.1  # if the pressure is too low we need to wait for the tank to heat up
        - [p1 > 1000] abort_2.1
    3. set series_fill_valve to open

In the above snippet, we've added a comment to the first deviation from step 2, explaining why we need to abort.

In order to implement this, we need to add a "comment" rule to the grammar specified in topside/procedures/proclang.py. This might require a bit more work than some of the other rules, because theoretically we would like to be able to add a comment anywhere in a proclang string. Perhaps we will need to drop the %ignore WS directive?

Add support for specifying default values for the parameters specified in PDL typedefs. A rough list of what this would probably entail:

• coming up with an unambiguous syntax to specify a default argument in PDL. Could be modeled off existing syntax, for example what Python does (parameter=default_value in the definition)
• storing default arguments in Files and Packages
• logic in File component validation to only throw an error if a parameter isn't present and it doesn't have a default value specified
• logic to use user-provided values if they're present, default values if not in the typedef-replacement stage of Package validation

A pressure relief valve is a valve with two states (open and closed). The valve is open if the pressure across it is greater than some value (called the set point), and closed otherwise. We currently don't have any way to model this behaviour in the plumbing engine, as plumbing components can't change their states independently.

Right now, a procedure step has info about the action carried out and what to do next. This lets us write procedures that look like this:

main:
    1. set series_fill_valve to closed
    2. [5s] set supply_valve to open
        - [p1 < 600] abort_1.1
        - [p1 > 1000] abort_2.1
    3. set series_fill_valve to open
    4. set remote_fill_valve to open

The "real" procedures that we want to eventually replicate look like this:

Currently, we have no way of annotating that a specific procedure step should be performed by a specific member of the launch team (OPS, PRIMARY, SECONDARY, etc.). Adding this information won't affect anything about how the procedure is executed by Topside, but we want to be able to display it and include it in the ProcLang.

Work required:

1. In topside/procedures/procedure.py, add a new field to the ProcedureStep class for "operator" or something similar.
2. In application/procedures_bridge.py, change the ProcedureStepsModel to return data from the new field instead of the hardcoded "Primary Technician"

We can handle adding it to the ProcLang as a followup issue.

Currently, a procedure/component/node name in ProcLang needs to follow these rules:

• it must be at least 1 character
• it can only contain letters, numbers, and underscores
• it must start with a letter or number

It would be nice to add hyphens as well, in case we want to name a valve something like O-MV-3.

Work required:

• Add hyphens to the STRING terminal in the ProcLang grammar specification (proclang.py)
• Add unit tests to test_proclang.py to confirm that the new names are parsed properly

The Mac PyInstaller build needs some fine-tuning for features specific to Apple if we want to be able run the executable on Mac, specifics are laid out here. The summary is that:

• apps need to be signed by Apple's notary service to run in Catalina,
• the notary service won't sign unless we have Hardened Runtime enabled, and
• in order for the executable to run with Hardened Runtime, we need to add certain entitlements.

Having a working build for Mac isn't a huge priority right now, but this is something that we'll want to do eventually.

Our fancy colour-coded procedures are currently generated with LaTeX (template here), and if we want to recreate them it makes sense to start by cross-compiling. We have most of the information available in the ProcedureSuite class already.

Currently, only WaitUntil and Comparison are supported ProcLang conditions. We would like to be able to write conditions like [p1 < 100 and (p2 > 200 or p3 < 500)].

Work required for this:

• Add and and or to Lark grammar
• Write ProcedureTransformer methods for handling new comparisons

Right now, Topside always starts up without anything loaded, and if I want to open a ProcLang file I need to open it from the menu. It would be nice if Topside, when closing, remembered what the last open file was, and then when opening, loaded the same file at startup.

The best way to do this is probably with QSettings - take a look at this documentation for reference (it's C++, but the Python API should be very similar).

Right now, we define an action as setting a given component to a given state (e.g. Set remote_fill_valve to open). In our real procedures, there are a lot of actions that don't take this form. Some examples:

Here, an action like "Approach the launch tower" doesn't have any direct impact on the fill system, but we still want to be able to list it in our procedure.

Work required:

1. Rename the existing Action class to something more specific (StateChangeAction, maybe?)
2. Add a new class (maybe MiscAction) that holds a string specifying the action text
3. Modify ProceduresEngine.execute to check the type of the action and do nothing if it's a misc action
4. Add code in procedure visualization to handle displaying the new action type

Code for steps 1 and 2 would go in topside/procedures/procedure.py. Step 3 will go in topside/procedures/procedures_engine.py, and step 4 will go in application/procedures_bridge.py.

CI currently only runs the build process for Windows; we should add builds for Mac (and possibly Linux) too.

We should add logging capabilities! It would be nice to intersperse an organized way of recording actions throughout Topside, plus spelunking through logging code is a good way to get familiar with how the different parts of the code interact for any newcomers. Stuff like rate-limiting logs makes for a good first issue, imo :).

#67 added procedures engine support for miscellaneous actions, but we currently don't have any syntax for them in the ProcLang. We want to be able to write ProcLang that contains misc actions so that we can accurately replicate our current procedures.

Work required:

• Add a new misc_action rule to the ProcLang grammar
• Modify the action rule in the ProcLang grammar to accept either a misc_action or an old state change action
• Modify the ProcedureTransformer to correctly parse action nodes and return the appropriate type
• Write unit tests for the above!

We haven't defined __str__ methods for Action, Transition, Step, Procedure, or any of the condition classes, which means that when we print them we get things like this:

<topside.procedures.conditions.GreaterEqual object at 0x0000024844702688>

Action(component='c1', state='open')

ProcedureStep(step_id='s1', action=None, conditions={<topside.procedures.conditions.Immediate object at 0x0000024815FC2748>: Transition(procedure='p1', step='s2')})

Procedure(procedure_id='p1', steps={'s1': ProcedureStep(step_id='s1', action=None, conditions={<topside.procedures.conditions.Immediate object at 0x0000024815FC2748>: Transition(procedure='p1', step='s2')}), 's2': ProcedureStep(step_id='s2', action=Action(component='c1', state='open'), conditions={<topside.procedures.conditions.Immediate object at 0x0000024844DA0448>: Transition(procedure='p1', step='s3')}), 's3': ProcedureStep(step_id='s3', action=Action(component='c1', state='closed'), conditions={})})

This isn't wrong, but when we want to add logging it will definitely be more convenient to have string representations so that it's easier to read.

We should add a demo script that combines the solving algorithm and layout to show what exactly the engine is supposed to do.

We don't know right now where the bottlenecks in the plumbing engine code are and how much speedup we can stand to get, but I think step may possibly cause us a bit of grief if we don't do a bit of optimization - the function is roughly O(N) in the number of nodes, which might not scale well to the number of nodes we typically have in our P&IDs.

We should spend some time profiling the PlumbingEngine to see what the hot paths are and where we can potentially save some time.

In general, we implicitly attach a "wait until steady state" condition to all of our procedure steps when we run through manually. In some cases (e.g. tank filling), it would be good to encode that logic in the ProcLang directly.

We can parse ProcLang and turn it into a procedure suite, but we can't go the other way around yet (turn a ProcedureSuite into a string of ProcLang).

We added personnel info to the ProcedureStep class in #68, but currently we don't have a ProcLang syntax for it (so we currently just hardcode to PRIMARY). It would be nice to be able to specify the operator performing the step in the ProcLang.

Work required:

1. Decide on the ProcLang syntax for operator annotations
2. Add the new syntax to the Lark grammar in proclang.py
3. Add a new handler to the ProcedureTransformer class in proclang.py
4. Add some unit tests to make sure it works

Right now, the plumbing engine stores time in microseconds but takes input in seconds. Furthermore, both its solve capabilities and the procedures engine are starting to depend on time comparisons and calculations in their functions. We should start using a standardized time library (datetime or our own version) across topside to reduce calculation errors and improve documentation.

Right now, our list of acceptable imports is hard coded in as a "standard library" of sorts. Importable PDL files live in the topside/pdl/imports folder, and the software keeps its record of what imports are allowed in a hard coded dict . This isn't a very flexible implementation and doesn't allow users to load in files that they'd like to import later on, which is functionality that we should support.

A basic version of this shouldn't be difficult to implement (at the package level anyways, actually adding the files is a different matter and might require some restructuring as to where things are stored). An initial idea could be to store a JSON object of {import name: import file path} in a file somewhere, and read it out as part of instantiation for Package (there's room for more sophisticated hashing and caching or whatever to improve efficiency here). We then update the contents of the file every time the list of imports is changed. The actual machinery for adding new imports doesn't necessarily have to be there yet, but it would be good to have import handling that's conducive to adding that feature when the time comes.

The code relevant to this issue lives in topside/pdl/package.py right now. In the same vein, it would be nice to add unused import detection as well (since these are both issues impacting the same code). Definitely more a QoL than a hard usability feature (so not super high priority), but again it would be nice to have.

We don't currently have a way of resetting a plumbing engine to its initial state, which would be useful when navigating through operations procedures (in order to be able to sync procedure resets with plumbing engine resets).

The actual code to reset shouldn't be difficult - likely just resetting some counters and reloading initial states and pressures, which is already done on initialization. It's a good chance to get familiar with the pieces of the plumbing engine though (since you'd need to figure out what exactly needs resetting), and this function is pretty conducive to writing unit tests if anyone wants to take a first crack at that.

(Or decide that we don't actually want to) The topside namespace is getting slightly cluttered, especially as we start building up the other parts of the program we might start getting overlap between names. However, flat is better than nested so we should figure out where to strike the balance between these two.

The code that currently produces the import_folder dict is inline and only able to look at the imports folder where we keep actual imports, which isn't very conducive to unit testing. We should pull it out into a function that takes the path to look for the folder in as an argument, and that also has a mode that yields a fake list of paths to test with. A function signature could look something like this:

def get_imports(folder_path, mockPaths=None)`

where folder_path is the path to a folder, and mockPaths is an optional list of fake paths to be used for unit testing. Depending on the value of mockPaths, the function either trawls the provided folder path for the list of files or uses the given list directly.

We can also ignore files without the .pdl extension in in this function, so that we have more flexibility in what goes into the imports folder.

Instead of parsing the entire imports folder on every instantiation of a Package, we should cache a {name: paths} object in a YAML file, as well as a record of the folder contents this cache is valid for. We can then just check the cache record and actual folder contents against each other on instantiation, and only reparse the entire folder if the cache is stale.

Right now, we display procedures like this:

Ideally, the Greater condition would be displayed much more concise; something like D > 400.

Changes required: In topside/procedures/conditions.py, add appropriate __str__ methods for all of the condition classes.

Reproduction:

import topside as top

parser = top.pdl.Parser(['application/resources/example.pdl'])
plumb = parser.make_engine()

print(plumb.fixed_pressures)
plumb.reset()
print(plumb.fixed_pressures)

We now have all of the pieces that we need to load a procedure and a plumbing engine from file, step through the procedure, step through time, and continue until the procedure terminates (or times out). We just need to pull all of these pieces together and write some code to visualize what's happening.

Work is currently ongoing to be able to do this inside the main application, but I think in parallel we should write a standalone script that we can run offline that executes the procedure non-interactively. This will be a nice tool to use as a sanity check, and I think we will be able to use the resulting code as the core of the "Procedure Validation" module - but let's start with just running it manually for now.