Add multi-up with simultaneous start in front end about openhtf HOT 8 CLOSED

I think this is most appropriately phrased as the question of what's our recommended best-practice for synchronizing numerous instances of openhtf, particularly when using shared test equipment. My thought is that many plugs for interacting with test equipment will end up included custom test start triggers in their codebase, such as one that waits for multiple stations to frob it before returning any station's call.

I recall discussing this with @grybmadsci a little while back, but I don't recall whether we came to any brilliant conclusions.

from openhtf.

Hi, wondering if anyone has any additional thoughts on this?

from openhtf.

The simplest solution is to have your entrypoint wait for the start event (pressing the start button or something) as well as start all the other test instances, sharing a threading.Event() instance between their start triggers. However, that requires the ability to run multiple openhtf instances in the same process, which I don't think is simple at the moment due to the use of gflags.

You could instead use multiprocessing.Process() to make all your instances from one binary, where you'd set FLAGS.http_port and other flags before running openhtf.Test(...).Execute(...). If you pass an multiprocessing.Event() in, then all the start triggers can wait() on that and have the main process set() it when the start event is triggered.

A third way is to write a binary that does the listening for the start event and also creates a bound socket.socket 'server' that just stores all its connections into a list. When it gets a start event, it sends some string into all the connections (catching and ignoring errors from clients that got disconnected in the process). The others would all connect to that socket and wait for data to come in. This allows starting and stopping the 'stations' independently, if that's something you want.

Sadly, they're all pretty complicated. If we could have multiple tests execute in the same thread, I would propose adding a function (possibly in contrib?) that accepted a list of openhtf.Test objects and a test_start and/or test_stop to synchronize the rest on.

from openhtf.

My team needs some sync stuff too, and I'm thinking of implementing a generic sync service in util as a run-able module. This would be similar to a simplified version of the third approach @fahhem lists above, where a separate service waits for frobs from a configured set of agents before returning any of the station's calls. The stations, on the other hand, would be able to use the corresponding start trigger and plug, with a configurable address of an individual instance of the sync service to contact.

from openhtf.

Folks, I want to revive the discussion around multi-up as it is going to be an important feature for my group as well. With looping taken out of the framework itself, we currently have potential for at least serial execution of multiple test objects within the same script. We've also talked about taking Execute() off of the main thread. I'd like to get a feel for what people need or want to see in terms of patterns for authoring multi-up tests, and how multi-up stations should look in the frontend (especially if running multiple tests in parallel).

from openhtf.

Execute() can already run off the main thread. I think at this point we need to collect some concrete requirements and see which of them we can solve by establishing conventions, and which we might need to solve with some helper utility code.

from openhtf.

Some requirements from my group:

• Frontend overview that shows all cells and a summary of their state on one screen.
• Code examples of multi-cell recommended patterns (requested for v1.0 release).
• Ability to start all cells simultaneously from the frontend.
• Ability to drill down into full test view on an individual cell in the frontend.
• Ability to see recent history from all cells in the frontend. Not clear whether some form of cell tagging
  is needed in the UI to indicate which cell a test run happened on.

from openhtf.

Closing this as obsolete because there's a whole new different frontend now!

from openhtf.