wisc-hci / coframe Goto Github PK

Fields to show:

• Name
• Description
• Center Position
• Center Orientation
• Free Orientation (T/F)
• Shape (determine whether it is a cube or sphere based on the presence of the shape-dependent fields, and allow the designer to switch between them)
• Dimensions (X, Y, Z for cube, Radius for Sphere)

@curthenrichs might have suggestions on the orientation uncertainty fields (I forgot what he said)

Currently lives in model/EvdScript

Possibly involves theming or a simple css swap.

Stretch goal: maybe look into how highlighting/selecting blocks could be influenced by the current frame [color].

Switch from actions to async pub. Also provide synchronous behavior through services.

Rationale is that actions have additional overhead / support not needed in EvD.

Switch the frontend code for the different modals to use Ant's modal components instead of fluentUI. Also update to use the zustand stores available.

Frontend is currently faking some of EvD's data that will eventually be provided by the backend.

More of the node data (and structure) should be expressed in this file to act as a reference for component development.

Look at discrepancy between Robot-Scene and RVIZ/URDF axes

Confirm features of React Blockly Drawer: https://www.npmjs.com/package/react-blockly-drawer

Also, determine if/how

• Blockly Highlighting
• Blockly position of specific blocks in screen space
• Block instances for skills, locations, waypoints, etc.
• JSON representation instead of XML?

Then,

• Define an EvD program structure that makes sense for blockly (how should program be stored so on component rerender blockly can be quickly updated)
• Rethink parameterization of skills
  • Generally, should skills have parameters (and does this need to be explicitly built into EvD)
  • Specifically, how to best express the built-in skill parameterizations
  • SimplePickAndPlace needs to expose trajectories not just locations

If time allows maybe consider type checking on nodes (in order to prevent behavior like executing a waypoint as a primitive).

Parse complete program and data from evdFakeData into EvdStore

Suggestions:

• Allow the user to dis-engage the confirm button on off-click or after some delay.
• Change the text and styling of the 'ok' to being a danger button with text 'Delete'

Backend data server does not handle reorder manifest changes. These should probably be treated separately from set changes.

Reorder changes occur either on reorder function falls or on insert function calls.

• Read from the issues server to determine feedback sections.
• Handle behavior of checking off sections when children tasks are completed.
• Handle how making changes to setup/program impacts requirements of re-doing check-offs.

After Blockly Spike,

Work on defining blockly's data structure needs? How is EvDscript going to be converted into it. How should it interact with zustand.

I need to use the fake frontend to prove out the core simulation behavior that generates traces for EvD.

Based on the joint processor results, this should be a matter of testing just the node itself and less the underlying simulation.

Currently manipulation of things is not handled. Tokens need to be updates for movement and machines need to generate / consume things.

EvDscript is currently under-documented in the backend.

Need to write a markdown document explaining what it is, how it is structured, and the usage.

Example code snippets from the debug_app should be provided.

Move Hololens code into this repository as a top-level project. That way we can remove the need to separately maintain that repo. Future work may use hololens as reference.

I need to be able to receive a dict with environment objects from the configuration topic. I need to convert these into a collision object in pybullet (but one that allows the robot to pass through).

Effectively, I need to capture the minimum distance between each collision object and a set of valid robot bodies.

Same code can work for both collision meshes and occupancy zones.

Figure out how pybullet does things and get a working set of features exposed.

Options are read in from a comm channel (ROS) and can be displayed to the user to support loading in specific versions of applications

I need to test out the joint processor for correct generation of joint and trace data.

This will require me to register some locations when I register the rest of the workcell (for the fake frontend).

Getting this to work will prove out the underlying pipeline for the trace process. Leaving only the trajectory generation plan, structure packing, and node logic to test in trace processor.

This is the high risk item as it still requires some investigation. I need to detect the minimum distance between bodies except at the point where they actually are joints.

This also theoretically can be done on the frontend since it is a non-intuitive function on the joint state for a particular robot.

• Factor in Frame/Selected Item/Active Panes/etc
• Also investigate methods for displaying this info in a better (e.g. empirically-backed) way

Certain meshes don't display. Resolve by creating a story in robot-scene storybook that displays each mesh in a grid formation. Then fix meshes that have errors.

• Waypoints
  • List
  • Preview
  • Detail
• Machines
  • List
  • Preview
  • Detail
• Things
  • List
  • Preview
  • Detail
• Locations
  • List
  • Preview
  • Detail

Integrate Lively-TK into EvD Core system

Must act as middle-layer between robot_template / interpolator and the pybullet simulation

Things have a thing_type property as a string. This should probably be handled as a type declaration type. That way the same behavior can be used in machines / regions.

Switch the frontend code for the different control component to use Ant's components instead of fluentUI. Also update to use the zustand stores available.

Plan tracer is currently written for the UR specific interface. In order to support pybullet and new program runner we must convert over to the new generalized robot_interface.

Tasks should register safety/environment objects and data objects like machines, things, etc. with the data server.

Data server should provide this as a partial update to program.

Data server should provide status updates when state changes (e.g., program loads), which can then trigger object registration.

Implement a naïve heuristic to generate syntax errors and warnings for EvD's program quality frame

Backend EvD script should provide edit-able and delete-able properties on all nodes. Properties should be read only for instantiated objects. Serialization/Deserialization should preserve edit and delete status.

Note that these properties are more or less frontend specific and probably will not build in behavior to check this at the server.

The current 3D printer machine tending task is only vaguely described. This should be made more concrete in order to make it useful for evaluation.

Current implementation of blockly needs to be revisited.

• Locations, waypoints, things, and machines should have their various instances provided in the toolbox. Likewise, each should provide their list of items for the instance dropdown.
• Skills should have both an instance block and a definition block (including prebuilts). Instance blocks should not take in parameters at this time.
• Type checking should be enforced. E.g., waypoints should not be inserted into program directly.
• Parameterization needs to be updated on various blocks to match new backend model (subject to change)

Removing the dataserver and shifting responsibilities of it and the issue server to the frontend. This is a pretty large change so things will break but it should simplify the interaction for all nodes in ROS (and all developers).

Mostly need to address set behavior. Currently this is stubbed out.

Should also build in a periodic update scheme.

Should build in a method of getting UUIDs and their associated types without rest of content from backend.

Another computationally cheaper update flag pub should be provided with option to toggle between types at interface creation.

Interpolator sits before Lively-TK and after the robot interface/template.

This module should roughly approximate the low-level movement behavior expected on the UR3e robot.

Module should inherit from robot_template (and/or own a robot template obj)

Robot interface now is a thin, robot agnostic wrapper. In order to support a robot, a node should extend this template.

Robot interface-template paradigm follows what was previously implemented for machines.

Machine operations should resolve placeholders.

Make the program node drag gable in the grid component.

Edit the PrimitiveBlock.jsx file to render the primitives differently based on type (e.g. available parameters, etc).

Tests should be written to confirm validity of program construction and communication between data client and data server.

Need to specify the new ROS interface from frontend. Since the data server is being removed, the frontend will need to drive state changes in all ROS nodes.

Current launch files need to be refactored. Better separation of concerns and confirm files in core do not depend on files from tasks.

I am leaving the actual grader implementation for the frontend as I just don't have the bandwidth for it in the backend plus Andy presumably will be able to make better choices for the format that he needs.

• Performance graders (time, speed)
• Wear-and-tear
• Collision Meshes (we need backend data)
• Pinch-points (we need backend data ~ this is the most uncertain)
• Payload
• Occupancy zones

And anything else Andy thinks of / needs

Support more refined edits in the EvdStore
Respond to updates from the data_server/update topic

• Performance (time/speed)
• Collision
• Pinch-Points
• Payload
• Wear-and-Tear
• Occupancy