swri-robotics / gilbreth Goto Github PK

The output of object recognition is not stable in z-direction, see the following picture. The z-direction's output jumps from 1.03 to 0.92. I observed this behavior in many parts. This behavior makes it pretty hard to tune the parameter for robot to pick up the objects.

There needs to be a node that will automatically spawn objects on the conveyor belt. It should be configurable in order to control the frequency at which things are spawned and should also allow some randomization on the placement pose of the object.

Objects are too big, sometimes they collide with the robot and the conveyor belt.
Shall we re-scale them? Smaller objects will also reduce object recognition time.

The install instructions need the following corrections:

• The wstool merge ... command points to a non existent file
• The step under "Be sure to install gazebo_ros_packages." is a repeat from the step above for installing just gazebo. The rosdep step below should now make this unnecessary
• Somewhere in there it should state that this software uses gazebo 7.0
• Use catkin tools instead of catkin_make

It appears that the robot is in collision regardless of what pose it's in even when no objects are in collision with it. To reproduce bring up the demo.launch file in the gilbreth_moveit_config package and drag the robot to a collision free pose.

There are a number of model and code files to support the gazebo simulation that aren't needed.

Below is a list of changes/improvements to be made to the conveyor spawner node:

• The conveyor spawner node currently requires loading each object's urdf description as a ROS parameter. Consequently, there's a launch file that loads these parameters and an associated configuration yaml that has to be manually synced to match the objects in the launch. While this approach works, it could become inconvenient on an environment with lots more objects and consequently many more ROS parameters. Therefore one proposed solution is to have the conveyor spawner node load the object's urdf file directly and use it to create the gazebo object spawn service request.
• The spawning parameters in the conveyor_objects.yaml such as the initial pose and placement randomization variables are the same for all the objects. It would be more convenient to rearrange the file structure to allow setting these same parameters on an per object basis for more flexibility.

This crash happens very frequently when running the main application,

[Dbg] [ObjectDisposalPlugin.cc:111] [deletion_wall] Removing model: object_2
gzserver: /usr/include/boost/smart_ptr/shared_ptr.hpp:648: typename  
boost::detail::sp_member_access<T>::type boost::shared_ptr<T>::operator->() const [with T =  
gazebo::physics::SurfaceParams; typename boost::detail::sp_member_access<T>::type =  
gazebo::physics::SurfaceParams*]: Assertion `px != 0' failed.  

• Occasionally commands to move robot/rail only moves robot
• Simulated kinect sensor has no parameters for creating/controlling depth noise

I have observed that the gilbreth_controller_spawner node always dies on the first attempt at starting the gazebo simulation (times out waiting for controller manager). All subsequent attempts to start the gazebo simulation succeed, provided I let the first attempt to "exit gracefully". By that, I mean, I wait until MoveIt! vents out all its anger via log statements about controller managers not being available and unwillingly gives me the green signal to start planning anyway. Once I terminate this instance of the launch and re-launch the simulation, everything works just fine.

Its a bit weird that the second and subsequent attempts do not time out where as the first one always does! It might be my "slow" hardware that causes this, but I would expect this to happen always in that case. I am running this simulation on a MBP with Intel Core i5 2.5GHz Dual Core, 8GB RAM.

Any ideas on what could be happening here? Or anyone else having faced the same problem?

Is there a reason we are asked to source the env_setup.bash file which actually has an exec command at the end? I do understand that the script itself is required to set gazebo specific environment variables.

I am not much of a shell scripter myself, but I was told by @gavanderhoorn that sourcing a script with exec commands is not commonly done.

The following issues have been identified from adding the voxnet recognition node #51

• Installation instructions need to be more detailed or a script should be produced in order to facilitate installation of voxnet and its dependences.
• Instructions needed prior to running the voxnet recognition node should be listed as well, (e.g THEANO_FLAGS variable, etc)
• Find a solution to the runtime error that occurs when running the cnn-recognition node. The error message is as follows:

.
.
.
WARNING (theano.sandbox.cuda.blas): do not use pad for BaseGpuCorr3dMM; please set padding in border_mode parameter, see the docstring for more details
WARNING (theano.sandbox.cuda.blas): do not use pad for BaseGpuCorr3dMM; please set padding in border_mode parameter, see the docstring for more details
Traceback (most recent call last):
  File "/home/ros-i-devel/ros/kinetic/gilbreth_ws/src/gilbreth/gilbreth_perception/scripts/recognition_cnn.py", line 70, in <module>
    tfuncs, tvars = make_test_functions(cfg, model)
  File "/home/ros-i-devel/ros/kinetic/gilbreth_ws/src/gilbreth/gilbreth_perception/scripts/recognition_cnn.py", line 21, in make_test_functions
    out = lasagne.layers.get_output(l_out, X)
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/lasagne/layers/helper.py", line 197, in get_output
    all_outputs[layer] = layer.get_output_for(layer_inputs, **kwargs)
  File "/home/ros-i-devel/src/voxnet/voxnet/layers.py", line 217, in get_output_for
    contiguous_filters = gpu_contiguous(filters)
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/theano/gof/op.py", line 615, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/theano/sandbox/cuda/basic_ops.py", line 3910, in make_node
    input = as_cuda_ndarray_variable(input)
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/theano/sandbox/cuda/basic_ops.py", line 46, in as_cuda_ndarray_variable
    return gpu_from_host(tensor_x)
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/theano/gof/op.py", line 615, in __call__
    node = self.make_node(*inputs, **kwargs)
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/theano/sandbox/cuda/basic_ops.py", line 132, in make_node
    dtype=x.dtype)()])
  File "/home/ros-i-devel/.local/lib/python2.7/site-packages/theano/sandbox/cuda/type.py", line 95, in __init__
    (self.__class__.__name__, dtype, name))
TypeError: CudaNdarrayType only supports dtype float32 for now. Tried using dtype float64 for variable None
.
.
.

@lianjunli please take a look at these issues and submit a PR if you identify a proper solution to them. Thanks

The perception node seems to fail at identifying every object at the moment.
To reproduce just run the application as instructed here and observed the perception terminal window for its console output.

When running the main application (see Demo.md), as a part comes close to the robot, the robot arm moves in position to pick the object, then it lowers its tool towards the conveyor just in time to touch the target object. Once the tool and object make contact the gripper appears to fail at making a successful grasp. After re-attempting a grasp for approximately 10 seconds the robot gives up and retreats.

It happens every time when the object is dropped to the bin, Gazebo dies with following infomation:
[gazebo-2] process has died [pid 28267, exit code 134, cmd /opt/ros/kinetic/lib/gazebo_ros/gzserver -e ode --verbose /home/alex/my_gilbreth/src/gilbreth/gilbreth_gazebo/worlds/gilbreth.world __name:=gazebo __log:=/home/alex/.ros/log/c22afa4e-02dd-11e8-a045-142d27f202bb/gazebo-2.log].

After the object was placed in the bin, it is teleport to somewhere around the first bin and not being recycled.

An application launch file needs to be created which in short will execute the full simulation. Ideally, this would be broken up into two launch files:

• A simulation setup launch file that brings up all of the necessary components for the simulation to run, gazebo simulator, perception, motion tool planning, motion robot planning, etc.

• Then a second launch file will fire up the node or nodes that get the simulation process going.