andyzeng / apc-vision-toolbox Goto Github PK

Hi, @andyzeng ,

Could you tell me the link about 3D online annotation tool mentioned in your paper? I'd like to try it.

Thanks!

Hello,I saw vertical stripes in the depth image in the training set, and there was no streaking in the depth image I acquired. Will these stripes have any effect? How are these stripes obtained?

Hello,I am learning how to make object mask. I want to learn the details. Can you give us a demo?Thanks！

#Hello,I have a question about this:
In line 107 to 124:

% Do 3D background subtraction
% pcwrite(pointCloud(objSegmPts'),fullfile(visPath,sprintf('vis.seg.%s',objName)),'PLYFormat','binary');
% pcwrite(backgroundPointCloud,fullfile(visPath,sprintf('vis.bg.%s',objName)),'PLYFormat','binary');
if useGPU
    [indicesNN,distsNN] = multiQueryKNNSearchImplGPU(backgroundPointCloud,objSegmPts');
else
    [indicesNN,distsNN] = multiQueryKNNSearchImpl(backgroundPointCloud,objSegmPts',1);
end
objSegmPts(:,find(sqrt(distsNN) < 0.005)) = [];
if strcmp(sceneData.env,'shelf')
  objSegmPtsBg = extBin2Bg(1:3,1:3) * objSegmPts + repmat(extBin2Bg(1:3,4) + [0;0;0.01],1,size(objSegmPts,2));
  bgRot = vrrotvec2mat([0 1 0 -atan(0.025/0.20)]);
  objSegmPtsBgRot = bgRot(1:3,1:3) * objSegmPtsBg;
%   pcwrite(pointCloud(objSegmPtsBgRot'),fullfile(visPath,sprintf('vis.objTestRot.%s',objName)),'PLYFormat','binary');
%   pcwrite(pointCloud(objSegmPts'),fullfile(visPath,sprintf('vis.objTestPre.%s',objName)),'PLYFormat','binary');
  objSegmPts(:,find(objSegmPtsBg(3,:) < -0.025 | objSegmPtsBgRot(3,:) < 0)) = [];
%   pcwrite(pointCloud(objSegmPts'),fullfile(visPath,sprintf('vis.objTestPost.%s',objName)),'PLYFormat','binary');
end

In line 115,There may be a missing line here:
objSegmConf(:,find(sqrt(distsNN) < 0.005)) = [];

In line 117 and 118:

  objSegmPtsBg = extBin2Bg(1:3,1:3) * objSegmPts + repmat(extBin2Bg(1:3,4) + [0;0;0.01],1,size(objSegmPts,2));
  bgRot = vrrotvec2mat([0 1 0 -atan(0.025/0.20)]);

Why do I add 0.01 to the Z axis and how do I get 0.025 and 0.2. What is the purpose of doing this?

Please help solve this question, thanks!

I encounter an error when I run the code
rosservice call /marvin_convnet ["elmers_washable_no_run_school_glue","expo_dry_erase_board_eraser"] 0 0
it outputs
ERROR: transport error completing service call: unable to receive data from sender, check sender's logs for details
how to fix the problem?

Hi, thanks for your work in this - could I ask when and how you included marvin into this project?

I have manually updated marvin.hpp with bug fixes from the Princeton directory to run cuda 8 (just the changes from https://github.com/PrincetonVision/marvin/commits/master/marvin.hpp which deal with cuda 8), and have surprisingly managed to get marvin_convnet to compile, but on running the marvin_convnet detect rosnode, I get a Tensor Coversion error:

Tensor conversion is not supported: from Type < to Type 
FatalError at LINE 2447. Program Terminated.
CUDA failure at LINE 2920: 17
FatalError. Program Terminated.
CUDA failure at LINE 4488: 17
FatalError. Program Terminated.

The error seems to eminate from marvin.hpp T* read(File * fp, int batch_size); function

I was hoping you might be able to shed any light on what might be happening here? Its odd that no recognized types seem to be displayed

Cheers!
Ubuntu 16.04,
Cuda 8.0
CuDNN v5
Nvidia GTX 1060

Hello,when I read apc.hpp, I have a question about this:

void prefetch() {

        checkCUDA(__LINE__, cudaSetDevice(GPU));

        std::string data_directory = file_data[0];

        for (int batch_idx = 0; batch_idx < batch_size; batch_idx++) {
            std::string curr_obj_name = object_list[curr_obj_idx];
            std::string curr_obj_directory = data_directory + "/" + curr_obj_name;

            // Select a random sequence from object directory
            std::vector<std::string> sequence_list;
            GetFilesInDirectory(curr_obj_directory, sequence_list, "seq-0");
            std::sort(sequence_list.begin(), sequence_list.end());
            int rand_sequence_idx = (int)floor(GetRandomFloat(0, (float)sequence_list.size()));
            std::string curr_sequence_name = sequence_list[rand_sequence_idx];
            std::string curr_sequence_directory = curr_obj_directory + "/" + curr_sequence_name;

            // Select a random image from the sequence
            std::vector<std::string> image_list;
            GetFilesInDirectory(curr_sequence_directory, image_list, ".color.png");
            std::sort(image_list.begin(), image_list.end());
            int rand_image_idx = (int)floor(GetRandomFloat(0, (float)image_list.size()));
            std::string curr_image_name = image_list[rand_image_idx];
            ...

In line 110：
GetFilesInDirectory(curr_obj_directory, sequence_list, "seq-0");
I don't understand "seq-0". Because when I browse the "./train_zip/training/shelf/barkely_hide_bones/scene-0000/frame-000000.color.png
", I didn't find "seq-0" in the path.
Please help solve this problem, thanks!

Hello，I am going to calibrate my camera's pose using getCalib.m. I saw the colored calibration board in benchmark. Where can I download these calibration boards?

In file "marvin_convnet/src/detect.cu", it tries to imread a HHA.png.
What's is HHA? The save_images service doesn't save any HHA pictures.

Hello,recently I'm working to achieve your code.When I use rosrun realsense_camera capture _display:=True,It shows:
RealSense error calling rs_is_stream_enabled(device:0xa01d38, stream:COLOR_ALIGNED_TO_DEPTH):
argument "stream" must be a native stream
And the image only can show 0.5 second(I guess),then it close and report an error.

And if I use rosrun realsense_camera capture,It works,and I can see the RBG image in rviz,but no RBG-D image.I use rostopic list to list the ROS topic ,it only have
/pointcloud
/rgb/camera_info
/rgb/image
/rgb/image/compressed
/rgb/image/compressed/parameter_descriptions
/rgb/image/compressed/parameter_updates
/rgb/image/compressedDepth
/rgb/image/compressedDepth/parameter_descriptions
/rgb/image/compressedDepth/parameter_updates
/rgb/image/theora
/rgb/image/theora/parameter_descriptions
/rgb/image/theora/parameter_updates
I subscibe to /rgb/image/compressedDepth,It shows:
Compressed Depth Image Transport - Compression requires single-channel 32bit-floating point or 16bit raw depth images (input format is: rgb8).

What's wrong? How can I get the RGB-D image?

Hello
I was wondering if you could provide a bit more documentation on how to train on own dataset. Say I want to use this project for getting the 6D pose of chairs. Could you clarify:

• what format does my dataset need?
• what about the labeling?
• where to place what in the folder structure?
• finally, how do we run the training and then evaluate results (commands to execute)?

Loved your project work. I'm a huge fan and I am excited to learn from your steps!

Hi,

After copying the ROS package ros_packages/.../realsense_camera into your catkin workspace source directory (e.g. catkin_ws/src ) and compiling using catkin_make , I am getting the following error.

In file included from /home/ra-1/catkin_ws_robot/src/realsense_camera/src/capture_v2.cpp:3:0:
/home/ra-1/catkin_ws_robot/src/realsense_camera/src/display.hpp:1:31: fatal error: librealsense/rs.hpp: No such file or directory
compilation terminated.
realsense_camera/CMakeFiles/capture.dir/build.make:62: recipe for target 'realsense_camera/CMakeFiles/capture.dir/src/capture_v2.cpp.o' failed
make[2]: *** [realsense_camera/CMakeFiles/capture.dir/src/capture_v2.cpp.o] Error 1
CMakeFiles/Makefile2:367: recipe for target 'realsense_camera/CMakeFiles/capture.dir/all' failed
make[1]: *** [realsense_camera/CMakeFiles/capture.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j4 -l4" failed

Please help to solve this

Hello,When I use the Marvin tool to draw the FCN32 diagram, there is an error：

>> plotNet()
The following error occurred converting from string to cell:
Conversion to cell from string is not possible.

Error in plotNet (line 21)
            nameofResponse(end+1) = out(j);

Please help to solve this

Hello,recently I'm working to achieve your code.I practiced using Marvin.I downloaded the complete dataset.But I have some problems.And I use the GPU-1070 8G.
Please help to solve this

Thanks for opensource this fantastic project!
By the way, when I was using F200 and run stream under apc-vision-toolbox/realsense-standalone/
Got a runtime erro.

I thought it is something to do with api convention.
would l try to solve it by myself first. Thank you.