Camera Documentation about glabstraction.jl HOT 30 CLOSED

Thats great :) Here is what I have so far (mainly from your original code):

w, r = glscreen(resolution=(1024,1024))
robj    = visualize(mesh)

(gripper_orient,gripper_position) = (FixedSizeArrays.Vec{3,Float32}((-2.3588722f0,-0.79126376f0,1.5828755f0)),FixedSizeArrays.Vec{3,Float32}((0.121540956f0,-0.06841759f0,0.38124424f0)))

lookatvec = gripper_orient 
eyeposition = gripper_position 

# I am not sure if they are in the same frame as the object mesh - and think I have to do a translation and rotation on the mesh to move them into the same frame. 

theta = Signal(Vec3f0(0))
translation = Signal(Vec3f0(0))
zoom        = Signal(1f0)

# I tried removing theta and translation, but then I get an error, so I have to leave them in. 

w.cameras[:my_cam] = PerspectiveCamera(
    w.inputs[:window_size],
    eyeposition,
    lookatvec,
    theta,
    translation,
    zoom,
    Signal(410f0), # Field of View
    Signal(0.1f0),  # Min distance (clip distance)
    Signal(100f0) # Max distance (clip distance)
)

GLVisualize.view(robj, method=:my_cam)

r()
yield() # yield to render process
sleep(0.01) # let render
img = screenbuffer(w, channel=:depth) # display depth image

from glabstraction.jl.

I also forgot to mention, that theta, zoom and translation are relative, so theta=Vec3f0(0,0,1) means, move by one in Z-direction.

from glabstraction.jl.

Thanks for that.. I am trying to stay put now. I would like to look at my mesh from the camera. So I have one view for every simulation run.

from glabstraction.jl.

Yup, that would be a problem...
On 14 Jan 2016 08:27, "kleinash" [email protected] wrote:

Thanks for that.. I am trying to stay put now. I would like to look at my
mesh from the camera. So I have one view for every simulation run.

—
Reply to this email directly or view it on GitHub
https://github.com/JuliaGL/GLAbstraction.jl/issues/33#issuecomment-171562084
.

from glabstraction.jl.

Why? I have the position and orientation of the camera and the mesh which is external. All I have to do is import the mesh (like before) and make it relative to position and orientation in world frame. Then direct the position of the camera to be looking at the mesh..

from glabstraction.jl.

Whoops, sorry... I answered per phone and must have clicked on the wrong email to reply.
This is the answer for JuliaGL/GLVisualize.jl#53.
In this issue I'm not sure if there is still an open question, or if everything is answered...

from glabstraction.jl.

Managed to get what I needed - i.e. am able to now position the camera where I want the view to be from rather than having it spin around an axis or depend on the bounding box. Thanks again.

from glabstraction.jl.

Great to hear :)

2016-01-18 15:04 GMT+01:00 kleinash [email protected]:

Managed to get what I needed - i.e. am able to now position the camera
where I want the view to be from rather than having it spin around an axis
or depend on the bounding box. Thanks again.

—
Reply to this email directly or view it on GitHub
https://github.com/JuliaGL/GLAbstraction.jl/issues/33#issuecomment-172534236
.

from glabstraction.jl.

Could I ask one further question? Now that I have moved things around - when I access the depth channel I get random numbers out rather than a whole bunch of 0's and the values of depth where the object exists.. any idea what could possibly case this? because it is rendering fine.

from glabstraction.jl.

It'd help a lot if you show me your code... The one for rendering, and the
one you "moved around"

2016-01-18 15:21 GMT+01:00 kleinash [email protected]:

Could I ask one further question? Now that I have moved things around -
when I access the depth channel I get random numbers out rather than a
whole bunch of 0's and the values of depth where the object exists.. any
idea what could possibly case this? because it is rendering fine.

—
Reply to this email directly or view it on GitHub
https://github.com/JuliaGL/GLAbstraction.jl/issues/33#issuecomment-172538882
.

from glabstraction.jl.

Sorry for the messy code.. but this is it..

using MAT, FileIO, GLVisualize, GLAbstraction, GeometryTypes, Reactive, DataFrames, HDF5

include("rotEuler.jl")
include("transl.jl")

function hand_depth_map()
#=
function that generates depth image from perspective of hand in scene and transforms data into an hdf5 file
    takes no inputs
    outputs training.h5 and test.h5 data file
=#
    ## ------------ Load raw input data
    mesh = load("<path_to>.obj")
    # load mat file files
    path = "<path_to>/"

    mat_filename = joinpath(path, "<path_to>.mat")
    vars = matread(mat_filename)
    data = vars["data"]

    # # -------------- Generate depth map from hand perspective
    # for every grasp save the depth image, grasp measure and gripper paramaters to h5 file
    total_num_grasps = 3#length(data["grasp_measure"])

    # create hdf5 file for input to convnet ~ https://github.com/JuliaLang/HDF5.jl/blob/master/doc/hdf5.md
    data_file = "<path_to>/input_data.h5"
    input_data = h5open(data_file, "w")
    group = g_create(input_data, "objects")
    depth_images = d_create(group, "depth_images", datatype(Float32), dataspace(1024,1024, 1, total_num_grasps))
    grasp_measures = d_create(group, "grasp_measures", datatype(Float64), dataspace(1,1))

    for i = 1:total_num_grasps

        grasp_num = i

        gripper_position    = data["hand_position"][grasp_num]
        gripper_orient      = data["hand_orientation"][grasp_num]
        grasp_measure       = data["grasp_measure"][grasp_num]

        # [TODO: need to -0.3 from the z position relative to the hand ]
        # camera_position       = gripper_position

        gripper_orient      = FixedSizeArrays.Vec{3,Float32}(gripper_orient[1], gripper_orient[2], gripper_orient[3])
        camera_position     = FixedSizeArrays.Vec{3,Float32}(gripper_position[1], gripper_position[2], gripper_position[3])

        # To put the object and camera in the same frame find rotation matrix of gripper_orient (camera)
        # and then multiply by [0 0 1] if its z that is facing the object
        rot_mx = rotEuler(gripper_orient[1], gripper_orient[2], gripper_orient[3])
        camera_normal = rot_mx * [0; 0; 1]
        camera_normal = FixedSizeArrays.Vec{3,Float32}(camera_normal[1], camera_normal[2], camera_normal[3])

        w, r = glscreen(resolution=(1024,1024))

        robj            = visualize(mesh)
        bb              = boundingbox(robj).value
        bb_width        = width(bb)
        lower_corner    = minimum(bb)
        middle          = lower_corner + (bb_width/2f0)
        # lookatvec     = minimum(bb)
        # eyeposition   = middle + (bb_width.*Vec3f0(2,2,2))
        lookatvec       = camera_normal #minimum(bb) # this should be the output of the rotation mx multiplied with [0 0 1]
        eyeposition     = camera_position #middle + (bb_width.*Vec3f0(2,2,2)) # gripper position

        theta = Signal(Vec3f0(0))
        translation = Signal(Vec3f0(camera_position[1], camera_position[2], camera_position[3]))
        zoom        = Signal(1f0)

        w.cameras[:my_cam] = PerspectiveCamera(
            w.inputs[:window_size],
            eyeposition,
            lookatvec,
            theta,
            translation,
            zoom,
            Signal(410f0), # Field of View 
            Signal(0.01f0),  # Min distance (clip distance) 
            Signal(100f0) # Max distance (clip distance) 
        )

        GLVisualize.view(robj, method=:my_cam)

        r()
        yield() # yield to render process
        sleep(0.01) # let render

        img = screenbuffer(w, channel=:depth) # display depth image

        depth_images[:, :, 1, grasp_num] = img # save depth image
        grasp_measures[:, :] = grasp_measure # save grasp_measure

    end # end of main for loop
    close(input_data)
end # end of function that generated depth map from hand perspective

from glabstraction.jl.

r() is blocking and renders the image as long as the window is open. So you need to call it like this @async r(), if you don't want to record data from a closed window.
This all shouldn't be in a loop anyways. Any reason you don't do it like in the example?

from glabstraction.jl.

I hear you start with the problem. My problem is that I want to create an HDF5 file with training data. Each data entry consists of a depth image (1024x1024), a binary decision (1,1), and paramaters (will get there).

I need to loop through 1000 (5 mat files with 200 entries) recorded simulations. I load the mesh.obj and take the recorded camera position (from the recorded simulation data) and generate a depth image from that angle (almost there but my image is only half on the page). Then I save the depth image in one line of one dataset, the binary decision in the corresponding line in the next dataset and the paramaters will go in the last dataset.

At the end I should have a .h5 file that has one group (objects) with each object having 3 datasets: depth image, binary decision and paramaters.

I dont need the render to happen in a loop, I try take it out completely but then I cant see if I am getting the right angle. I am going to remove the loop for now and try just get the right angle then put the loop back in just now.

Will be in touch 👍

from glabstraction.jl.

there's nothing that speaks against the loop.. just move the window creation and @async r out of it... Like in the cat example. You should be able to see if you get the right angle in any case.

from glabstraction.jl.

by the way, I'm rebuilding the rendering and camera code at the moment, which should make it a little easier to do these kind of tasks...so if you can wait a little, it probably would make sense to wait for this!

from glabstraction.jl.

Sure thing.. I did just that (move the @async r()) out the loop - but I get a nil() error when it goes through the loop again - so am happy to wait for further instructions :)

from glabstraction.jl.

Any chance this is on its way? I am getting this error:
ERROR: LoadError: MethodError: width has no method matching width(::GeometryTypes.HyperRectangle{3,Float32}) but my GLVisualize GLAbstrction and MeshIO are up to date..

from glabstraction.jl.

Sorry for the delay... Right now I'm fixing up the boundingboxes. Camera is
almost done. Is GeometryTypes up to date as well? What do you mean by up to
date anyways? On master and you did a git pull? Or did you do Pkg.update()?

2016-01-26 10:18 GMT+01:00 kleinash [email protected]:

Any chance this is on its way? I am getting this error:
ERROR: LoadError: MethodError: width has no method matching
width(::GeometryTypes.HyperRectangle{3,Float32}) but my GLVisualize
GLAbstrction and MeshIO are up to date..

—
Reply to this email directly or view it on GitHub
https://github.com/JuliaGL/GLAbstraction.jl/issues/33#issuecomment-174917822
.

from glabstraction.jl.

Just looking at the GeometryTypes now. Yip - Pkg.update(), Pkg.checkout("GeometryTypes") and just for good measure did git pull in the root dir.

Thank you.

I am looking at finding a way to solve the width issue - brb

EDIT: Looks like I can remove these lines because I dont use them really:

bb = boundingbox(robj).value
bb_width = width(bb)
lower_corner = minimum(bb)
middle = lower_corner + (bb_width/2f0)

Which solves the width issue.

Next error:

LoadError: MethodError: convert has no method matching convert(::Type{GLAbstraction.PerspectiveCamera{T}}, ::Reactive.Signal{GeometryTypes.SimpleRectangle{Int64}}, ::FixedSizeArrays.Vec{3,Float64}, ::FixedSizeArrays.Vec{3,Float32}, ::Reactive.Signal{FixedSizeArrays.Vec{3,Float32}}, ::Reactive.Signal{FixedSizeArrays.Vec{3,Float32}}, ::Reactive.Signal{Float32}, ::Reactive.Signal{Float32}, ::Reactive.Signal{Float32}, ::Reactive.Signal{Float32})
This may have arisen from a call to the constructor GLAbstraction.PerspectiveCamera{T}(...),
since type constructors fall back to convert methods.
Closest candidates are:
GLAbstraction.PerspectiveCamera{T}(!Matched::Reactive.Signal{GLAbstraction.Pivot{T}}, !Matched::Reactive.Signal{GeometryTypes.SimpleRectangle{Int64}}, !Matched::Reactive.Signal{T}, ::Reactive.Signal{T}, ::Reactive.Signal{T}, !Matched::Reactive.Signal{FixedSizeArrays.Mat{4,4,T}}, !Matched::Reactive.Signal{FixedSizeArrays.Mat{4,4,T}}, !Matched::Reactive.Signal{FixedSizeArrays.Mat{4,4,T}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T}})
GLAbstraction.PerspectiveCamera{T<:Real}(::Reactive.Signal{GeometryTypes.SimpleRectangle{Int64}}, ::FixedSizeArrays.Vec{3,T<:Real}, !Matched::Union{FixedSizeArrays.Vec{3,T<:Real},Reactive.Signal{FixedSizeArrays.Vec{3,T<:Real}}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T<:Real}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T<:Real}}, !Matched::Reactive.Signal{T<:Real}, !Matched::Reactive.Signal{T<:Real}, !Matched::Reactive.Signal{T<:Real}, !Matched::Reactive.Signal{T<:Real})
GLAbstraction.PerspectiveCamera{T<:Real}(::Reactive.Signal{GeometryTypes.SimpleRectangle{Int64}}, ::FixedSizeArrays.Vec{3,T<:Real}, !Matched::Union{FixedSizeArrays.Vec{3,T<:Real},Reactive.Signal{FixedSizeArrays.Vec{3,T<:Real}}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T<:Real}}, !Matched::Reactive.Signal{FixedSizeArrays.Vec{3,T<:Real}}, !Matched::Reactive.Signal{T<:Real}, !Matched::Reactive.Signal{T<:Real}, !Matched::Reactive.Signal{T<:Real}, !Matched::Reactive.Signal{T<:Real}, !Matched::Any)

from glabstraction.jl.

well, if geometrypes is up to date, the width issue suggests that you're
not up to date on GLVisualize/MeshIO/GLAbstraction... Maybe give them a
checkout and make sure that you're on master?

2016-01-26 13:07 GMT+01:00 kleinash [email protected]:

Just looking at the GeometryTypes now. Yip - Pkg.update(),
Pkg.checkout("GeometryTypes") and just for good measure did git pull in the
root dir.

Thank you.

I am looking at finding a way to solve the width issue - brb

—
Reply to this email directly or view it on GitHub
https://github.com/JuliaGL/GLAbstraction.jl/issues/33#issuecomment-174972856
.

from glabstraction.jl.

julia> Pkg.checkout("GLVisualize")
INFO: Checking out GLVisualize master...
INFO: Pulling GLVisualize latest master...
INFO: No packages to install, update or remove

julia> Pkg.checkout("MeshIO")
INFO: Checking out MeshIO master...
INFO: Pulling MeshIO latest master...
INFO: No packages to install, update or remove

julia> Pkg.checkout("GLAbstraction")
INFO: Checking out GLAbstraction master...
INFO: Pulling GLAbstraction latest master...
INFO: No packages to install, update or remove

from glabstraction.jl.

okay... so by the way, where is width missing? width got deprecated in favor of widths. Sorry, I'm not really sure why we didn't properly deprecated this...It's helpfull if you always include the code you're executing and the the stack trace of the error...

from glabstraction.jl.

Yay! althought I dont need it right away, widths was the answer.

Sorry about that.. Its just the PerspectiveCamera set up which I know you are working on:

w.cameras[:my_cam] = PerspectiveCamera(
    w.inputs[:window_size],
    eyeposition,
    lookatvec,
    theta,
    translation,
    zoom,
    Signal(410f0), # Field of View 
    Signal(0.01f0),  # Min distance (clip distance) 
    Signal(100f0) # Max distance (clip distance) 
)

from glabstraction.jl.

@sjkelly, was this because it has a slightly different meaning and doesn't really replace width? Are we even fully decided on this? In some areas, we still have width, I think...

from glabstraction.jl.

redirect from closed comment @sjkelly

from glabstraction.jl.

I see it seems we missed some deprecation warnings. I'll take a look in a couple of hours

from glabstraction.jl.

Managed to get rid of the above error. It was because one of my functions was returning a Float64 rather than a Float32.
Also thumbs up, the screen is rendering really quickly now.

from glabstraction.jl.

Hi.

This is a screen shot of the glscreen on the right and plot function on the left. I was hoping to see a similar view - I can see that the data is right, but can't for the life of me figure out why I am looking at it from the side in the plot?

from glabstraction.jl.

Ok - not to worry on this one.. I see line 292 in renderloop.jl is return rotl90(img). So the values are coming out right - but because I am using PyPlot when I plot its going to be different. I guess we can finally close this one :)

from glabstraction.jl.

Glad you figured it out :)

from glabstraction.jl.