oftheo / ofxkinect Goto Github PK

i'm planning on moving the depth processing, registration, manipulation stuff into a separate addon called ofxDepthUtils, and making ofxKinect the absolute minimal code required to get the kinect talking to OF. this way people can swap out ofxOpenNI for ofxKinect and still have all the useful tools for manipulating and rendering stuff. it should clean up of ofxKinect a bunch, too.

i think it's mostly been arturo and i developing this, and he's on board -- but i'd love to hear if anyone else has any ideas.

everything is mirrored.

Howdy all,

I just committed a universal build (x86, i386, ppc) of libusb to the develop branch. Please test this. So far it works fine for me on 10.6.

This is a build of libusb following the libfreeenct readme using the following ./configure options:

./configure LDFLAGS="-framework IOKit -framework CoreFoundation" CFLAGS="-arch i386 -arch x86_64 -arch ppc" --enable-static --disable-shared --disable-dependency-tracking

Once you've built lib usb, the .a is found in a hidden dir: libusb/.libs

It would be awesome if ofxKinectPlayer could loop like ofVideoPlayer

someone sent this to me:

"found this around the interwebs: http://groups.google.com/group/openkinect/browse_thread/thread/31351846fd33c78/e98a94ac605b9f21?lnk=gst&q=stephane&pli=1

It's much more accurate then the way you use in ofkinect."

hopefully this error is my fault... but i'm not the only one. i cannot get ofxKinect to compile on my machine,

i'm using Xcode 3.1.3, osx 10.5.8 and the most recent commit "ofTheo-ofxKinect-a15abc7"

"ofxKinectCalibration::k1", referenced from:
__ZN20ofxKinectCalibration2k1E$non_lazy_ptr in ofxKinectCalibration.o
"ofxKinectCalibration::k2", referenced from:
__ZN20ofxKinectCalibration2k2E$non_lazy_ptr in ofxKinectCalibration.o
"ofxKinectCalibration::k3", referenced from:
__ZN20ofxKinectCalibration2k3E$non_lazy_ptr in ofxKinectCalibration.o
"ofxKinectCalibration::k4", referenced from:
__ZN20ofxKinectCalibration2k4E$non_lazy_ptr in ofxKinectCalibration.o
ld: symbol(s) not found
collect2: ld returned 1 exit status

(also on the openFrameworks forum http://www.openframeworks.cc/forum/viewtopic.php?f=8&t=5372)

any insight on how to solve the problem would be awesome.

thanks.
Ken

When doing our project we noticed that performance suddenly got very bad framerate sometime after this commit: 78ad750

With the latest code we get perhaps half the fps we got before and is pretty much unusable for what we use it for. Has anyone else noticed that?

Our project is available at https://github.com/blunden/medinect but it still has lots of things to implement.

Not sure how to debug this but after building and running the example a few time I get a massive hang. The app will not launch - after stepping the code, it hangs at kinect.open();

Is anyone else getting this...

getColorAt doesn't work properly with the IR image

I haven't figured out yet how to suggest a change in a proper Git fashion.. but this works for me :

int index = (y * width + x) * bytespp;
ofColor c;
c.r = videoPixels[index + 0];
c.g = videoPixels[index + (bytespp-1)/2];
c.b = videoPixels[index + (bytespp-1)];
c.a = 255;

return c;

Any plan for use with Windows?

Driver for Windows: http://ajaxorg.posterous.com/

Thanks
vSaAmTp

http://board.ump2002.net/thread.php?threadid=40890

the ofxKinect example project seems to run fine but previous projects updated with
16ac9f9 has the following errors and it noticeable as the video stutters

[Stream 70] Expected 1748 data bytes, but got 1908. Dropping...
[Stream 70] Expected 1748 data bytes, but got 1908. Dropping...
Isochronous transfer error: 1
Isochronous transfer error: 1
Isochronous transfer error: 1
[Stream 70] Expected 1748 data bytes, but got 1908. Dropping...
[Stream 70] Expected 1748 data bytes, but got 1908. Dropping...
Isochronous transfer error: 1
Isochronous transfer error: 1
Isochronous transfer error: 1

you can download the previous version that worked well here:
https://github.com/jvcleave/ofxKinect_triangle

Mac OS X 10.6.5
Xcode Version 3.2.5

i've noticed that the color image from the kinect is really crap, but if you use the yuv->rgb mode it looks way better. inside ofxKinect::threadedFunction at freenect_set_video_format try passing FREENECT_VIDEO_YUV_RGB instead of FREENECT_VIDEO_RGB and you'll see what i mean

an example of using multiple kinects can be found in the cinder block http://forum.libcinder.org/topic/new-kinect-support#23286000000454019

What should the license be? Libfreenect is using Apache and GPL2 ... if we decide on one, I can add the files and header comments.

consider using filling in from janus machine, see fillHoles() here https://github.com/kylemcdonald/TheJanusMachine/blob/master/CaptureApp/src/addons/ofxStructuredLight/src/FastThreePhase.cpp

I have openframeworks properly installed on Ubuntu, with demos building and running. The ofxKinect app does not work, however. I have it cloned in apps/examples/ofxKinect. When I run make I get a lengthy list of errors.

The error log can be found here: https://gist.github.com/796064

it's now possible to get the raw infrared video stream from the kinect, we should wrap it OpenKinect/libfreenect@334c1ee

dan's background filtering, if(depthPixelsBack[k] < 1024), needs to be removed as those values are legitimate in general and should instead be handled by his application specifically.

this issue was here a couple days ago, i think github lost it in the crash...

The kinect should be closed and shutdown cleanly. libfreenct has added function stubs for now and I've added the functions to the shutdown section in the thread function. They are commented for now. Once libfreenect has these functions implemented, these calls in ofxKinect should be uncommented.

The last pull request which I merged in brought 30fps for mac and a lot of stuff from the new api ( ie cleaner / faster code ). I updated the xcode project but the other projects need updating.

feels weird that both near and far planes clip to 0 - feel like near should clip to 255 so at least you know something is there?

I just spent the last 6 hours or so mounting and testing an application only to find that the far clipping plane is set at 4m ... totally skewing my setup since I naturally assumed the default is unlimited and was trying to track down my "thresholding bug". I only discovered this thanks to comparing to an earlier app that was magically showing me things that are much farther away then the new app.

The near plane is at 0, how about the default far plane is something like 100m?

ofxKinect::calibration needs an accessor function so we can do things like set the near/far clipping planes. or can it just be public?

We're having some quite weird problems with compiling the ofxKinect source for our ofxKinect-based openFrameworks-application where the code doesn't always link correctly.

The linker complains that several libfreenect definitions are "not declared in this scope" intermittently. It is reproducible if we pull down our latest source from git and then proceed to make clean, make with a successful build, make clean again and then with the next make we get the errors.

http://pastebin.com/fnAdyMdn --- for the error messages we receive.

We would be very grateful for any help with this issue.

I think the directory structure should mimic the other addons with the libs and src in the main tree:
ofxKinect/libs
ofxKinect/src

and move the example project into:
ofxKinect/example

I will try setting it up now.

i've already gotten three people annoyed with this, and i realize now it was a bad idea to start switching to the next OF release so soon.

i'm going to:

1 modify all the source so it uses ofx* instead of of*
2 update the current OF project to include ofx* for the typedefs
3 add a secondary OF project that uses the old style XCode project
4 rename both projects so there is ofxKinect.xcodeproj (<=r62) and ofxKinect-beta.xcodeproj (>=r70)

//------------------------------------
ofTexture & ofxKinect::getTextureReference(){
if(!rgbTex.bAllocated()){
ofLog(OF_LOG_WARNING, "ofxKinect: getTextureReference - texture is not allocated");
}
return depthTex;
}

//---------------------------------------------------------------------------
ofTexture & ofxKinect::getDepthTextureReference(){
if(!depthTex.bAllocated()){
ofLog(OF_LOG_WARNING, "ofxKinect: getDepthTextureReference - texture is not allocated");
}
return rgbTex;
}

Each one is returning the opposite texture they should be!

There's a fix for the memory leak in darwin_usb.c, which Tim & I've posted info on here: http://www.openframeworks.cc/forum/viewtopic.php?f=14&t=4947&start=150

You won't find it in libusb 1.0.8, but it is available here:

"I'm not libusb's darwin port maintainer so I can't push the patch to the mainline, but I think it's available at Peter Stuge's branch at http://git.libusb.org/?p=libusb-stuge.git;a=summary;js=1

The relevant commit is http://git.libusb.org/?p=libusb-stuge.git;a=commit;h=81fca5f3e33f1d4f1012c2546304b2c53d71b4b1;js=1" – Hoi-Ho (who created the patch)

Can you update usb-1.0.a to resolve this memory leak?

the kinect automatically shuts down and power cycles when it loses the pattern (e.g., too far away, outdoors, etc.) or when something gets too close to the device (presumably to keep people from being blinded).

this is disabled by the xbox, and has been added to the unstable libfreenect branch.

it currently makes more sense to just call update() every frame than to use isFrameNew(). should we just remove isFrameNew()? it doesn't make sense to add grabFrame because kinect runs differently than the video grabber.

Hi Theo.

I'm new to all of this and have some problems but not sure they are down to my misunderstanding.

1. I've installed OF from here - https://github.com/openframeworks/openFrameworks
2. I downloaded ofxKinect from here - https://github.com/ofTheo/ofxKinect/archives/master
3. I installed code::blocks from scripts/linux/ubuntu
4. I then installed the dependencies from the same folder as (3)
5. I built the OpenFrameWorks debug and release libraries within code::blocks without issue.
6. I had to remove accelerometer, utils and motors references from the kinectExample.cbp file.
7. I then had to remove all references of unicap from the kinectExample.cbp file too.
8. The project then almost built, but complained about an undefined reference to "freenect_get_raw_accel". If I removed these references the project built and ran (if the kinect is disconnected, having it connected produced a segfault), showing 3 windows within the console and a fps count in the region of 50fps.

Clearly something is wrong but I'm not clear what. Can you help?

I'm on ubuntu 10.04 lucid lynx.

Thanks!

m.

Hello. I've gotten ofxKinect working in Windows with the most recent openkinect/libfreenect with a few modifications. Curiously, the change that ultimately caused the addon to work was adding a specific api call in ofxKinect::update() to let freenect process usb events. (Otherwise, I was noticing, that none of the callbacks were being run.)

Basically:

void ofxKinect::update(){
    freenect_process_events(kinectContext); // This is new.

    if(!bGrabberInited){
        return;
    }

    // ...

I'm not aware of the ramifications for OSX or whatever else you guys are running. But once I added this everything started working wonderfully. ;)

The experimental branch is broken against the latest OF 007 due to ofBaseVideo now requiring an ofPixels accessor. I will refactor the internals to use ofPixels to fix this.

I see there are recorder/player classes, however these seem a bit special case situations. I've written my own threaded saver - very similar to ofxKinectRecorder, except I just save the rawDepthPixels in a separate thread etc. When I want to play this back, it is a bit tricky. Ideally I would like an API such as :
kinect.pauseCapture(); // stops the thread which is reading from the device
kinect.updateData(unsigned char* colorData, unsigned short *depthData);// takes in arrays from an external source (provided by my app), and updates everything necessary if any of them are non-NULL. This would aid the ability to reuse ofxKinects calibration and/or drawing routines with minimal code. I started looking into adding this, but there is so much going on inside ofxKinect, that without fully being away of all the internal workings, it's quite dangerous for me to do this kind of stuff without being aware of the implications of starting/stopping capture etc.

Grab it here: https://github.com/OpenKinect/libfreenect/issues#issue/22

So far isFrameNew has been a dummy function returning true. This is a useful function in that it can be used to avoid redundant computation on a existing frame.

I'm using XCode 3.25. Unfortunately, when building ofxKinect.xcodeproj, I get a huge number of errors (995, in fact). For example, in main.cpp:

Glew.h: No such file or directory
Cv.h: No such file or directory

Any pointers would be appreciated. Note that other openFrameworks projects build just fine.

i still need to get linux running on my machine so i can't do this right now... sorry :(

if someone could update the linux projects to account for the fact that:

1 i just restructured ofxKinect to sit in the addons folder and
2 there should be a normal project file as well as a -beta project file due to the fact that the next release of OF has a different project file structure than previous versions.

Hi,

I get 3 errors 2 related to error: cannot declare field 'testApp::mKinect' to be of abstract type 'ofxKinect'

And the other cp: /Users/nardove/Programming/openframeworks_master/my apps/Kinect/PointCloud07/bin//PointCloud07.app/Contents/MacOS/libfmodex.dylib: No such file or directory

Any help will be much appreciated

• rS

maybe even write a wrapper with ofxXmlSettings to load from an xml file.

by default, use the parameters from nicolas burrus http://nicolas.burrus.name/index.php/Research/KinectCalibration

this is essential if you want to:

1 get an rgb value for each depth point
2 get proper xyz location of a given pixel

I see a bunch of new functions etc that have been added here and there over time and no comments!

For instance, what do the following functions return? meters, mm, cm, km, float 0-1, ... ?

float getDistanceAt(int x, int y);
float getDistanceAt(const ofPoint & p);

Sure, "self documented code" and all that, but in this case we need to know what we are getting back.

It dosen't take very much time to comment your work and helps the rest of us out. More and more people are using this addon and, so far, the headers are their only source of documentation.

As far as I'm concerned, an update isn't finished until its been commented and tested. Also, I find having to explain an addition in a sentence or 2 is a good usability test.

very useful for a variety of applications, but maybe it should be handled by a separate depth map processing addon, or a separate class?

http://www.flight404.com/blog/?p=472

considering how many possibilities are available, i feel like maybe we should start to separate kinect interfacing from depth processing?

if you want the fastest possible framerate, it's best to disable the rgb/ir image.

its at the end of the void ofxKinect::threadedFunction()

remove these two lines:
freenect_set_tilt_degs(kinectDevice, 0);
freenect_update_tilt_state(kinectDevice);

See http://theo.tw/i/__calibratedpixels-issue.png
Looks like the math is doing the opposite.

I understand the need for the libusb logging for debugging the usb calls ... but can't this be a define somewhere for those working down in freenect that is normally off for the rest of us? I tried setting the libfreenect logging level but no change.

10 prints a second by default sucks.

because libfreenect uses a reference to libusb.h by saying:

#include <libusb-1.0/libusb.h>

instead of the old:

#include <libusb.h>

now libusb.h isn't recognized anymore simply by dragging/dropping it into a file. you also need to add it to the header search paths.

future solutions could be: waiting for a stable version of libfreenect and modifying it for os x, or creating a framework for libusb that can be dropped in and manage its own paths.