Git Product home page Git Product logo

js-aruco's Introduction

js-aruco is a port to JavaScript of the ArUco library.

ArUco is a minimal library for Augmented Reality applications based on OpenCv.

Demos

100% JavaScript (see details bellow):

3D Pose Estimation:

Visual Debugging:

Videos

Webcam video adquisition

3D Pose estimation

Visual Debugging

Markers

A 7x7 grid with an external unused black border. Internal 5x5 cells contains id information.

Each row must follow any of the following patterns:

white - black - black - black - black

white - black - white - white - white

black - white - black - black - white

black - white - white - white - black

Example:

Marker

Usage

Create an AR.Detector object:

var detector = new AR.Detector();

Call detect function:

var markers = detector.detect(imageData);

markers result will be an array of AR.Marker objects with detected markers.

AR.Marker objects have two properties:

  • id: Marker id.
  • corners: 2D marker corners.

imageData argument must be a valid ImageData canvas object.

var canvas = document.getElementById("canvas");
    
var context = canvas.getContext("2d");

var imageData = context.getImageData(0, 0, width, height);

3D Pose Estimation

Create an POS.Posit object:

var posit = new POS.Posit(modelSize, canvas.width);

modelSize argument must be the real marker size (millimeters).

Call pose function:

var pose = posit.pose(corners);

corners must be centered on canvas:

var corners = marker.corners;

for (var i = 0; i < corners.length; ++ i){
  var corner = corners[i];

  corner.x = corner.x - (canvas.width / 2);
  corner.y = (canvas.height / 2) - corner.y;
}

pose result will be a POS.Pose object with two estimated pose (if any):

  • bestError: Error of the best estimated pose.
  • bestRotation: 3x3 rotation matrix of the best estimated pose.
  • bestTranslation: Translation vector of the best estimated pose.
  • alternativeError: Error of the alternative estimated pose.
  • alternativeRotation: 3x3 rotation matrix of the alternative estimated pose.
  • alternativeTranslation: Translation vector of the alternative estimated pose.

Note: POS namespace can be taken from posit1.js or posit2.js.

Flash Demo (deprecated)

It uses Flashcam, a minimal Flash library to capture video.

js-aruco's People

Contributors

jcmellado avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

js-aruco's Issues

Set a matrix3d transform on an element ?

Hi,

I have a question about ArUco js capability, as I don't find any doc on it : is it possible that ArUco get its tracker position as a 3D matrix ? I'm just trying to make, let's say, a blue square (html element) that would follow its transformation using css transform matrix3d() property.

I found in the example, a function that returns and update object rotation / scale / translation. I would basically do the same thing, updating a CSS property transform: matrix3d().

Any idea ?

DrawImage

I used the example webcam live demo and it works very well ... doing various experiments have not been able to use DrawImage instead of strokeText to insert an image on the marker ... how can I do?

`
function drawImage(markers) {

    var corners, corner, x, y, i, j;
    for (i = 0; i !== markers.length; ++i) {
        corners = markers[i].corners;
        xMin = Infinity;
        yMin = Infinity;
        xMax = 0;
        yMax = 0;
        for (j = 0; j !== corners.length; ++j) {
            corner = corners[j];
            xMin = Math.min(xMin, corner.x);
            yMin = Math.min(yMin, corner.y);
            xMax = Math.max(xMax, corner.x);
            yMax = Math.max(yMax, corner.y);
        }
        my_W = (xMax - xMin);
        my_H = (yMax - yMin)
        base_image = document.createElement('img');
        base_image.onload = function () {
            context.drawImage(base_image, xMin, yMin, my_W, my_H);
        };
        base_image.src = "img/photo.jpg";
    }
}`

I think the refresh rate is too high to upload and insert an image in this way ... someone can help me?

How to calculate pitch/yaw/roll of offset object?

Hi!
I have a question regarding pose:
I want to calculate the angles of a marker. This is fairly simple using the rotation matrix when the object is right in front of the camera, but I'm having a hard time doing it when the marker is offset to the center of the camera.

i.e. If the marker in question is directly in front of the camera, I can use the rotation matrix directly to calculate the marker's rotation relative to the camera position. My problem begins when I "pull" the marker along the X-axis (right/left), without rotating it. if the initial angle was 45 deg (when the marker was directly in front of the camera) and then I moved it left a few cm, but haven't rotated it, I need my angle measurement to return the same 45 deg as it did earlier.

How can I calculate it using the data js-aruco provides?

Thank you very much for your help,
Dan

6X6 improvement

Nice work! Thank you so much for sharing.

I try with the original Aruco code and it worked perfectly. However when I try with 6x6 it not working. Can you help me to know how to make this work with 6x6?

The references :
https://chev.me/arucogen/
Dictionary: 6x6(50,100,250,1000)

Thank you again!

code without minified

Hi, I wanted to ask if you have the code without minified? I would like to have it and want to convert it to es6 module.
I look forward to your reply.

working webworker ?

Impressive work! i just stumbled into it!

is that possible to run it in a webworker ?

Scanning 3x3 marker

Awesome work, loved it man!

Had tried with 7X7 marker and works like a charm.

Just one question, I have gone through your documentation and you have stated clearly that it will work on 7X7 marker, whereas here I am in the need of scanning 3X3 marker, so how would I achieve that?

Frankly speaking I am novice, so please try to be nice with me on this. Any help would be appreciated. ๐Ÿ‘ :)

Thank you so much in advance.

Perspective transformation based on markers position

Hi, @jcmellado and thanks for this amazing library.

I wonder if it's possible to apply a perspective transformation using the CV.js methods, I tried using CV.warp (and converting the returned array into a Uint8ClampedArray), but I'm having trouble with the warpSize parameter.

My goal is to remove the perspective distortion of the image (I have aruco codes in each corner), any help would be much appreciated.

Thanks!

Some js errors in posit1.js

Hi

Im playing with js-aruco, it is a lot of fun

Nevertheless i often get an js exception as error1 and error2 being undefined in line 118. it seems to occurs randomly when i move the marker in front of the camera. it occurs even with your debug.html.
i know this is old code, but it gained popularity with the gogglepaper.com stuff :)

Any hint on how to fix it ?

from https://github.com/jcmellado/js-aruco/blob/master/src/posit1.js#L118

  return error1.euclidean < error2.euclidean?
    new POS.Pose(error1.pixels, rotation1, translation1, error2.pixels, rotation2, translation2):
    new POS.Pose(error2.pixels, rotation2, translation2, error1.pixels, rotation1, translation1);
};

Samples not working

What steps will reproduce the problem?
1. Click the samples provided at http://code.google.com/p/js-aruco
2. On IE9 and Google Chrome 17.0 versions

What is the expected output? What do you see instead?
I expected there should be a flash window displaying atleast video feed.

What version of the product are you using? On what operating system?
v3

Original issue reported on code.google.com by [email protected] on 20 Mar 2012 at 2:35

Reference

Hi, I find your work really interesting, but I was wondering if you have a reference to this library.
I cannot really understand how it works just by reading the sample codes.

I'm trying to recognize one (or ideally two) markers, all I would need to retrieve is their position and size on the canvas and, if possible, rotation.

Any help will be appreciated,

Thanks!

Is this repo dead?

There wasnt a contribution in teh last two years. I would be really happy to have aruco available in javascript. @jcmellado What are the chances this lib get updated?

Position for specified marker card

Hello,

is there a way to check if a marker card
is new in the webcam stream?

I want to use the same marker twice and set models
on the positions with ThreeJs but if I move one card, I dont know
which model is the right one for the card I moved.

A solution is to remove every model every frame and place new ones,
but thats very ugly I think. 

Do you track the markers?
A unique ID for a marker would be nice.

This way I would be possible to bind a 3D model to a specified card.

Is there already something like this?
I hope you can help me. 

BTW Nice library, the results are very good, even with low quality videos.

Thorsten




Original issue reported on code.google.com by [email protected] on 10 Jun 2012 at 6:33

Id Markers

Hello Juan, I am studying the JS-Aruco library and saw how to generate the fiducial marks; however, I am unable to get the ID of the markers as captured in the image. Do you have any link with the example? (Found in the generation of markers, a reference to markers [0] .id, however, it seems that this attribute is not instantiated by default in AR.detector).

Able to get online demos working, but cant get it to work when I recreate it locally?

What steps will reproduce the problem?
1.Copy the demo to local folder
2.Click the html file in folder and display in chrome
3. Webcam permission is asked, but when you click allow no video is displayed.

What is the expected output? What do you see instead?
Expect to see video stream, see nothing instead

What version of the product are you using? On what operating system?
I am using Windows 7, Google Chrome 20.

Please provide any additional information below.

It works fine if I do it on the website 
http://inmensia.com/files/aruco/getusermedia/getusermedia.html

but wont work locally, your help will be greatly appreciated!!

Original issue reported on code.google.com by [email protected] on 9 Jul 2012 at 5:34

samples/debug-posit shows incorrectly posed rectangles

I ran debug-posit.html on a stationary marker cube. Attached are three captured frames, and a composite.

Several solid-color rectangles are shown with implausibly incorrect orientations. These orientations are stable, not transients.

I have not explored whether these are caused by the sample display code, and/or by the underlying pose estimators.

debug-posit--composite
debug-posit--frame1
debug-posit--frame2
debug-posit--frame3

Three.JS Demo isn't working

What steps will reproduce the problem?
1. Go to this URL: http://inmensia.com/files/aruco/debug-posit/debug-posit.html
2. Share the video feed,
3. Use the 963 AR marker (as per the video example)

What is the expected output? What do you see instead?
Expected output is a 3D earth shows at the marker - instead no earth is showing.

What version of the product are you using? On what operating system?
Using the base version on the website.

Please provide any additional information below.
I am *sure* this used to work but I can't get it to work on any of the systems 
I have here now.

Original issue reported on code.google.com by [email protected] on 6 Mar 2015 at 2:19

Suggestion on better pose estimation ?

js-aruco is great at detecting the markers. It is fast and simple, i love it :)

It is a little less cool about the pose estimation, using posit1 or posit2 seems to produce significant inaccuracies unfortunatly. So it makes it unsuitable to add some 3d on top of the marker without having significant issues.

Is it me using it poorly ? (i experience it even on the examples of this repository too)

I would love to help or to implement other alternative for pose estimation... but im a bit lost on which algo to implement. Can you suggest which one i could use ?

Any suggestion would be great :)

Marker id more than 1023

Hello)
Please help, i am develop ar kids book and me need more than 1024 marker. How can i increase the count of the marker id?

Marker detection range

Hello,
I have performed marker detection with demo code with different resolutions and was able to perform maker detection up to 2 meters (approx).

If anyone able to achieve maker detection beyond this range please do share configuration. It will be a great help.

Commonjs?

Are you interested into porting this to npm?
May I bother you with questions in order to get it working?

How to create custom AR codes?

Hi Juan,
Just wanted to say that JS-Aruco is a great library - I have been experimenting with it for the last few days. I have a question, how do you create the AR codes with custom IDs? Do you have or use some software the you can recommend? thanks,
Antonio.

sorting the contours

Hi,

Thumbs Up..!! for the amazing scripts.

I would like to know if there is a method to sort the contours.

Thanks in advance.

averaging several markers to increase robustness ?

Hi,

This is not to report a bug, this is more about asking to an expert in ar. When using the library, it is amazing work. but the pose reported are shaky if done over a single marker. maybe we could go further. This is likely a naive question im just starting in the field :)

i was thinking "could we improve the robustness by having multiple markers and average them?". do you have code or pointer on how to do that ? i included a screenshot of a toy im playing with. you can see the detected markers on the topleft. i would like to average all the markers.

screen shot 2015-04-11 at 13 44 33

here is a video of somebody doing the same i think https://www.youtube.com/watch?v=CzD48UkGsK8

i would take pointer or code about how to do that :)

thanks

Memory leak with getImageData()

Hey! Thanks for the awesome library, I'm using it for a graduate computer vision course at Boston University.

I've been playing around with your demos, and noticed a memory leak that occurs when you try to run them on large (HD) video sources. Thankfully, it's not a problem with your code, but instead with the way you're grabbing image data from the 2D rendering context.

In your debug and debug-posit examples, you have the following function updating a 2d canvas context:

    function snapshot(){
      context.drawImage(video, 0, 0, camera.width, camera.height);
      imageData = context.getImageData(0, 0, camera.width, camera.height);
    }

With HD video sources, calling context.getImageData() multiple times quickly causes memory usage to skyrocket (100's of MB per second).

For now, it seems like I'll have to hack something together where I downsample the HD video stream before writing to/reading from the canvas in order to get the image into the correct format for this library.

How tough would it be to add support for images in formats other than ImateData from a canvas object?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.