JavaScript Augmented Reality - JSARToolkit test

JSARToolkit is a JavaScript library ported with FLARToolkit (Flash) and designed to track AR Markers on video. ARToolKit converts data from markers into 3D coordinates, using them you can overlay images or 3D objects on a flat surface.

You've probably already seen JSARToolkit in action on the Ilmari Heikkinen demo - Remixing Reality .
The Ilmari demo is part of Mozill's “ Web O 'Wonder ,” a site showing off new technologies to be added to Firefox 4 .

Research HTML5 clips

The customer set us the task - to evaluate the possibility of using JSARToolkit for online HTML5 clips. (We were asked to consider only those users who used the latest version of Firefox and Chrome)
Here are some of the questions that we would like to answer:
- Will the processing be fast on slow computers?
- How many AR Markers can we track at one time?
- How fast can you move the marker so that it becomes un trackable?
- What is the greatest distance at which the camera can track the marker?

The answers to these questions, source code and demos can be found below.

Video recording

I used Flip Ultra HD to record test video. Flip Ultra HD video quality is acceptable, considering that this is not a professional camera. We knew that we won’t get anything good from such a camera, but that was enough for the test. The main problem with such a low-quality camera is the inability to switch the shutter speed.

This meant that there was nothing we could do about the blurry AR Tags if we moved too fast. We were surprised at how quickly we lose the marker, as we move from side to side.
Nevertheless, we are sure that when shooting in a well-lit studio using a camera with a high shutter speed, there will be very few unreadable AR marks.

Printing AR Markers

I printed a few markers that came with JSARToolkit and started taking test shots in the kitchen. I did not even expect that everything will work the first time, each video added to the library turned out to be working.

Some of the results were not high-quality, but I want to repeat that the quality of our camera was poor compared to professional ones. We shot videos without focusing on individual markers, which could add accuracy to their tracking.

Recoding video in VP8 WebM

We recorded video in H.264 Mpeg format. For everything to work with HTML5 video, we had to convert the video to WebM. The first video decoder I tried was Ffmpeg2Theora, judging by the name, it was supposed to encode the video in WebM. However, I realized that there are problems with Ffmpeg2Theora. When I encoded the video under Linux, it might not play on Windows and vice versa.

After testing other encoders, I settled on Miro Video Converter. Unfortunately, Miro is not able to process several videos, but the video that it produces works on all OS and browsers.

Wrap Creation

I wanted to write a simple JSARToolkit-based API that could be reused. Something that I could connect to another JavaScript library, for example Popcorn.js. The code I found in the Ilmari demo was task specific. Since the code was not full of comments, I had problems understanding what was going on. I had to experiment. I came up with an easy way to write a wrapper for JSARToolkit.

The first step in using the JSARToolkit wrapper is to install your tracker. An example of how to do this:

// Пример создания трекера со всеми возможными опциями
    var myTracker = jsartoolkit.tracker({
        src       : 'my-video.webm',                  // Исходник для видео
        autoplay  : true,                             // Стоит ли включить видео сразу
        repeat    : true,                             // Включить повтор
        volume    : 0,                                // Звук из видео
        target    : doc.getElementById('DOMTarget'),  // DOM element в который будет добавлен canvas
        width     : 720,                              // Ширина кадра
        height    : 360,                              // Высота кадра
        threshold : 100,                              // Настройка освещенности кадра
        ratio     : 0.5,                              // Настройка размера скрытого canvas для трекинга (1 = 1в1 как видео)
        debug     : false                             // Выводить ли отладочную информацию - для порога освещенности кадра
    });

Once the tracker was created, the next step was to add content to the markers.
We added a static image, and then a 3D object exported from Blender3D:

    // Добавления картинки первому маркеру
    myTracker.marker(0).image('my-image_01.png');
    // Добавление модели Blender3D
    myTracker.marker(2).model('HTML5_Logo001');

This example shows how to update marker properties after it has been created:

    // Настройка свойств маркера 0
    myTracker.marker(0)
      .scale(1)
      .axis(0, 0, 1)
      .angle(0)
      .position(0,0,0)
    ;

You can also add more complex behavior using JSARToolkit-Wrapper. The following example demonstrates how to update marker properties in real time. This code causes the first marker to spin and pulse:

    // Анимация свойств маркера Marker_0 по таймеру
    var interval = global.setInterval( function(){
      var date    = + new Date(),
          scl     = 1.5 + (Math.sin( date/200 ) * 0.5),
          axs     = Math.cos( date/300 ),
          posX    = Math.sin( date/300 ),
          posY    = Math.cos( date/300 )
      ;
      myTracker.marker(0)
        .scale(scl)
        .axis(0, axs, 0)
        .position(posY, posX, 0)
        .angle(date / 230)
      ;
    }, 20);

To access the video tracker, you can do something like this:

    // Доступ к видео элементу трекера и установка currentTime
    myTracker.video.currentTime = 1;

Answers on questions

Will processing be fast on slow computers?
Video processing and marker positioning is very fast. I almost did not notice the difference in tracking 1 marker and 100 markers. The main burden is overlaying content on the video.

How many AR Markers can we track at one time?
I tracked 100 markers at the same time without any problems.

How fast can you move the marker so that it becomes un trackable?
It all depends on your camera and how you shoot. If we use a camera with a high shutter speed, such as those used for recording sports, we will get a minimum of blurry markers (if at all) and they will be tracked very well.

What is the greatest distance at which the camera can track the marker?
Again, it all depends on several factors - the speed of movement of the camera / objects and the lighting. In a well-lit room (not a professional studio). I managed to track Markers from 10 meters in the direction of the lens in 720p resolution. The higher the resolution of the camera, the higher the quality of tracking markers. One thing worth noting: you can shoot your video in 1080 resolution, cache tracking results and reduce the number of processing on the client side. You can spin the ratio if something is poorly tracked or threshold if the frame was poorly lit.

Conclusion

pros

- Easy to implement
- Optimal tracking algorithm, does not load the processor
- You can track at least 100 markers
- You can export directly from Blender3D
- You can overlay any content: pictures, video, 3D objects

Minuses

- Too many global variables in the source library code
- You need to tweak the export from Blender3D a little
- Multiple videos are not supported simultaneously
- Too many expensive calls getElementById ()

You have to do a lot of work if you want to use this library on production or make it part of a large library. But after a huge number of tests and experiments, I will say that this code works amazingly!

Examples, Tests and links

JSARToolkit-Wrapper Demo
JSARToolkit Video Tests
JSARToolkit Marker Images
JSARToolkit-Wrapper on Github

You need the latest version of Firefox 4 or Chrome to view it.

From translator

Unfortunately, the author has not yet posted the demo. The use of AR in the form in which it is presented in translation is not very wide, but with the advent of HTML5 Device Element ( another article ), everything can change.
AR Soon in your browser;)