Overview of popular AR frameworks

Hello! Recently, we realized that we have grown to a corporate blog on Habré, although we have been working on the development market for ten years. During this time, managed to fill a lot of cones and accumulate life experience. Our team now has enthusiasts who are ready to share their expertise and discuss their favorite topics here with colleagues.

We plan to talk about in our blog about:

• Augmented Reality
• blockchain and everything related to it;
• highly loaded sites and services (our favorite section);
• mobile development;
• experience in creating custom IT products.

For the first post, we prepared an overview of some popular cross-platform AR frameworks that existed at the beginning of 2018.

Recently, orders for developing applications using augmented reality have increasingly begun to come to us. Therefore, we decided to review the currently existing AR frameworks.

Most of our customers ’requests were related to image recognition (poster, cover, page, etc.) and overlaying any description or accompanying video. Thus, usually we needed to: recognize the target image and render the object.

We tested the following AR frameworks:

Framework	Price
	Free	One time	Periodical
Wikitude	Only trial	-	2490/2990/4490 € per year
EasyAR	+	499 $	-
ARToolkit	Open source
Vuforia	+	499 $	99 $ per month

Testing was carried out on the Android platform.

Wikitude

This framework left the best impression, which is probably why it is more expensive than the others. To begin with, they provide an online studio for overlaying simple static augmented reality objects. To do this, upload the target image to the studio, add AR objects, generate JavaScript code and paste it into your project. Thus, all rendering falls on the ArchitectView from the Wikitude JS SDK. The studio is as follows:



To place simple static objects in our UI component, it is enough to initialize ArchitectView with the license key of the developer and transfer the path to the environment generated by JS AR studio. In the simplest case, this is all that is needed to recognize Image targets and overlay augmented reality. In this case, the Wikitude JS Android SDK does all the work related to recognition and rendering.

But what about the native?

If necessary, you can transfer json objects to native code from JS code. To do this, you need to implement ArchitectJavaScriptInterfaceListener and add a listener to ArchitectView.
But what to do if we want to take on the rendering and add dynamics to our AR or somehow customize the behavior? To do this, you can write your JS code using the Wikitude JS SDK, or if we need performance and full control over rendering, Wikitude provides a Native SDK.

To work with this extension, we need to pass wikitudeSdk an implementation of the ImageTrackerListener interface, which contains callbacks for object recognition, loss, etc. and a link to our custom rendering (this could be InternalRendering or Externalrendering). In essence, this means that in the internal implementation of GLSurfaceView, the Wikitude SDK is provided, and in external, all openGL rests with the developer. In this case, the studio is used only to generate the wtc database of target images, which we then track.

Also, this framework supports work with unity and the integration of C ++ plugins.

EasyAR

EasyAR, unfortunately, does not provide any tools that make life easier for the developer. All we have is an SDK, instructions for running their examples, a little documentation that describes the basic principles of object recognition and documentation for C ++, it’s basically enough to familiarize yourself with the SDK classes, because the whole environment is a wrapper over the pros.

The targets are pictures and their description in json format. To work, we need CameraDevice to provide access to the camera, CameraFrameStreamer to transfer data from the camera to the tracker and ImageTracker itself. To track targets, we continuously poll the status of the tracker.

In the provided example, they have a certain entity, let's call it AREnvironment, which contains initialization of SDK and trackers, work with the camera, initialization of OpenGL. This class also describes what and where to render (it turned out to be a certain God object, in which most of the work is mixed). This AR environment aggregates GLView, GLView is in turn called from activity in onResume and onPause:

public class GLView extends GLSurfaceView {
   private com.example.developer.easyartest.AREnvironment AREnvironment;
   public GLView(Activity activity) {
       super(activity);
       …
       this.setRenderer(new GLSurfaceView.Renderer() {
           @Override
           public void onSurfaceCreated(GL10 gl, EGLConfig config) {
               synchronized (AREnvironment) {
                   AREnvironment.initGL();
               }
           }
           @Override
           public void onSurfaceChanged(GL10 gl, int w, int h) {
               synchronized (AREnvironment) {
                   AREnvironment.resizeGL(w, h);
               }
           }
           @Override
           public void onDrawFrame(GL10 gl) {
               synchronized (AREnvironment) {
                   AREnvironment.render();
               }
           }
       });
   }
...
}

The documentation contains only basic principles and class specifications. There are no comprehensive full-fledged guides at the moment, so to figure out how to build the framework into your project, you need to understand their examples. In the examples, the code is mixed: rendering depends on the standard OpenGL and on their internal data structures, it is also tightly bound to the target images and camera.

ARToolkit

I wanted to take a look at existing open source projects, after googling, the choice fell on ARToolkit. I downloaded their examples and the first thing that surprised me was the version of the collector:
classpath 'com.android.tools.build:gradle-experimental_b.2.1'.

Experimental version of the hail despite the fact that the last stable release of the library was in March 2016? It looks a little strange, maybe they just didn't update the samples. Okay, try to build the project, it crashes. Probably, you need to delve into the hail, but they did not begin to allocate time for this, nevertheless, much was not expected from open source. If you run diagonally through the project code, it becomes clear that all work with AR occurs in C ++ code, and in the native - rendering.

This solution is probably usable, but it did not suit us because of a higher entry threshold. However, if you have a desire to code on the pros or consider the framework from the inside, then study it in more detail.

Vuforia

Vuforia, like Wikitude, is a pretty powerful tool. They also have an online studio, but it can only generate a database of target images and show the points at which recognition occurs:



But unlike Wikitude, here you can upload as targets: an image, a cuboid, a cylinder or a 3D object.

In general, their examples are similar to the EasyAR examples, but there is a separation between rendering, the AR environment, and application components. Therefore, understanding their code is easier, and cutting out any components and immediately using them in your project without going into details of the implementation will not be difficult. In addition, this framework has more extensive documentation in comparison with EasyAR, but they generalized it for all platforms and did examples on a similar pseudo-code, which Wikitude did not have (those guys got confused by full-fledged guides for each platform). Therefore, samples to help.

ARCore

Mention should be made of ARCore. Google immediately warns that this is currently a preview version: ARCore is currently in preview. There might be breaking changes before the 1.0 release.

The framework currently supports motion tracking, environmental awareness, and lighting assessment. Rendering will be done through OpenGL or unity. Examples can be downloaded from the developers.google.com/ar official site, but so far the range of devices on which ARCore can be used is quite limited. As a commercial use, this solution is too early to consider.

To summarize:



Thus, if you want to do something simple with a minimum entry threshold, then choose Wikitude, if you plan to render objects yourself, Vuforia or EasyAR will be cheaper, and if you want to delve into a lower level and understand how the framework works inside - ARToolkit is a good option.

Note that solutions were considered primarily for target recognition and AR overlay. In this paper, we did not consider many other framework functions: SLAM, cloud, etc. For more information, see the official sites.

Resources:
www.wikitude.com
www.artoolkit.org
www.easyar.com
www.vuforia.com
developers.google.com/ar