Overview over the UnifeyeSDKMobile framework

The metaio Unifeye SDK mobile is implemented in a modular way, providing mobile Augmented Reality functionality based on three low level components:

• Capture component
• Tracking component
• Rendering component

Depending on the user’s background mobile AR development with the metaio Unifeye SDK mobile can be done in different ways.

This overview image shows the four abstraction layers of our framework.

The first layer is the application layer, which changes depending on the mobile platform and is used for mainly GUI operations and application logic. As a programmer you will only use this layer. It hides all implementation details and gives direct access to the high level API layer. For non-programmers the implementation and API details are hidden through a standard application which can be easily configured based on an XML configuration file. Information on XML-based application configuration can be found in Tracking Configuration.

The high level API layer enables the programmer to interact with the UnifeyeSDKMobile framework. Several classes and convenient functions are defined in this layer.

The third layer consist of several components for tracking, rendering and capturing. They are used by the high level API layer to provide you in a convenient way our Augemnted Reality solution. The capture component start and configures the camera or can load images. This image data can be processed by the tracking component to find various markers and calculate their correct position, scale and orientation. Further, these parameters are used to update the scene graph with all loaded models, which belong to a father node representing a marker.

The last so called common layer combines all components our framework is depending on.

Fundamental approch to writing an Augmented Reality application

For nearly every Augmented Reality application you need three things: Capturing, Tracking and Rendering. Every component has to be initialized and is used during the Augmented Reality experience.

Capturing provides an image from a camera or can also be a loaded image. It has to be configured beforehand and therefore it is important to get the intrensic camera parameters. They have to be provided in a XML-file. For more information please refere to ####LINK####. On every frame the camera image has to be updated by the programmer and send to anUnifeyeSDKMobile instance. This gives the programmer the freedom supend the camera image as he chooses.

Tracking takes the captured image and calculates the position, scale and orientation of the marker relative to the camera. Before you can use the Tracking, you have to load a tracking configuration, which tells the tracking how the marker look like, to which coordinate system (COS) they are asigned to and how to configure the tracking further. This tracking configuration is also an XML-file. More specifics to the format can be look up in this tutorial. By calling the render()-method on every frame the tracking is performed and objects in the scene graph are updated.

Rendering lets you load models and assign a COS them. You can load an remove model during runtime or assign them to another COS.

Note: Removing a model from the scene graph will invalidate the used object and can cause crashes, if not handled properly.

General workflow in an Augmented Reality application

Before you continue your first tutorial, you should take a quick look at our UnifeyeSDKMobile architecture . This overview gives you some insides into our framework and several components. For an Augmented Reality enabled application, there are several needed steps to configure them right. There is always the initialization and the tracking phase. In the initialization phase you have to:

• Initialize an instance of the MobileSDK.
• Activate the camera
• Load the tracking data
• Configure your 3D-scene by
  • loading Models,
  • defining Animations,
  • loading movie textures etc.

After you have fully configured every detail, your augmented reality application will start in a loop and has to perform serveral things in the continues tracking phase in order to work correctly. All of that will be done using the render-method:

• Calling the render()-method will:
  • Display the latest image of the camera,
  • perform the tracking on that image,
  • read signals of the GPS, gyro and compass sensors,
  • update the objects in the scene and
  • finally render everything.

Now you have a general understanding what steps you have to implement in your application. The above described steps are already included in your examples, so you can use them as code base to start your development.