Planar Markerless Tracking Configuration Tool

Introduction

The planar markerless tracking configuration tool lets you create, load, modify and save tracking configuration files for markerless tracking (.xml).

Planar markerless tracking means that the optical tracking component of Unifeye will track the position and orientation of a given image in the image source. You are free to choose any image you want, though some images are better suited for markerless tracking than others. Therefore this tool provides you with additional mechanisms to measure the quality of your selected images and to optimize your tracking results.
For a general overview of supported tracking systems please go here.

First steps

This three step tutorial will show you how to create a simple markerless tracking configuration using the planar markerless configuration tool.

1. Starting this tool

The planar markerless tracking configuration tool can be reached from the Unifeye Design menubar entry "Tools"->"Planar Markerless Configuration"

The main window of the planar markerless configuration tool looks like this:

(This tutorial explains the standard view. The expert view is explained below.)

The five buttons in the upper right allow you to:

1. Create an empty, new tracking configuration.
2. Open an existing tracking configuration.
3. Save your tracking configuration.
4. Save your tracking configuration under a different name/path.
5. Exit this tool.

At the bottom of the main window are four additional buttons, three on the lower left and one on the lower right. Except for the leftmost button these buttons are deactivated upon startup of this tool. From left to right they allow you to:

1. Add one or multiple new image(s) to this tracking configuration
2. Edit the settings of an image.
3. Check the quality of one or multiple image(s).
4. Delete one or multiple image(s) from this tracking configuration.

2. Creating a simple example tracking configuration

The first thing we will want to do when creating a new markerless tracking configuration is adding an image. Click on the "Create Patch" button (the colored button on the lower left of the main window). This will open the "Edit patch" window:

This window allows you to:

1. Load an image (Note: it can be used for loading multiple images as well)
2. Set it's physical size in millimeters
3. Edit image regions

(For this simple tutorial, we will ignore the "Edit Regions" button. It's functionality is explained in section "Adding/editing image regions".)

The first step is to load an image. Im principle you can choose any image you want (as long as it is encoded as JPG, PNG or BMP). However, for the course of this tutorial we will use this image. Download the image to your hard drive and load it into the markerless tracking tool by pressing the magnifier button on the right of the "Load image for patch" text field.

The second step is to specify the width and height of the image. It is assumed that you printed out the image in order to use it in an AR environment. Set the width and height accordingly (e.g. 284x210mm for a full A4 print of the example image used in this tutorial).

Finally, press the OK button to finish.

3. Saving and using the tracking configuration

After adding the image to the empty tracking configuration the main window should display the image in the "Markerless Patch Pool":

Using the four buttons on the bottom of the main window we could add another image, edit this image again, check it's quality (should be very good!) or delete it. We will save the markerless tracking configuration file to the hard drive instead, using the "File"->"Save" menu entry.

To use this markerless tracking configuration file in Unifeye Design, go back to the main window of Unifeye Design, click on the configuration category in the upper right, go to the Tracking Configuration tool and then select the tracking configuration file that you have just stored on your hard drive:

After the markerless tracking configuration is loaded, start your camera in Unifeye and hold the printed out example image in front of the camera. You should see your geometry placed at the position of the printed out image and should be able to rotate it and move it around. It is assumed that you know how to start your camera in Unifeye and know how to load a 3D model into Unifeye and assigned it to COS 1 (If you are not familiar with these things, read the Unifeye GUI documentation first).

On success, your result should look something like this:

Note 1: The markerless tracking configuration file created and used in this tutorial can be downloaded here.

Note 2: If you haven't printed out the example image, you can alternatively just display the image on your computer screen and point your camera on the screen. This is, however, not recommended.

Expert view

To switch from standard view to expert view, choose menu entry "Expert" from the "View" menu in the main window:

The main window of the expert view looks like this:

The expert view has two additions over simple view mode:

• You can select how your images should be stored.
• You can modify the coordinate systems of your tracking configuration.

Storage of your reference images

Selecting "Yes" in this radio box will store a copy of your images inside the .xml file of your tracking configuration. This means that all information required to use this markerless tracking configuration is contained within a single file. Selecting "No" will only store the path to the images. Be aware that with this option selected you need to copy the referenced images in addition to the .xml file if you want to copy your markerless tracking configuration to another machine.

Coordinate System Configuration

The coordinate system configuration let's you add, modify and remove coordinate systems to your tracking configuration. In simple view mode, the planar markerless configuration tool automatically creates one COS for each added image, with no offsets or rotations. Expert view allows you to create the COSes by hand.

In expert view, the bottom of the main window contains four new buttons (from left to right):

1. "Add Cos"
2. "Add patch"
3. "Edit"
4. "Delete"

Unlike standard view, expert view will not automatically create a coordinate system for each image you add. However, every markerless tracking configuration needs at least one coordinate system. Therefore you will have to click the "Add Cos" button on the lower left to create at least one new coordinate system. After that, you will want to assign one of your images to this coordinate system. This can be achieved by pressing the "Add patch" button. The following window should appear on your screen:

Besides selecting one of your images, this window also offers you to define translations and rotations for your COS.

Once you have assigned an image to the coordinate system, your main window should look something like this:

The "Edit" and "Cancel" buttons can be used to edit or delete images/coordinate systems.

Improving your configuration / choosing the right image

To select a suitable image for planar markerless tracking, follow these guidelines:

• The image should have good, distinguishable features
• Color gradients in images are also good
• Only text is bad
• Only white is bad
• Do NOT use a high resolution image
• Define patches to mask bad areas of an image and spread them across the image (if possible)
• The resolution of the image should be approximately the size later in the camera image (e.g. if the camera resolution is 640x480 the reference image should be around 300xXXX and try to refere to the instructions below)
• to get a good tracking performance, choose a reference template size verifying:

1. (A x B) such that min(A,B) = pow(2,n)*25, where n is a positive integer. “Good sizes” would be: (A x B) where min(A,B) is equal to 25, 50, 100, 200, 400, 800, 1600, 3200,... or slightly above, i.e. (A x B) where min(A,B) is equal to 26, 52, 105, 1606,...Avoid using sizes in-between or sizes that are slightly below these values. “Bad sizes” would then be: (A x B) where min(A,B) is, for example, equal to: 47, 98, 300, 1500, …
2. For optimal performance, additionally, choose min(A,B) close to max(A,B). Example of optimal sizes: (100 x 100), (200 x 200), … Condition 2. only makes sense when condition 1. is fulfilled.

• In rare cases, tweak quality threshold in tracking data file manually („qualityThreshold“ should not be below 0.7, or tracking will get unstable):

Compare these two images:

The planar markerless tracking configuration tool contains routines to check the quality of your images.

Adding/editing image regions

Defining regions on an image is a way to tell the markerless tracking algorithm which parts of your image are more interesting (contain more features) than others. This might be useful if your image contains unichrome areas with simply no features. You can drag a frame around the area you want to use for tracking in order to determine the resulting tracking space on your image.

To create or edit regions of an image press the button labeled "Edit Regions" inside the "Edit Patch" window. This will open a new "Region Selector" window:

Regions can be selected by pressing and holding the left mouse button while moving the mouse over the image. Each defined region will be listed in the table below the image. The three buttons at the bottom of the "Region Selector" window can be used to:

• Delete all regions.
• Edit the selected region.
• Delete a single region.

Example selection of regions in an image:

Notice how those parts of the image with most detail were selected, whereas the rather uniform part in the middle was not put inside a region.

Checking the quality of an image

The planar markerless tracking configuration tool provides you with a built-in quality check that you can use if you want to know how well suited an image for markerless tracking is. The built-in quality check for an image can be accessed from the main window of this tool. It's the third button below the list of images:

Checking the quality of an image can take a minute or more, depending on your computer, the resolution of the image to be checked and the number of regions defined.

The results of the quality check will be presented to you in a new window (results for the tutorial example image):

Images with quality results in the range of 70-100% can be considered well suited for the planar markerless tracking. If you score quality results below 70% you can try to optimize your results.

Loading multiple images

Selecting multiple images from the file system in the "Edit Patch" window will bring up a separate, new window:

This window can be used to set the width and height of multiple images at once or individually.

-- SupportMetaio - 2011-01-26