Optical See Through Calibration

→Return to System Configuration

Optical See Through calibration

One way of visualizing an Augmented Reality scene is the application of see-through displays, which augment the user’s direct perception of the real world. The augmentation is thus not created by rendering virtual content on an image stream of the real world but is directly overlaid onto the user’s view of the real world. The virtual content is therefore positioned in a viewing volume, which is built up from the user’s eye and the head-mounted display. Within the AR scene, the combination of the user’s eye and the HMD therefore is represented as an own coordinate frame (see figure above). The objective of an optical-see-through calibration is the determination of the different unknown parameters in the see-through AR scene:

• the intrinsic parameters of the camera which consists of the user’s eye and the HMD
• the relative position of the visualization camera coordinate system and the “Eye-HMD” coordinate system to each other

To calibrate these parameters, metaio offers a tool for optical-see-through calibration, which is based on a setup of a camera, which is rigidly connected to a see-through display device. During the calibration the user is asked to identify corresponding points of real world features and their 2D representation in his view in the display. Using these correspondences the needed parameters – as specified above – can be calculated. The result is a calibration file, which can be loaded in the Unifeye graphical user interface, to activate the optical-see-through state and set the required parameters accordingly.

Optical-See-Through Calibration Tool

This tool allows to perform an optical-see-through (OST) calibration for a see-through Head-Mounted-Display (HMD). The calibration is realized by collecting correspondences of 2D points (in the HMD display) and tracking poses (from the HMD visualization camera to a given point in 3D). The tracking a given tracking object (e.g. a marker) which is described in the “Configuration file for marker and supporting geometries”. The necessary steps to perform an OST calibration are listed below.

First some configuration is required:

• Make sure that the screen resolution of your primary screen corresponds to the screen resolution of your HMD!
• Load the camera image of the camera which is attached to your HMD and set the resolution of the camera based on available camera calibration data.
• Load a camera calibration file for the visualization camera. A message box will inform you in case camera and calibration resolution do not match.
• Optional it's possible to set a different initial pose for the calibration. Click on the "Set" button in the "Initial Pose" section to open a dialog with the option to set translation and rotation values. Enter your initial pose and press ok to continue.

• The “Configuration file for marker and supporting geometries” defines the tracking target (e.g. marker) used for tracking during the calibration process and poses for 18 supporting geometries that can be used to guide the user while making the calibration shots from different angles of view. A default configuration file is given with the installation (tools/OSTCalibration/config/AS_OST_Tracking_Data.xml) and loaded on startup. As default a marker with ID 1 and size 140mm is used for calibration. You can also load your own configuration files but this is only recommended for advanced users. The easiest way to create your own configuration files is to make a copy of the default AS_OST_Tracking_Data.xml and then edit that copy. Please make sure that the original file is not deleted or overwritten! The configuration files used always have to contain the same number of Sensor-COSes and COSes as the default file. The x- and y-translation-values of the Sensor-COSes have to be set to 0 because otherwise the geometries will not be bound to the cross indicators used for aiming.
• Choose the 3D point you wish to use as target in the calibration process (The coordinates have to be provided relative to the origin of the marker). Note: The supporting geometries used for guiding the user into different angles of view will only be positioned correctly if the 3D point is set to (0,0,0).
• If you want to be guided in different angles of view by matching a wireframe model of a marker with the real marker while taking the calibration shots check the “Show supporting geometries” checkbox. (If the target 3D point is not set to (0,0,0) a warning message will be displayed.)
• Finally choose an “OST result file” to store the calibration results to.

The “Start calibration”-button will now be enabled. By clicking it the calibration process is started. The tool changes to fullscreen mode and the background-image delivered by the camera is disabled. The camera image is only used for tracking.

• Aim the displayed cross at the defined target (If you have set the “Translation of target 3D point from marker origin” that’s the point to aim at otherwise it’s the center of the marker). When a tracking pose is available the cross is surrounded by a circle (you can only take calibration shots if a tracking pose is available otherwise an error message will be displayed).
• Press the ENTER-key or the left mouse-button to take a calibration shot (which means that you collect a correspondence). The cross will move to a different position.
• Change your viewing position. For a good calibration, different shots from different distances and orientations are important. (If you checked the “Show supporting geometries” checkbox you will be shown a wireframe-marker that can be used as a template to find the next viewing position by trying to fit the the real marker into the wireframe model of the marker. It is not compulsory to use these views but maybe helpful in achieving perspective variety.)
• Repeat the first three steps several times. A good calibration usually should consist of 18 shots from different angles of view! (At least 6 shots from different angles of view have to be taken to compute a calibration result.)
• Cross positions (and positions of supporting geometries) can be skipped without taking a calibration shot by pressing the N-key or the middle mouse-button. This might be necessary if the displayed cross can’t be aimed at the target 3D point without “loosing” the tracking.
• Shots can be deleted by pressing the DEL-key or the right mouse-button (you can go back as many shots as you like).
• When enough shots are collected, the calibration can be finished by pressing the ESC-key.

The collected shots are now handed to the OST calibration component which calculates the result using a non-linear approach based on the Levenberg-Marquardt algorithm. The result is stored in the specified “OST result file” and is also available locally for a result check: Pressing the “Preview of calibration result” button will load a model of the marker used for calibration as a wireframe geometry to check the results of the calibration. If you look at the calibration marker through the HMD the wireframe model of that marker should be aligned properly to the real marker. Press the ESC-key to return to the calibration GUI.

In case ART Tracking is used for the calibration process, the results of the calibration can directly be included in the ART Body file.

• Click on the "Apply Calibration on ART Body File" button at the right bottom of the calibration tool to open a dialog in which you can load an “OST result file” and an “ART Body file”. If both files are loaded press ok to apply the calibration result to the ART body file.

• Note that also the “OST result file” will be changed, the pose data will be set to a (0,0,0) translation vector and a (0,0,0,1) rotation quaternion.

Using Optical-See-Through Calibration in Unifeye GUI

After the optical-see-through calibration is done, you can load resulting OST calibration file into Unifeye using "OST Configuration" tool in the Configuration panel of the Unifeye GUI (see Figure below).

By clicking the "Perform new OST Calibration" button, you can start the OST-Calibration Tool and perform a new optical-see-through calibration. The resulting calibration file can be loaded using file-selection-text field and activated by clicking the "Activate OST" button.

→Return to System Configuration

-- SupportMetaio - 2011-01-21