Metaio IR Tracking

→Return to Tracking Configuration

Introduction

metaio provides its own Infrared Tracking solution. The main benefit of this system is its robustness and the fact that it is an outside-in tracking system. The outside-in approach doesn’t need to alter the environment which should be augmented. E.g. there is no need to place a marker in the scene which has to be seen by the camera but there has to be a 3D Target attached to the camera. Robustness is achieved by using infrared light and thus creates a controlled environment for tracking. Also the environment lighting can change dynamically from complete darkness to very bright light without influencing the tracking system. These two main benefits make this tracking system particular suitable for show-systems. Another advantage is the complete six-degrees-of-freedom (6DOF) characteristics of the targets. In contrast to a planar marker the target can be seen from any angle and allows the accurate recovery of its position and orientation.

The metaio IR Tracking is based on metaio's own IR tracking cameras and arbitrary fiducial marker targets which can be configured to fit the given tracking object. Several types of IR tracking are possible:

• Mono - IR Tracking: only one IR camera is used for tracking
• Stereo - IR Tracking: two IR cameras are used for tracking

For the fiducial targets, active as well as passive fiducials can be used to provide optimal flexibility with respect to environment and lighting conditions. The following figure shows a setup for a metaio IR stereo tracking.

Configuration

To configure the Infrared Tracking system an inter-camera calibration has to be done. This calibrates the offsets between the IR tracking cameras automatically and is done in a few minutes by using the according calibration tool provided by metaio.
Depending on the goal to accomplish, a Hand-Eye-Calibration may be done afterwards. This computes the offset of the target (usually mounted on the visualization camera) measured by the IR tracking system to the image observed by the camera. This procedure also offers a convenient way to define the origin of the world coordinate system (align the real and virtual world).

The configuration has only to be done once if the setup of the tracking system or the camera changes and is done in several minutes by using the supplied Unifeye SDK toolset.

The XML snippet given below is an example of how to integrate Infrared Tracking. The Unifeye SDK also installs an example ( AS_Tracking_Data_IRStereoTracking.xml located in <UnifeyeInstallationDirectory>/config/SensorSources) that shows how the IR-Tracking can be included and configured. The camera image and the tracking data of the IR system may arrive at the computer at different times, thus Synchronization may be needed.

The SensorCOSes are defined with unique SensorCosID s and a parameter that points to a file that holds information about the optical fiducial to be tracked. A third special parameter can be added optionally to the SensorCOS information: SensorCOSDirection. Usually the tracking result is given as a transform from the IR world origin (in the master camera) to the target. Including this parameter with value TargetToWorld will invert the tracking result. An example where this case applies is given in the Offset Referencing section.

<?xml version="1.0"?>
    <TrackingData>
    <Sensors>
        <Sensor type="IRMultiViewSensorSource">        
            <SensorID>StereoTracker1</SensorID> 
            <Parameters>stereoConfig\IRMultiViewSensorSourceConfig.xml</Parameters> <!-- Sensor specific parameter: IR Tracking parameter file -->
            <SensorCOS>
                <SensorCosID>CalibAngle</SensorCosID>
                <Parameters>stereoConfig\CalibAngle.txt</Parameters>
            <SensorCOSDirection>TargetToWorld</SensorCOSDirection> <!-- optional parameter to invert the tracking information -->
            </SensorCOS>
        </Sensor>
    </Sensors>    
    <Connections>
        <COS>
            <Fuser type="BestQualityFuser">
                <Parameters></Parameters>
            </Fuser>
            <SensorSource trigger="1">
                <SensorID>StereoTracker1</SensorID>    
                <SensorCosID>CalibAngle</SensorCosID>
                <HandEyeCalibration>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset> 
                </HandEyeCalibration>
                <COSOffset>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset> 
                </COSOffset>
            </SensorSource>
        </COS>    
    </Connections>
</TrackingData>

The general sensor configuration consists of the name of the sensor ( <SensorID>) and a path to the IR configuration file within the <Parameters> section of the sensor configuration. This file contains the following information:

• Information on the camera configuration:
  • Camera information for the master IR camera
  • Camera information for the slave IR camera
  • StereoCalibration: inter-camera calibration (result of the stereo calibration process)
  • CameraSettings: settings for camera latency, maximum and slowdown frames per second, shutter, gain and brightness
• Information on the tracking parameters :
  • EpipolarDistanceThreshold
  • RayDistanceThreshold
  • DistanceThreshold
  • AngleThreshold
  • MaxReprojectionError
  • BlobCharacteristics
  • BinarizationThreshold

The parameters are described below in the sample XML file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<IRMultiViewConfiguration>
    <CameraConfiguration>
        <!-- Configure the cameras used for tracking. -->
        <Camera>
            <!-- Region of interest of the camera's image that should be used for tracking. -->
            <ROI>
                <width>1100</width>
                <height>1024</height>
            </ROI>
            <!-- The camera calibration file for the master camera. This can be either a
          ** bin file if the cameras have been calibrated based on the Luhmann distortion
          ** model or an xml file if the cameras have been calibrated with metaio's
          ** Sextant calibration application. -->
            <ID>184983684</ID>            
            <Calibration>184983684.bin</Calibration>
            <!-- The world offset (relative to the center of the master camera) to put the world
            ** coordinate system somewhere usefull. -->
            <WorldOffset>
                <TranslationOffset>
                    <x>0.0</x>
                    <y>0.0</y>
                    <z>0.0</z>
                </TranslationOffset>
                <RotationOffset>
                    <x>0.0</x>
                    <y>0.0</y>
                    <z>0.0</z>
                    <w>1.0</w>
                </RotationOffset>
            </WorldOffset>
        </Camera>
        <Camera>
            <!-- Region of interest of the camera's image that should be used for tracking. -->
            <ROI>
                <width>1100</width>
                <height>1024</height>
            </ROI>
            <!-- The camera calibration file for the slave camera. This can be either a
          ** bin file if the cameras have been calibrated based on the Luhmann distortion
          ** model or an xml file if the cameras have been calibrated with metaio's
          ** Sextant calibration application. -->
            <ID>184983685</ID>
            <Calibration>184983685.bin</Calibration>
        </Camera>
        <!-- Refers to the intercamera calibration file (R|t) from the master camera
          ** to the slave cameras. In a stereo setup this file contains only one 
          ** transformation. Paths can be passed fully qualified or relative to the 
          ** configuration file. -->
        <StereoCalibration>result.txt</StereoCalibration>
        <CameraSettings>
            <!-- Optional parameter, default: 0.0. If the latency of the camera is known, it 
              ** can be set over here. The unit of the latency is given in milliseconds. Usually
              ** the latency is only required in systems where data from multiple sensors
              ** will be fused. -->
            <CameraLatency>0.0</CameraLatency>
            <UseExternalTrigger>0</UseExternalTrigger>
            <!-- Optional parameter, default: 25.0. The system tries to reach the requested fps. 
              ** If the frame rate is not possible to achieve the system configures the cameras
              ** to the maximum possible frame rate. -->
            <MaxFps>500.0</MaxFps>
            <!-- Optional parameter, default: 0.0. Sometimes the AVT cameras (e.g. when connected 
              ** to the same bus) can not achieve the maximum physical frame rate and deliver data
              ** only at half the possible frame rate. In such cases it usually helps to slow 
              ** down the master camera by e.g. 0.5 fps. Usually this value has to be determined 
              ** empirically. -->
            <SlowDownFps>1.0</SlowDownFps>
            <!-- Optional parameter, default: 5.0. The shutter value of the camera given in 
              ** milliseconds. The minimal value is 0.02 milliseconds -->
            <Shutter>5</Shutter>
            <!-- Optional parameter, default: 5.0. The gain value of the camera. If the gain value 
              ** is choosen too high, the images will become noisy and the tracking will loose its 
              ** stability. If the value is too low, BLOBs may not be detected properly at some 
              ** distance -->
            <Gain>10</Gain>
            <!-- Optional parameter, default: 50.0. The camera's brightness parameter. The influence 
              ** is similar to the one of the camera's gain. -->
            <Brightness>50</Brightness>
        </CameraSettings>
    </CameraConfiguration>    
    <TrackingParameters>
        <!-- Unit: pixel, optional parameter, default: 2.0. The epipolar distance is the 
          ** distance between an extracted BLOB and its corresponding epipolar line. The 
          ** parameter defines the maximal distance at which such BLOBs are assumed to be 
          ** belonging to the epipolar lines. All BLOBs with a distance bigger than this 
          ** value are assumed to be false alarms. -->
        <EpipolarDistanceThreshold>3.0</EpipolarDistanceThreshold>
        <!-- Unit: millimeter, optional parameter, default: 1.0. The ray distance is the average 
          ** distance between a reconstructed 3D point and its corresponding 3D rays. The 
          ** threshold defines the maximal average distance a 3D point may have to its rays 
          ** before being ignored for the initialization. -->
        <RayDistanceThreshold>2.5</RayDistanceThreshold>
        <!-- Unit: millimeter, optional parameter, default: 3.0. The distance threshold is the 
          ** value at which a reconstructed 3D segment is matched to any 3D segment within all 
          ** configured markers. These matches are our initial hyposthesis when trying to detect 
          ** bodies during the intialization phase. -->
        <DistanceThreshold>2.0</DistanceThreshold>
        <!-- Unit: degrees, optional parameter, default: 1.0. The angle threshold is the value 
          ** at which a candidate segment is assumed to be matching a model segment during the 
          ** phase of initialization. It is used to verify hyposthesis established based on the
          ** distance threshold. -->
        <AngleThreshold>1.0</AngleThreshold>
        <!-- Unit: pixel, optional parameters. The maximum reprojection errors are used as a
          ** measure to determine wether an estimated pose is correct or not. All poses that
          ** are estimated by the system must be yielding reprojections errors lower than
          ** those defined in the following parameters before being sent to clients. -->
        <MaxReprojectionErrors>
            <Initialization>5.0</Initialization> <!-- Default: 2.0. The threshold used during the initialization state. -->
            <Tracking>5.0</Tracking> <!-- Default: 2.0. The threshold used during the tracking state. -->
        </MaxReprojectionErrors>    
        <!-- This section describes characteristics of blobs that must be met in order to be
          ** processed in the system. -->
        <BlobCharacteristics>
            <MinSize>3</MinSize>
            <MaxSize>100</MaxSize>
            <MinCompactness>0.0</MinCompactness>
            <MaxCompactness>1.8</MaxCompactness>
            <MinIntensity>1.0</MinIntensity>
            <MinArea>0.0</MinArea>
        </BlobCharacteristics>    
        <!-- The binarization threshold is the grayscale value at which pixels within the image
          ** are marked to be investigated during the BLOB extraction. -->
        <BinarizationThreshold>10</BinarizationThreshold>
    </TrackingParameters>    
</IRMultiViewConfiguration>

Using metaio IR tracking configuration files

To activate metaio IR tracking within the Unifeye SDK you simply have to load an according tracking configuration file and use the API function setTrackingData() (see http://doxygen.metaio.com/UnifeyeSDK/UnifeyeSDK_doxygen/group__Configuration.html). You can also load the according tracking configuration file within the Unifeye GUI using the Tracking Configuration Tool.

→Return to Tracking Configuration

-- SupportMetaio - 2011-01-21