General Tracking Framework and Configuration

→Return to System Configuration

Tracking framework

From version 3.0 on, the Unifeye tracking module/framework was restructured to provide a more flexible and open configuration of different tracking systems. In addition the tracking system configuration has been changed, to separate sensor/tracking specific information from general world properties such as hand eye calibration and offset values. The tracking framework is configured by so called tracking configuration or "Tracking Data" files, which are human readable XML files that can be loaded into the Unifeye SDK by using the function setTrackingData (see the according section in the API documentation).
The tracking configuration files contain tracking/sensor specific paremeters as well as general parameters. Information about the configuration for the specifc, supported tracking systems can be found in the tracking configuration section. This section gives some general background information and general configuration parameters.

General structure

The general layout of a tracking configuration XML file looks like this:

<?xml version="1.0"?>
<TrackingData>

<!--Upper part-->
<Sensors>
    <!--Definition of sensors and their sensor coordinate systems -->
    <Sensor type="...">
        <SensorID>ID_OF_SENSOR</SensorID>
        <Parameters>
            <!--Sensor specific parameter section-->
        </Parameters>
        <SensorCOS>
            <!--Common sensor coordinate system parameters-->
            <SensorCosID>ID_OF_SENSOR_COS</SensorCosID>
            <Parameters>
                <!--Sensor coordinate system specific parameters (for this sensor)-->
            </Parameters>
        </SensorCOS>
        <!--Further sensor coordinate systems for this sensor-->
        ...
        
    </Sensor>
    <!--Further sensors -->
    ...
</Sensors>

<!--Lower part-->
<Connections>
    <!--Definition of coordinate systems (i.e. linking sensors and sensor coordinate systems 
        from above to actual coordinate systems -->
    <COS>
        <!--Common coordinate system parameters-->
        <Name>OPTIONAL_NAME_OF_COS</Name>
        <Fuser type="...">
            <!--Fuser specific parameters-->
            <Parameters>
            </Parameters>
        </Fuser>
        
        <!--We can link 1 to n sensor coordinate systems (even from different sensors
            to one coordinate system.-->
        <SensorSource trigger="1">
            <SensorID>FROM_THE_UPPER_PART</SensorID>    
            <SensorCosID>FROM_THE_UPPER_PART</SensorCosID>        
            <!--Optional Hand-Eye calibration values-->
            <HandEyeCalibration>
                <TranslationOffset>
                    <x>0.0</x>
                    <y>0.0</y>
                    <z>0.0</z>
                </TranslationOffset>
                <RotationOffset>
                    <x>0.0</x>
                    <y>0.0</y>
                    <z>0.0</z>
                    <w>1.0</w>
                </RotationOffset> 
            </HandEyeCalibration>
            <!--Optional static offset (e.g. if two markers define one COS, their relation
                to each other goes in here-->
            <COSOffset>
                <TranslationOffset>
                    <x>0.0</x>
                    <y>0.0</y>
                    <z>0.0</z>
                </TranslationOffset>
                <RotationOffset>
                    <x>0.0</x>
                    <y>0.0</y>
                    <z>0.0</z>
                    <w>1.0</w>
                </RotationOffset>
            </COSOffset>
        </SensorSource>
        <!--Further sensor coordinate systems that define this actual coordinate system-->
        ...
        
    </COS>
    <!--Further coordinate systems-->
    ...
    
</Connections>

</TrackingData>

The tracking configuration XML files have two major sections:

1. The upper section (within the <Sensors> tag) configures sensor/tracking specific parameters (every tracking has specific parameters. Details about those specific parameters can are available in the tracking configuration section).
   In general every sensor can track one or several so called sensor coordinate systems ( SensorCOS) which are specific for the sensor. E.g. a mechanical tracking arm like the FARO arm can only track one coordinate system (the arm itself), whereas a marker tracking sensor can track as many (distinct) markers (and thus sensor coordinate systems) at the same time as you configure. In addition to that you are not limited to only one tracking system but can configure to use several tracking systems/sensors at the same time and e.g. "fuse" the results.
2. The lower part of the tracking configuration file (i.e. the part within the <Connections> section of the XML file) configures general parameters and connects the outputs from one or several tracking systems/sensors to actual coordinate systems ( COS) which are used for binding 3D model data to it. Throughout the Unifeye SDK API, most of the time you will deal with those coordinate systems (COSes), for example for binding 3D models to it, for getting tracking values etc. Those coordinate systems are simply addressed by the numerical number corresponding to the appearance within the XML file. So the first coordinate system has the COS ID 1, the second the COS ID 2 etc.

Common parameters for all sensor coordinate systems

As mentioned above every tracking system/sensor can have specific parameters. But there are also parameters within the <Sensors/Sensor/SensorCOS> section that are available for all sensor coordinate systems (although they might not make sense for all tracking systems):

• <SensorCosID>SENSOR_ID</SensorCosID> identifies the name of the sensor coordinate system and has to be unique within the Sensor where it is defined. For defining actual coordinate systems within the <Connections> section of the tracking configuration file a sensor cos will be addressed by the combination of SensorID and SensorCosID.
• <SensorCOSDirection>DIRECTION</SensorCOSDirection>, either: WorldToTarget (default) or TargetToWorld. Usually all tracking values are returned from the camera to the tracked object, so WorldToTarget if we define that the origin of the world is within the camera (the actual tracking device). The below image illustrates this fact using normal marker tracking.
  
  
  So the tracking values of the marker will always be relative to the tracking camera with the direction: tracking camera ("World") To marker ("Target"). This holds true for all tracking systems so e.g. also in case of a mechanical tracking like FARO arm tracking the tracking values will be from the origin of the world/the tracking device (in this case the arm itself) to the probe (the tracked item).
  However by specifying TargetToWorld the direction will be inverted. Usually this is done when you use an outside in tracking like e.g. infrared tracking ART tracking. In such cases you usually mount an IR target to a camera used for recording the scene. The IR target (and thus the camera) itself is tracked by the infrared tracking. In addition you even usually define a static offset from the IR tracking cameras to some point into the real world which you define as your origin:
  
  So the above image illustrates the final outcome: "Final Tracking Values" is what you need: tracking values from the (visualization) camera to the world. To achieve this a couple of things are necessary but most important for the IR tracking TargetToWorld has to be defined as otherwise, per our definition, the arrow between the IR tracking camera and the IR target would point into the other direction (from the tracking device (ie IR camera) to the tracked item (ie IR tracking target)).
  The other elements are described in the according sections: Hand Eye Calibration and coordinate system offset (COSOffset) in the next section.
• <ReferenceCOSID>REFERENCE_COS</ReferenceCOSID> can be used in special cases where you do NOT want to get the tracking values from the world/tracking device to the tracked item/target but relative to another tracked item within the same tracking system/sensor. So if you e.g. specify a marker tracking with three markers and define for the first marker:
  

      <SensorCOS>
           <SensorCosID>Marker1</SensorCosID>
           <Parameters>
              <MarkerParameters>
                 <Size>140</Size>
                 <MatrixID>1</MatrixID>
              </MarkerParameters>
           </Parameters>
           <!-- Marker 1 will now deliver the tracking information relative to Marker2 (from Marker2 to Marker1)-->
           <ReferenceCOSID>Marker2</ReferenceCOSID>
        </SensorCOS>

  the tracking values for marker 1 will not be given from the tracking camera to the marker 1 but from marker 2 to marker 1 (as shown below).
  
  
  The values will be updated accordingly if you e.g. move around marker 1.

Tracking offsets/ parameters

The properties which are general for all tracking systems/coordinate systems (see the <Connections> part in the tracking configuration file) are as follows:

• an optional Hand Eye Calibration offset which is applied before the actual tracking information from a sensor coordinate system is used (i.e. "multiplied from the left"),
• an optional coordinate system offset which is applied after the tracking information from a sensor coordinate system is used (i.e. "multiplied from the right"),
• a Fuser type (see the Fuser section below)
• an optional name for the coordinate system. Please note (as mentionend above) that you will be using the numerical COS ID (which is the numerical number corresponding to the appearance within the XML file) usually throughout the Unifeye SDK API when dealing with coordinate systems.

The overall tracking chain is thus: "Hand Eye Calibration offset" x "Tracking pose from sensor coordinate system" x "coordinate system offset". The different offsets are given as translation in 3D and a rotation quaternion.

Fusers

The Fuser allows to combine the tracking information of several SensorCOSes to one COS or to perform specific actions on the result of a SensorCOS. Depending on the choice of fuser the combination is done based on different rules and with different outcomes. In a standard scenario you will usually have one SensorCOS per COS like one Marker (one marker SensorCOS) defining one coordinate system ( COS). However, as just mentioned, you can combine several SensorCOSes (even from different sensors/tracking systems) to set up one coordinate systems. In such cases, usually the offsets between the SensorCOSes are fixed and their relations measured beforehand and put as coordinate system offset into the tracking configuration file (see Tracking offsets/parameters and Multiple SensorCOSes building one coordinate system).
The following fusers are currently available:

• BestQualityFuser: The best quality fuser chooses the tracking result from the SensorCOS with the best quality and provides it as output for the COS. It does not require any parameters and is the default fuser for tracking configurations in Unifeye.
  

  <Fuser type="BestQualityFuser">
      <Parameters></Parameters>
  </Fuser>

• AverageFuser: The average fuser calculates the average of the given SensorCOS information. It can be applied in two ways.
  • Either the fuser is used to fuse the results of two or more SensorCOSes. In the default case, all SensorCOSes need to deliver valid pose information for fusion. To track also with less valid poses, the according minimum number of valid SensorCOS poses needs to be specified through the configuration.
  • Or the average fuser is used to fuse the history of poses within one SensorCOS. Then the configuration needs to specify the <SingleCOSHistoryFusion> flag and indicate the size of the history for fusion ( <SingleCOSHistoryFusionNumber>). This value has to be a power of two for valid results (e.g. 2, 4, 8, 16).
    

    <Fuser type="AverageFuser"> <!-- Average Fuser for fusion over several SensorCoses -->
        <Parameters>
            <MinNumberOfAvailableCoses>1<MinNumberOfAvailableCoses> <!-- Minimum number of SensorCoses 
                that need to be available for fusion. If not specified, all SensorCoses need to return valid data for fusion. -->
        </Parameters>
    </Fuser> 
    
    <Fuser type="AverageFuser"> <!-- Average Fuser for fusion over history of one SensorCos -->
        <Parameters>
            <SingleCOSHistoryFusion>1<SingleCOSHistoryFusion> <!-- Flag to switch on the single SensorCos fusion -->
            <SingleCOSHistoryFusionNumber>4<SingleCOSHistoryFusionNumber> <!-- History size (power of 2)-->
        </Parameters>
    </Fuser>

• InvertFuser: The invert fuser inverts the first valid pose information found in the list of SensorCOSes. The pose returned by the tracked SensorCOSs is inverted and forwarded as tracking result for the COS. The invert fuser does not require any parameters.

  <Fuser type="InvertFuser">
      <Parameters></Parameters>
  </Fuser>

• DESPFuser: The Double Exponential Smoothing and Prediction fuser is not really a fuser but rather a filter. Thus it can only be used for one SensorCOS at a time. It smoothes the pose/tracking values of the according SensorCOS over a pose history for rotation and translation and performs (limited) prediction into the future. The filter can also cut-off the rotation and/or translation values at certain limits (optional). Please refer to the comments in the snippet below for more details.

      <Fuser type="DESPFuser">
         <Parameters>
            <!-- Alpha:
               Is a value between 0 and 1 and determines how much influence old values
               have for the calculated pose. The bigger Alpha the less influence old values
               have on the current pose. 
            -->
            <Alpha>0.2</Alpha>
            <!-- Gamma:
               Is a value between 0 and 1 and determines how much influence 
               old values have for the predicted pose. The bigger Gamma the less influence old values
               have for the predicted pose.
            -->
            <Gamma>0.2</Gamma>
            <!-- NumberOfPredictions:
               Determines the number of predictions.
            -->
            <NumberOfPredictions>10</NumberOfPredictions>
            <!-- FuseTranslation:
               If translation values are to be fused.
            -->
            <FuseTranslation>1</FuseTranslation>
            <!-- FuseRotation:
               If rotation values are to be fused.
            -->
            <FuseRotation>1</FuseRotation>
            <Constraints>
               <!-- Rotation: 
                  You can limit the rotation to not exceed the values given.
                  Euler angles in DEGREES [-180..+180] 
               -->
               <RotationX min="-1" max="1" />
               <RotationY min="-5" max="5" />
               <RotationZ min="-25" max="25" />
               <!-- Translation: 
                  You can limit the translation to not exceed the values given.
                  distance in millimeters [-inf..+inf] 
               -->
               <TranslationX min="-300" max="300" />
               <TranslationY min="80" max="-80" />
               <TranslationZ min="-300" max="-200" />
            </Constraints>
         </Parameters>
      </Fuser>

Default tracking configuration

When the Unifeye SDK system is launched, the default tracking configuration file <UnifeyeInstallationDirectory>/config/AS_Tracking_Data.xml is loaded. This file configures a marker tracking with three markers and three coordinate systems. The XML content is shown below.

<?xml version="1.0"?>
<TrackingData>
    <Sensors>    
        <Sensor type="MarkerBasedSensorSource">
            <SensorID>Markertracking1</SensorID>
            <Parameters>
                <MarkerTrackingParameters>
                    <trackingQuality>robust</trackingQuality>
                    <thresholdOffset>110</thresholdOffset>
                    <numberOfSearchIterations>1</numberOfSearchIterations>
                </MarkerTrackingParameters>
            </Parameters>
            <SensorCOS>
                <SensorCosID>Marker1</SensorCosID>
                <Parameters><MarkerParameters>
                        <Size>140</Size>
                        <MatrixID>1</MatrixID>
                        </MarkerParameters>
                </Parameters>
            </SensorCOS>
            <SensorCOS>
                <SensorCosID>Marker2</SensorCosID>
                <Parameters><MarkerParameters>
                        <Size>140</Size>
                        <MatrixID>2</MatrixID>
                        </MarkerParameters>
                </Parameters>
            </SensorCOS>    
            <SensorCOS>
                <SensorCosID>Marker3</SensorCosID>
                <Parameters><MarkerParameters>
                        <Size>140</Size>
                        <MatrixID>3</MatrixID>
                        </MarkerParameters>
                </Parameters>
            </SensorCOS>    
        </Sensor>    
    </Sensors>
    <Connections>
        <COS>
            <Name>Cos1</Name>
            <Fuser type="BestQualityFuser">
                <Parameters></Parameters>
            </Fuser>
            <!-- There are 1-n SensorCoses -->
            <SensorSource trigger="1">
                <SensorID>Markertracking1</SensorID>    <!-- Who tracks... -->
                <SensorCosID>Marker1</SensorCosID>        <!-- what? -->
                <HandEyeCalibration>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset> <!-- e.g. Transform between camera and ART body -->
                </HandEyeCalibration>
                <COSOffset>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset><!-- offset  -->
                </COSOffset>
            </SensorSource>
        </COS>
        <COS>
            <Name>Cos2</Name>
            <Fuser type="BestQualityFuser">
                <Parameters></Parameters>
            </Fuser>
            <SensorSource trigger="1">
                <SensorID>Markertracking1</SensorID>
                <SensorCosID>Marker2</SensorCosID>
                <HandEyeCalibration>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset>
                </HandEyeCalibration>
                <COSOffset>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset>
                </COSOffset>
            </SensorSource>
        </COS>
        <COS>
            <Name>Cos3</Name>
            <Fuser type="BestQualityFuser">
                <Parameters></Parameters>
            </Fuser>
            <SensorSource trigger="1">
                <SensorID>Markertracking1</SensorID>
                <SensorCosID>Marker3</SensorCosID>
                <HandEyeCalibration>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset>
                </HandEyeCalibration>
                <COSOffset>
                    <TranslationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                    </TranslationOffset>
                    <RotationOffset>
                        <x>0.0</x>
                        <y>0.0</y>
                        <z>0.0</z>
                        <w>1.0</w>
                    </RotationOffset>
                </COSOffset>
            </SensorSource>
        </COS>
    </Connections>
</TrackingData>

Multiple SensorCOSes building one coordinate system

fda

todo: multiple COS setup, DESP fuser, Rel.Cos, invert per sensorCOSID level (up).

→Return to System Configuration

-- SupportMetaio - 2011-01-21