Installation and development process

→Return to Installation Guide

This section describes the setup process of your development environment (Adobe Shockwave) for working with the Unifeye Viewer Xtra. It also contains deployment considerations and a HowTo for creating your first Shockwave application using the Unifeye Viewer Xtra. Please note that you can also use the provided ShockwaveARFrame application if you want to start without programming. This application is configurable in terms of models, sounds, animations, tracking information and so on and can be deployed directly to your web server. For more details please refer to section 9.

Prerequisites

If you want to have full control over the application logic we recommend the development of a customized application from the authoring/development environment for Shockwave is Adobe Director 11.5. Please go to http://www.adobe.com/de/products/director/ for details and purchasing information. Section 3.1 also contains some further URLs where you can find an introduction to Director and Shockwave.

You can also start without programming by using the already compiled/published Shockwave movie from the sample application ShockwaveARFrame which is delivered within the Unifeye Viewer package (see 6.2.2). For simple applications this might be sufficient. The application uses an XML based configuration file called ApplicationConfiguration.xml. With this configuration file you can exchange or add 3d models, trigger the play back of animations of 3d models or play back sounds while a 3d model is shown to the user. You will not need a license of Adobe Director 11.5 when using the published ShockwaveARFrame movie. A simple text editor to edit the configuration file is enough.

Additionally you will have to prepare content for your application. In order to create 3D data, you will need an adequate 3D modelling tool like 3D Studio MAX or Maya which can export 3D scenes to the Shockwave 3D file format (.w3d). For more details on the content preparation process please refer to section 9.2 and section 10.

Installation of the Xtra for development purpose

For using the metaio Unifeye Viewer Xtra within Director, copy the appropriate subfolder Windows or Mac from the UnifeyeViewer _Xtra_Develop folder (see section 6.2.3) to the Adobe Director Xtra folder located at <DirectorInstallationFolder>/Configuration/Xtras/.

If you want to develop cross platform Shockwave projects consider the following:

1. Developing on Windows (for Windows and Mac OS):
   Please copy the Mac version of the development Xtra file ( UnifeyeViewer_Xtra_Develop\Mac\metaioUnifeyeViewer.xtra\Contents\MacOS\metaioUnifeyeViewer) to the directory: <DirectorInstallationFolder>\Configuration\Cross Platform Resources\Macintosh\Xtras\ and rename the file extension to “.cpio”. After that you should have a file named:
   <DirectorInstallationFolder>\Configuration\Cross Platform Resources\Macintosh\Xtras\metaioUnifeyeViewer.cpio
2. Developing on Mac OS (for Mac OS and Windows):
   Please copy the Windows version of the development Xtra file ( UnifeyeViewer_Xtra_Develop/Windows/metaioUnifeyeViewer.x32) to the directory:
   <DirectorInstallationFolder>/Configuration/Cross Platform Resources/Windows/Xtras/
   After that you should have a file named: <DirectorInstallationFolder>/Configuration/Cross Platform Resources/Windows/Xtras/metaioUnifeyeViewer.x32

This ensures that the Director Shockwave application you create will download the metaio Unifeye Viewer Xtra correctly on Mac when you develop on Windows and vice versa.

As next step adjust the Xtra configuration file: <DirectorInstallationFolder>/Configuration/xtrainfo.txt and add the line:

[#namePPC:"metaioUnifeyeViewer", #nameW32:"metaioUnifeyeViewer.x32", #package:"http://www.ar-live.de/download/metaioUnifeyeViewer/Xtra/metaioUnifeyeViewer", #info:"http://www.metaio.com"]

Your development configuration is now finished.

Start a new Director project. From the menu choose: “Modify -> Film -> Xtras”. Click on the “Add” button and select the metaioUnifeyeViewer Xtra. If you want to download the Xtra to the client’s computer in the deployment phase (recommended), select the Xtra and uncheck and check again the checkbox “Download if needed”.

Please note that when you run your Director project within Director, the Xtra from within the <DirectorInstallationFolder>/Configuration/Xtras/ folder will be used. If you publish your Director project to the web browser, the Xtra will be downloaded (if you selected so, see above) and put into the location:

1. Windows:
   %APPDATA%\Adobe\Shockwave Player 11\xtras\download\metaio GmbH\UnifeyeViewer\
   where %APPDATA% maps on Windows XP to:
   to C:\Documents and Settings\<USERNAME>\Application Data\,
   and on Vista and higher to:
   C:\Users\<USERNAME>\AppData\LocalLow
2. Mac:
   ~/Library/Application Support/Adobe/Shockwave Player 11/Xtras/download/metaio GmbH/UnifeyeViewer/

and then used from that location by the web browser. This can lead to confusion during the development phase as the downloaded Xtra will be the deployment Xtra and not the development Xtra. We recommend overwriting the downloaded Xtra file with the development Xtra file after the first download during the development phase. By doing this you can make sure that you always use the development Xtra no matter of publishing to your web browser or running your application directly within Director. You can do this by copying the Xtra development file from (assuming a Windows environment):

UnifeyeViewer _Xtra_Develop\Windows\metaioUnifeyeViewer.x32
to
%APPDATA%\Adobe\Shockwave Player 11\xtras\download\metaio GmbH\UnifeyeViewer\metaioUnifeyeViewer.x32

For the final testing phase you should of course switch to the deployment Xtra (after you received the encrypted tracking configuration file). This process is covered in more details section 7.4.

System configuration

The tracking system of the Unifeye Viewer Xtra is initialized by a tracking configuration file which is also called “TrackingData” file. It defines the markerless patterns to be tracked by the systems. It is a mandatory file needed by the Xtra. If the Xtra doesn’t initialize properly this is usually due to a missing or malformed tracking configuration file. Also note that the development Xtra only handles unencrypted tracking configuration files in contrast to the deployment version of the Xtra which handles only encrypted tracking configuration files.

Another configuration aspect is setting the camera parameters respectively the camera calibration (also known as the intrinsic parameters of a camera). This includes parameters like focal length or distortion values. If you want to have highly accurate augmentations, you should provide such parameters for each camera you want to use. Using the camera calibration is an optional step. You do not have to provide a camera calibration in order to get the tracking to work. Default values will be used internally if you omit this step. However if you use it, augmentation will be better and usually more stable.

Please note that the Unifeye Viewer Xtra package contains no tools for generating a tracking configuration or camera calibration file for you. You should use the tools provided by Unifeye SDK/Unifeye Design.

Tracking configuration/Tracking values

The tracking configuration file is used by the Unifeye Viewer Xtra to configure the tracking system. The format of this file is the exact same format as in other Unifeye products like Unifeye SDK, Unifeye Design or the Unifeye Mobile SDK. It is passed to the Unifeye Viewer Xtra upon instantiation and is basically an XML file. It configures the tracking approach to be used (currently only planar markerless tracking) and the targets/pattern to be tracked. The Unifeye Viewer package contains an example configuration file. Please note that the development Xtra only processes unencrypted tracking configuration files in contrast to the deployment version of the Xtra.

The Unifeye Viewer function getTrackingDataAndImage() returns the tracking values for every defined coordinates systems (COS) within your tracking configuration file. The tracking values are used to place 3D objects in space. The function returns Transforms that can be used directly within Shockwave to place your 3D objects.
In case of planar markerless tracking, each defined COS is bound to a tracking pattern (i.e. a reference image). You can use arbitrary images which are sufficiently well textured, for tracking purposes. To use the tracking approach the following steps are necessary:

• Select suitable image data,
• create the tracking configuration file,
• apply/load the tracking configuration file into the Unifeye Viewer Xtra.

Input data

Planar markerless tracking is based on unique features identified in the given reference image. To assure good quality tracking, several rules should be taken into account:

• The reference image has to be suitable with respect to its texture and content.
• Depending on how close the reference image is put in front of the camera, the resolution should be chosen accordingly. As a rule of thumb, the resolution of the reference image should be similar as it is seen from a "default position".
  For example: If you want to track a DIN A4 (210mmx297mm) sized print-out and the default resolution of the camera is 640x480 pixel, the reference image should be scaled to 350x495 pixel. Looking at the whole page with the camera, the printed image seen in the current camera view will have approximately the same size as the reference image. The reference image should not be high-resolution unless you want to look very close at the printed image.
• The digital version should have the same aspect ratio than the printed version (e.g. no severe stretching of the image).

Figure 1 Reference image and printed image

Tracking configuration

Given the reference image, the tracking configuration file can be created. To do this, the Planar Markerless Tracking Configuration Tool provided by Unifeye SDK/Design is recommended. But you can also create an according tracking configuration file manually as it is a human readable XML file. An example tracking configuration for planar markerless tracking is shown in Figure 6:

• In the Sensor section a Sensor of type "FeatureBasedSensorSource" is defined with a unique SensorID.
• The “Parameter” section should usually be empty as no specific parameters are needed. There are however some optional parameters:
  • The parameter “qualityThreshold” can be set to 0.75 or 0.7 (or lower) to help the tracking to perform better on not well suited reference images. However reducing it also reduces the tracking quality and stability.
  • “cameraIsCalibrated” can be set to “0” if you experience a bad and unstable tracking performance with cameras when not using a camera calibration (see also 7.3.2). This especially effects the situation when the reference image is facing the camera directly.
• Then a “SensorCOS” is added, configuring an image pattern to be used as reference.
• A „SensorCOS“ consists of:
  • a unique „SensorCosID“,
  • parameters specific for this “SensorCOS”: for the planar markerless pattern this is the reference image with its width and height in millimeters (and optional tracking patches). It can be an absolute or relative path (to the tracking configuration file itself) for offline scenarios. If the tracking configuration file is located on a web server, the path to the reference image also has to be given as an absolute URL.
• Then the “Connections” are defined, which link a “SensorID” with a “SensorCosID” in terms of “COSes”.
• In this case one “COS” is defined which
  • has a name (“Cos1”),
  • uses a “Fuser” of type “BestQualityFuser”,
  • links the “SensorID” “FeatureTracking1” with the “SensorCosID” “Patch_Roivis”,
  • holds a “HandEyeCalibration” transform which is the identity (no translation and no rotation),
  • holds a “COSOffset” transform which is also the identity.

Please note that you usually will define one “SensorCOS” for every tracking pattern/reference image you want to track and one according “COS” in the “Connections” section that links to this “SensorCOS” (see Figure 7).

As a “Fuser” you will usually always use “BestQualityFuser” with “HandEyeCalibration” and “COSOffset” being the identity.

Figure 2 Tracking configuration file for planar markerless tracking with one reference image

Figure 3 Tracking configuration file with two reference images

Camera setup and calibration

The Unifeye Viewer Xtra requires an image/frame source for tracking. The image source is usually a webcam. Webcams and cameras in general have several intrinsic parameters that vary due to its obstructed lenses and electronics. Therefore the given image can vary amongst different cameras. If you don’t adapt your augmentation according the intrinsic properties of the camera, it might appear incorrect. In order to neutralize the intrinsic properties, you have to use the Unifeye Viewer Xtra method “setCameraParameters()” and set the camera property of Shockwave 3D called “fov” (field-of-view).

The camera parameters passed to “setCameraParameters()” can vary for different cameras and need to be measured. You can measure the intrinsic parameters of the camera by using tools provided by Unifeye SDK or Unifeye Design (e.g the Sextant tool). You do not necessarily need to call “setCameraParameters()” for an augmentation. The Unifeye Viewer Xtra will automatically be initialized with parameters that work well for most cameras. If you do use “setCameraParameters()” please note that when you specify undistortion values different from 0, an undistortion of the image will occur. This task consumes some processing power and can be omitted for usual webcam scenarios by specifying 0 as undistortion values.

Besides calling “setCameraParameters()”, it is mandatory to setup the virtual 3D camera of the Shockwave 3D renderer. You should always reset it’s position and rotation to (0,0,0) and setup its field-of-view. The field-of-view can be calculated from the width and height of the output image of the currently active webcam. Please refer to the example applications for details regarding this process.

From Development to Deployment

The following and the illustration Figure 8 outlines the process from the development to the deployment phase.

1. Plan and develop your AR Shockwave application with the Unifeye Viewer development Xtra. This includes the creation of a tracking configuration file, writing the application logic, preparing the content and testing it within Director and potentially on an internal web server. The Unifeye Viewer Xtra ships with two example applications you can use as reference and templates (for more information about the sample applications, see sections 7.5 and 8).
2. If your application development is done, send the tracking configuration file to metaio. You will receive an encrypted tracking configuration file in return to be used with the deployment version of the Unifeye Viewer Xtra. In order to install the deployment version of the Xtra on your system, you have to publish to the web (see below) and run your movie once in the web browser. If everything is set up correctly the Shockwave plugin will ask you to download the Unifeye Viewer Xtra automatically to a specific location. If you already did this the download process might not be triggered. In these cases, please manually delete the Unifeye Viewer Xtra from the download location of your web browser. Delete the UnifeyeViewer folder (including the content) from the following locations (see also 7.2):
3. 1. Windows:
      %APPDATA%\Adobe\Shockwave Player 11\xtras\download\metaio GmbH\UnifeyeViewer\
      where %APPDATA% maps on Windows XP to:
      to C:\Documents and Settings\<USERNAME>\Application Data\,
      and on Vista to:
      C:\Users\<USERNAME>\AppData\LocalLow
   2. Mac:
      ~/Library/Application Support/Adobe/Shockwave Player 11/Xtras/download/metaio GmbH/UnifeyeViewer/
4. Publish your application for the web and the usage of the deployment version of the Xtra. Do not forget to adjust your Lingo code to use the encrypted tracking configuration file now.
   In the menu bar under “File -> Publication settings” make sure that “HTML” and “Shockwave-File (DCR)” is checked. Finally click on “File -> Publish”. This will generate the Shockwave movie (the compiled Shockwave application) for you.
   If you have adapted the xtrainfo.txt file of Director before (see section 7.2), everything should be fine and the published movie should reference the deployment Xtra automatically. Double click the generated HTML file. If the Unifeye Viewer Xtra wasn’t installed on your system before or the installed Xtra is an older version, the newest version of the deployment Xtra should be downloaded and installed on your system automatically (see also the previous point). Just agree the dialog when you are asked if you want to install the Unifeye Viewer Xtra on your system.
   IMPORTANT NOTE:
   Published (.dcr) Shockwave movies cannot reference/load local files of your hard disk (due to security restrictions). Therefore your published application might not work correctly. In order to get your application to work, simply upload the HTML page and its referenced content on a web server and call it again from there. You may also consult the Adobe Director help file for more details on that.
5. Create your own HTML page and embed the previously generated Shockwave movie file (.dcr). Have a look at the HTML page that was published along with the Shockwave .dcr file to see how it is done.
6. Create a so called “pre-movie” with Director. The “pre-movie” will act as entrance page for the actual Shockwave application you created. It only references the Xtra (pretending to need it without using it) in order to trigger the download process. This is necessary due to a general issue with Xtras in Shockwave and recommended by Adobe. The ShockwaveARFrame contains an example of such a “pre-movie”.
7. Put the “pre-movie”, the HTML page with the embedded Shockwave application along with all necessary content online and see if it works.

Figure 4 During development stage the development Xtra is used. You are responsible for creating the Pre-Movie, the Shockwave application and the tracking configuration. When you are done, use the encrypted tracking configuration file provided by metaio. After you published your movie and Pre-Movie successfully, embed it into an HTML page. Upload everything to your server. End users/clients can now launch your HTML page including an AR enabled Shockwave application.

Figure 5 Physical location after deployment

Notes:

• The “pre-movie” is a standalone Shockwave movie/application that forwards the user to the real Shockwave AR application. It downloads all required Xtras in advance if necessary. A “pre-movie” is recommended because, according to Adobe’s documentation, a Shockwave movie can have problems using an Xtra directly after downloading it. According to Adobe, using a “pre-movie” is the official way for using Xtras in online scenarios. The ShockwaveARFrame example application uses a “pre-movie”. Open PreMovie.dir for more details.
• Before you launch your application, we recommend testing it on different browsers.
• If you are developing online applications you should try to keep the file size of your content and assets small (as they have to be downloaded). Especially 3D models should not contain more than 100000 triangles as on weak computers the rendering performance and thus the overall system performance may drop.

→Return to Installation Guide

-- SupportMetaio - 2010-07-14