Example 2: Advanced Content

This example demonstrates how you can implement a simple AR application using some more advanced content.

It includes tracking, loading models, using animations and user interaction.

Contents

Requirements

• Downloaded MobileSDK from www.metaio.com/software/mobile-sdk/.
• A running development environment as described in the Getting Started Guide.
• A device running Anroid 2.2.1 or later. You cannot use the emulator because there is no camera image available.
• Recommended: A print-out of the MetaioMan tracking pattern. You can also display the image on your PC screen and track that.

Package Overview

MobileSDK Package

The following illustration shows a brief overview of the packages and classes used in the example application.

The simple package

Here we focus on the GPSLocationBasedActivity class which you can find inside the simple package. The following graph shows a brief overview of what is inside the simple package.

Setting up advanced content

Advanced content may require some extra work to get started. We will have a look at a few common cases here.

Animations

Animations can be started on geometry objects. Geometry objects are instances of the IUnifeyeGeometry class. The animations need to be defined by using a 3D-editor like Blender, 3Ds Max or Maya. For more details about content creation, please refer to the corresponding content tutorials.

For using animations in the built-in renderer every animation needs to have a unique name as identifier. Using this name the respective animation can be triggered.

In this example we like to make the MetaioMan come to life by triggering his animations. He should be in a looped idle-animation on start and react to user interaction with further animations. The MeatioMan has seven animations included: idle, close_down, close_up, close_idle, shock_down, shock_up and shock_idle. The idle animation is our default one so we start it when the model gets visible.

mMetaioMan = loadGeometry("metaioman.md2");
mMetaioMan.startAnimation("idle", false);

Note the second paramter of startAnimation: It tells the MobileSDK if this animation should be looped or not. If you have only one animation, using true here is OK. Otherwise you may want to start a different animation right after the current animation is finished. That is what we want to do later so we choose false.

User interaction

User interaction means, in this context, that the 3D world shall react on touch events on the screen of the device. To realize that we simply need to overwrite the onGeometryTouched() method. The method gets automatically called when the user touches the screen and hits a geometry.

For touch events two common use cases are shown in this example:

• Controling animations and
• triggering movies

Controlling animations

Our goal is to animate the MetaioMan when he is touched via the screen of the device.

The model has two different animation cycles, 'close' and 'shock' (see graph below). For simplicity we let a random number decide which cycle should be started upon a touch. To do so we override the onGeometryTouched() method.

     @Override
     protected void onGeometryTouched(IUnifeyeMobileGeometry geometry) {
         /**
          * Make sure to use 'equals' here, because the pointer to the geometry coming
          * from the MobileSDK might be 
          */
         if ( !mIsAnimationRunning && geometry.equals(mMetaioMan)) {
             Logger.log("UnifeyeCallbackHandler.onGeometryTouched: " + geometry);
             if ( Math.random()< .5 )
             {
                 mMetaioMan.startAnimation("close_down", false);
             }
             else{
                 mMetaioMan.startAnimation("shock_down", false);
             }
             mIsAnimationRunning = true;
         }
         else if ( geometry.equals(mMoviePlayButtonPlane))
         {
             mMoviePlane.playMovieTexture();
             mMoviePlane.setVisible(true);
             mMoviePlayButtonPlane.setVisible(false);
         }
     }

Note the following here:

• The equals() method must be used here instead of the ==-operator because the member mMetaioMan is not visible for the MobileSDK directly.
• The method onGeometryTouched() overwrites the method in ARViewActivity. The ARViewActicty itself implements a callback interface of the MobileSDK and registers itself to receive callbacks.

Until now we have only started an animation. Now we like start the respective following animations when the currently running animation is finished.

To get notifications about the end of an animation we need to implement the IUnifeyeMobileCallback class. It is an abstract class in the MobileSDK. As it is not an interface we cannot use implement with our main activity (PatternBasedTrackingActivity). But we can create a nested class that extends the abstract class:

private final class MobileSDKCallbackHandler extends IUnifeyeMobileCallback {

        public void onAnimationEnd(IUnifeyeMobileGeometry geometry,
                String animationName) {
            Logger.log("MobileSDKCallbackHandler.onAnimationEnd: "
                    + animationName);

            String nextAnimationName = "idle";
            if ( animationName == "close_down")
            {
                nextAnimationName = "close_up";
            } else if ( animationName == "close_up")
            {
                nextAnimationName = "close_idle";
            } else if ( animationName == "shock_down")
            {
                nextAnimationName = "shock_up";
            } else if ( animationName == "shock_up" )
            {
                nextAnimationName = "shock_idle";
            } else
            {
                mIsAnimationRunning = false;
            }
            geometry.startAnimation(nextAnimationName, false);
            
        }

    }

To use this class an instance of it has to be created and passed to the MobileSDK instance in the onCreate() method.

private MobileSDKCallbackHandler mMobileSDKCallbackHandler;

@Override
public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        mMobileSDKCallbackHandler = new MobileSDKCallbackHandler();
 }

Movie textures

If you looked carefully at the onGeometryTouched() method above you might have seen a bit of the movie-handling already.

Movie textures have to be set manually using setMovieTexture() with the according movie:

     /*
      * Create a plane. Use one of our special movie planes here. 
      */
     mMoviePlane = loadGeometry("movieplane16_9.md2");
     /*
      * Movie textures may be set explicit.
      * mMoviePlane is an instance of IUnifeyeMobileGeometry.
      */
     mMoviePlane.setMovieTexture(getAssetPath("demo_movie.3g2"), true, true);

The playback of the movie can be controlled using the following set of methods of the geometry object:

• mMoviePlane. playMovieTexture();
• mMoviePlane. pauseMovieTexture();
• mMoviePlane. stopMovieTexture();

Basically you can use any geometry with movie textures. For convenience we recommend using one of our predefined planes. These are packaged with the example application. You can find them in the assets/movieplanes-folder.

Important notes

Movie textures require certain texture coordinates to be displayed correctly. This is different compared to normal textures. As you can see in this example, normal textures are scaled to fit their targets best. But for performance reasons this step is omitted for movie textures.

You can read more about this topic in our Juniao guide about Sounds, Movies and Images.

Environment mapping

Environment Mapping, also known as "Refelection Mapping" is a useful way to make models in your application more appealing.

Instead of having a simple uniform colored global illumination the model is put into a large virtual textured cube.

Example of an Environment Map
negativ_x	negativ_y	negativ_z	positive_x	positive_y	positive_z

In this example we use a environment map with a sky-look for the truck model. The map consits of six images which have to have specific names (see table above). We recommend putting those files into a subfolder of the projects assets folder. In this case we called the folder "env_map". Loading the map works now by calling loadEnvironmentMap on the MobileSDK instance:

mMobileSDK.loadEnvironmentMap( getAssetPath("env_map" ) );

How much the environment map reflects on the model is dependet on its refelection map, which has the name <modelname>_reflective.<extension>.

For further details please refer to the EnvironmentMapping-tutorial in this Wiki.

-- SupportMetaio - 2011-11-30