Step 1

CREATE A NEW PROJECT IN PFTRACK

To begin, go to File => New Shot or click on the toolbar to create a new project file.

Step 2

IMPORT FOOTAGE

Next, go to File => Import Footage or click on the toolbar to load the footage.

Step 3

EXTRACT CAMERA INFO FROM LIVE FOOTAGE

In order to extract camera information from the footage accurately, it’s necessary to define two types of movement in the scene: camera movement and object movement. In simpler scenes, typically only camera movement is needed for analyzing and calculating the 3D camera information. After importing the footage, a new group with the default name Scene appears. It’s solely for camera tracking and calculation and not for anything else. First, draw masks around moving objects in the scene to separate them from the rest of the shot. In this case, the people and the hand that is holding the beer in the foreground are masked out. This ensures the area will be excluded from being analyzed and calculated while retrieving the camera information later on. You can also link the mask to a User Feature that will enable it to track the object automatically. Sometimes, it’s necessary to animate the mask to match the object’s deformation.

Next, go to Camera => Solve Camera. PFTrack will automatically extract camera-moving data by analyzing and tracking a certain number of the points in the footage. Those points being tracked and used will turn green. When the calculation is finished, you can see the camera transformation curve in the Camera F-Curves window.

Step 4

TRACK AND EXTRACT YOUR MOVING OBJECT OUT OF LIVE FOOTAGE

After successfully extracting the camera information from the footage, track and extract the moving object information from the shot, in this case, the beer bottle. However, before you begin tracking, create a new group for the bottle in the scene to separate it from the previous calculation for the camera.

Next, click on the new group and import a CG model of the beer bottle that was created in Maya previously (File => Import Tracking Geometry). Adjust the CG bottle to line up with the bottle in the footage. Then draw a mask around the bottle to define the region that will be tracked, similar to what was done to the camera. Click on Track geometry forwards/backgrounds to start the tracking calculation. When it’s done, the transformation of the beer bottle will be recorded in the Motion F-Curves window.

Now some manual calibration work may be needed. After reviewing playback, you may see that the CG bottle is slightly off or jumpy within some sections of the shot. This can happen when the tracked object is obscured by other foreground elements in the scene. To fix this, go to the erroneous frames and manually adjust the position of the object, then repeat the above tracking process until the CG bottle perfectly lines up with the live action bottle.

Step 5

IMPORT CAMERA AND OBJECTS INTO MAYA

So far, we have retrieved the important information of the camera and the object. Now you need to import them to Maya to recreate the scene. Right click Camera Exports => New and select MAYA ASCII 4.5(*.ma) as file format. Select the new groups you just created ( scene, obj) and click OK. PFTrack will then generate a Maya file containing a new camera and a 3D beer bottle object, along with its animation.

Open the.ma file in Maya. You will see that the Motion F-Curves and Camera F-Curves generated in PFTrack have now turned into Maya’s animation curves. The match-moving process is now complete. Lighting, texture mapping and rendering in Maya come next.

Step 6

LIGHT AND RENDER IN MAYA

Since the CG bottle must seamlessly blend into the live action, it’s necessary to simulate the lighting situation in the real world for each scene. We chose mental ray as the main renderer because of its advanced lighting features, such as Final Gathering and Global Illumination, to simulate the real lighting situation. We also created a unique HDRI environment map for each scene by analyzing the positions of the light sources and lighting directions and, in doing so, recreated much of the real environment in CG, in this case the locker room.

Step 7

CREATE YOUR FINAL COMPOSITE IN SHAKE

The rendered bottle animation sequences are then brought into Shake for final compositing and color correction. Some rotoscoping is necessary at this point, masking the original footage so CG elements can blend into the live-action world.

Color correction is the final step, when you can adjust the overall brightness and contrast, reduce the sharpness from the CG elements, add motion blur, film grains and noises, and apply shadows based on the light direction.

Tools Used: Autodesk Maya; Apple Shake; Pixel Farm PFTrack; mental ray

Your Guide

Jianchao Li

VFX Director, Creative Group, China

Originally from China and a graduate of Savannah College of Art and Design with a masters degree in Computer Art, Jianchao has worked as an animator, art director and VFX supervisor with several animation/post-production companies in the U.S. He joined Creative Group NY in 2004 and in 2006 launched Creative Group Beijing, an animation/VFX division of the company. With his team in Beijing, and in conjunction with Mantra, the design division of Creative Group, in NY, he has been producing 3D, 2D and Flash animation, TV commercials and promos for clients including ESPN, CBS, Saatchi & Saatchi and MTV. His honors include Promax/BDA Gold, Telly Awards Gold and Videographer Awards.

Jianchao Says Keep in Mind…

Our client ESPN asked us to re-purpose a spot it had run during last year’s ESPN Monday Night Football for the same event this year. The sponsor, Miller Lite, had changed its label from last year and our task was to replace all the labels in the spot with the new one. Because many of the scenes were not static – either the camera or the beer bottles were moving – we had a difficult tracking job ahead of us. A different team had first tried to do it with traditional 2D tracking in Flame, but it didn’t work: the bottles were moving not only within the X-Y plane but along the Z-Axis, and rotating in different directions. By analyzing each scene carefully, we decided to use the 3D tracking/compositing method described here for difficult scenes and 2D tracking for simpler scenes.

Creative Group/Mantra Design

www.mantradesign.tv

ph. 646.723.5700