When the visual effects crew for Speed Racer arrived in Berlin four months ahead of production, they put systems into play that would rock filmmaking. With visual effects occupying an increasing amount of screen time in films, tasks formerly thought of as post production have moved further upstream and visual effects artists have taken on a more integral role in production. Speed Racer drives that notion to the extreme. “On Speed Racer, I believe post-production began on the very first day,” says John Gaeta, co-visual effects supervisor.

Directed by Larry and Andy Wachowski, the high-octane kid flick about a fanatical race-car driver and his family powers onto the screen in blazing pop-art colors. Drivers in candy-colored cars compete at breakneck speed on twisting, roller-coaster tracks, winning the races using “car fu” prowess. Actors, filmed on green-screen stages and, 15 percent of the time, on brightly colored sets, perform in an often-surreal world in which the environments heighten emotion and build story points.

To create their homage to the 1960s television cartoon series Speed Racer, which was based on the cult Japanese manga series Mahha gà´ gà´ gà´, the Wachowski brothers turned to Gaeta and co-visual effects supervisor Dan Glass. Gaeta had won an Oscar for visual effects on the brothers’ legendary film, The Matrix. Dan Glass had co-supervised the second two films in the Matrix trilogy with Gaeta and supervised effects for the Wachowskis’ V for Vendetta.

“We had a maximum of 12 weeks for shooting,” says Gaeta. “We had done a proof-of-concept test early in development so that everyone could see how it felt to execute a shoot, but once [we were] in Berlin, the green-screen shots started expanding.”

Gathering Virtual Backgrounds

Beginning in June 2006, teams of artists that would include production designer Owen Paterson, art director Hugh Bateup, previz artist Euisung Lee of Halon, and matte painter Lubo Hristov of Christov Design and others had worked with Gaeta and Glass to previs the 3D cars and tracks and the virtual backgrounds.

Although they knew the racing would be heavily CG to realize the directors’ wish for extreme sports action with stunt-performing cars, the idea of creating virtual backgrounds for the narrative half of the film grew largely from budget concerns. “Filling the Racer family’s day-to-day world with custom-designed sets and props would be too expensive,” Glass says.

Gaeta and Glass had been creating impossible camera moves through virtual environments constructed from photographs for years, but to give this film an anime look, they pushed that idea in new directions. Using high-resolution QuickTime VR-based technology, they worked with 360-degree “bubbles” stitched together from photos that Hristov enhanced with exotic colors. They stacked these bubble spheres one inside another, cut windows using transparency from one layer to another, and shifted the layers. The effect was as if they had used rostrum and multiplane camera techniques on environments created with high-contrast, highly color-saturated, manipulated, and enhanced photo collages. Gaeta dubs the technique “photo anime.”

Then, working with Lee, Gaeta and Glass experimented by putting green-screen footage and 3D characters into these sometimes-animated VR bubble environments, and by moving the virtual camera in interesting ways.

“If you stretch or scale the bubble along the axis of the camera or move the camera slightly offset from the center, you get strange distortions that would be anomalies in live action,” Lee says. “You know something is not quite right but you can’t pinpoint what it is. It was stylistically interesting enough that we started using it more and more. John [Gaeta] wanted to push the potential of it.”

For the proof-of-concept test, Dennis Martin, who had been a virtual cinematographer for The Matrix, shot photographs at a futuristic 1960’s Lautner house with expansive windows in Los Angeles. Then, he stitched the photos into bubbles that Hristov enhanced.

“We tested the image quality and methods of compositing,” Gaeta says. “We started doing these previsualizations ahead of the HD shoot to explore how far we could push camera moves and composition inside the backgrounds. We stacked bubbles – a city vista with a lot of neon that Lubo gave us for outside the windows, sky layers with two or three different times of day. We thought this would be the best way for David [Tattersall, Director of Photography], Larry, Andy and the actors to be constantly aware of what was behind and around the characters.”

The test happened in February 2007 at the Warner Bros. studio. “Larry and Andy directed a mini scene that David shot,” Gaeta said. “We were testing various HD camera systems and also getting source material for compositing the look.”

Setting Looks in Early Composites

While Tattersall was shooting the actors on green screen, Darren Poe, compositing supervisor at Digital Domain, pumped the footage through Codex Digital’s virtual file system into “Sparky,” real-time, game-engine based compositing software developed at Digital Domain, and worked with the directors on graphic treatments for the quick composites. “We did a lot of what would usually happen in DI in composite because it was so necessary to get everyone’s head around the shots,” Poe says. He experimented with color treatments, using shapes for stylized defocus, and sliding bubble layers.

“That was a great confidence-building moment,” Gaeta says. “We had temp backgrounds so the actors understood their surroundings, and for the directors to stage shots. It was a very interactive, interesting process. Larry and Andy designed shots around the environment. At the same time, David had information about lighting the actors on the green-screen stage based on the environment. We had our recipe.”

They also had their camera: the Sony F23. “The Sony had greater depth of field, which is not necessarily cinematic when you’re trying to compose close-up shots,” Gaeta says. “But we didn’t want the texture of the movie to be like film. We wanted a different feel. I remember talking to someone from another company who said, ‘But we haven’t met anyone who didn’t want the cinematic depth-of-field quality.'”

The One-Man Location Shoot

In Berlin, the crew revved up the recipe to accommodate the fast, 12-week shooting schedule. Location supervisor Marco Giacalone offered the Wachowskis options based on Paterson’s concept art, which sometimes involved real locations and sometimes didn’t. “We told the location department they didn’t have to worry about bringing in a film crew,” Gaeta says. “We’d be sending one guy. So the sky was the limit as long as permissions could be had.”

Martin and his world team of photographers shot the locations using Canon EOS 5D still cameras on a special rig built in Germany that, combined with software from Digital Domain, automatically took rows and rows of overlapping panoramic photos to create full spheres ‘ the bubbles. Martin’s team in Berlin then stitched low-resolution versions of the photos into the bubbles and gave them to Hristov, who was on the Babelsberg Studios’ lot in Potsdam, south of Berlin – the same studio where, fittingly, Fritz Lang filmed Metropolis.

“We made sure the versions handed to Lubo [Hristov] were literal proxies of the real environments in the correct color space so he could adapt his Photoshop files when he created final versions,” Glass says. “We were thinking through everything in our process to maximize efficiency.”

Hristov and his team of five artists received the bubbles approximately two weeks ahead of the shooting schedule for the shots. “Dennis would give us 10 bubbles and I would decide which ones to use,” Hristov says. “With the exception of one, a Mercedes museum that we turned upside down and I colored to become the base of the dressing room for the Grand Prix, all the bubbles were combinations of photographs, 3D elements, and matte paintings.”

For example, to create the city where a rally race takes place, Hristov combined the following: A 360-degree matte painting, three panoramic bubbles of photographs taken in Santorini, gates, minarets, towers, and other elements that Martin’s team shot of houses in Morocco, a section of a 3D model of a city square with walls and towers, and a semi-photographic sky with painted clouds.

When the directors wanted to shoot the locations from multiple angles, Hristov’s team created full bubbles that might be cylindrical or spherical depending on the shot design. To help keep costs and processing requirements under control, the artists also created 2D flat matte paintings that might be split into separate layers for multiplaning.

“We made most of the decisions about whether we should approach the work as bubbles or flat artwork from storyboards done by Geof Darrow,” Hristov says. “That gave us the staging for some of the shots. But the storyboards were done at an early stage.”

That Crazy Technicolor Look

Part of Hristov’s job was to give all the environments the same “crazy Technicolor look,” as he puts it. “It was something Larry and Andy wanted,” he says. “And John was constantly looking for more and more strong colors. So my job, in a way, was treating the photographs with anime colors to get more atmosphere from them with changing them too much, and then to add details and elements which would make them unfamiliar.”

He also created a variety of sky paintings at various times of day for the post houses to later layer behind set pieces and the 3D stadiums for the races. “That helped make the whole movie consistent in terms of color treatment and style,” Hristov says.

Typically, during production, Gaeta and Glass picked a hero bubble, the one most likely to be flexible enough for most of a scene, and had Hristov paint that in high resolution to use for color and lighting. Tattersall received printouts a few days before the shoot when possible. “At the least, we had him sit with Lubo [Hristov] so he and his gaffer would know where the main light sources were coming from,” Glass says. “We talked about lighting the actors to stand off from the backgrounds as in fashion photography or commercials, but we found it too distracting to do that all the time. So, quite often we had main key lights in the right colors and coming from the right directions.”

One of the beauties of working with HD, Glass notes, is that they could see the real colors during this process. “If you’re shooting on film, the image you get from the camera is a video feed and not a true representation of what film looks like,” he says. “We could see the real colors of the background and from the camera, and it was all being recorded without chemical irregularities.”

Getting the Right Light on Set

Often, Gaeta would show Tattersall the backgrounds on a laptop early in the day. “I’d have the bubble as a QuickTime VR that we could spin around to look at all the different sections,” Gaeta says. On the green-screen stage, they sometimes put a grid on the floor with key areas of architecture. “I might say to David, ‘OK, in this Moroccan bedroom, there’s an ornate shaped door with sun streaming through.’ And, he’d say, ‘OK, I see what you’re doing here. I’ll compose a good complementary for the characters.'”

Although they could have put the virtual backgrounds into the camera’s viewfinder, they found it was better to feed a composite image onto a monitor onstage. “He could see that while he was lighting,” Gaeta says. “Occasionally, he’d walkie-talkie a request to orbit the bubble while he was lighting.”

At the hub of this on-set process was the Codex Digital recording system. “It’s the breakthrough technology that unlocked the flow of production,” Gaeta says. “We could put all the captured material onto it, reformat [it] and send material out in different formats quickly. So it could not only join the onstage compositing, it was the centerpiece for editorial as we moved images around and in and out of the Avid.”

Matt McDonald, a visual effects supervisor at Evil Eye who had worked as a sequence lead for Gaeta and Glass on the last two films in the Matrix trilogy, helped with the on-set compositing. On stage, the directors had a bank of monitors with an additional side monitor that displayed the live composited image ‘ the background bubbles composited with the HD feed. The crew didn’t have real-time tracking data, so the background didn’t respond accurately in real time to what the camera was doing. Instead, a compositor watched and controlled the backgrounds.

“Using Sparky, Matt and I would try all sorts of lens options while the scenes were going on,” Gaeta says. “We could capture whole takes if we felt like it and mix and match focal lengths to see how the background could look.”

In addition to the hero bubble painted in high-res for color and lighting, Gaeta and McDonald also had other bubbles and backgrounds that Hristov’s team had created. As the directors worked with Tattersall and the actors, Gaeta and McDonald could switch bubbles to find the best matches for compositing using a map created by Dennis Martin and his world team of perspectives relative to location.

“We saved off frames as we went to have a later reference for composition in high resolution and sent them to editorial,” Glass says. “Editorial also got standard dailies and the low-res textures.”

Expanding the Role of Editorial

The experimentation continued in the editorial department, with the directors working with film editor Roger Barton and the assistant editors doing composite-oriented edits using the layers. In addition, the visual-effects team, with Tattersall’s encouragement, played with what Gaeta calls post cinematography, moving characters off-center, for example, or breaking perspective in ways that worked graphically.

“We had a powerful triangle of collaboration between editing, cinematography and visual effects,” says Gaeta. “We had this strange post-cinematography that visual effects was doing, and editing’s role went beyond the standard creative assembly of the storyline. Larry and Andy really got into it. Pushing layers is like moving a camera through space, and at some point it moves past collage and into composition and 2D camera motion that falls between cinematography and editing.”

Showing the Second Unit a CG Car

While the brothers filmed the actors on the green-screen stage, a second unit filmed the actors playing race-car drivers in fully dressed cockpits on motion control platforms on another stage. To give the second-unit directors the ability to see the CG car to better frame shots, the crew used General Lift’s Encodacam system. “It took feeds from the camera and from the motion platform so we got a very real representation,” Glass says.

The crew utilized three methods for controlling a motion-base platform provided by Don Gray. First, data from animatics approved by the directors could move the platform. Second, the directors could control the spin and tip of the motion platform in real time using a small controller. “They could do it themselves, but they usually called out instructions to Don,” Grass says.

And, third, the visual effects team loaded the CG tracks into Image Space’s rFactor driving simulator. “The driving simulator would calculate the motion and orientation of the cockpit based on suspension configurations and the nature of the track and terrain,” Glass says.

The Thunderhead track, for example, which was made of metal grates with holes, produced a rough ride that differed from the cracked earth in the desert flats. Rather than having the simulator work with the actual loop-de-loopy track designs, the team discovered it was better to use infinite flat planes with textures that gave the car a base movement.

“The most practical way to use the driving simulator was to supply large terrains with a particular texture,” Glass says. “Then Don would steer the platform by either watching a loop of the previs or a loop of the track itself. Someone would call out the action for the actor. It was a little tricky initially, but we got better at it.”

Finally … Post-Production

After production, Hristov took the files for the selected backgrounds back to Christov Design in Burbank and Bulgaria to recreate elements in full 12K resolution and then sent the bubbles and background elements on to teams in a variety of studios for their shots. At Digital Domain, Matrix alums Kim Libreri and Mohen led teams that created a majority of the racing shots using Hristov’s skies and data from the motion platform. BUF, Imageworks and ILM also contributed 3D shots. In addition, Jake Morrison at Exhaust handled in-house compositing with an assist from Rising Sun, CIS, CafeFX, Evil Eye, and Rainmaker.

“With Sin City, 300, and now Speed Racer, I think an artistic approach to movies has come back,” says Hristov, who also worked on 300. “We’ve gone through the documentary style of the ’80s and the computerized style of the ’90s, and I think at some point everyone started to get a little bored, so we’re going back almost to that interesting time of the German expressionists where the personalities of artists with one vision or one style can be recreated.”

He adds: “But, this was the first time I’ve been involved in a movie where most of the visual-effects action happened before production and during the production. John [Gaeta] and Dan [Glass] had that idea from the beginning. It was very innovative. It’s exciting that visual effects is becoming more a part of the artistic side and storytelling of movies.”