ILM Camouflages CG Work for Jarhead

When principal photography began on the film Jarhead, the visual effects crew at Industrial Light & Magic expected they would create 89 shots during a scheduled 10 weeks of post-production. By the time the post ended, the studio had crafted more than 400 shots – around 37 minutes of visual effects, all of which the crew hopes will go unnoticed. With the exception of a CG scorpion and a dream sequence, the effects were largely in the background — photorealistic set extensions and digital environments.

Based on a memoir by a U.S. Marine, Anthony Swofford, Jarhead takes place during the 1991 Gulf War as “Swoff” (Jake Gyllenhaal), a lance corporal in a scout/sniper platoon, progresses from boot camp to active duty in Desert Storm. Once the troops are deployed, there’s a lot of sitting around and waiting. Although the VFX crew at ILM touched shots throughout the film, most of its work can be seen toward the end.

“Every shot in the third act has visual effects,” says ILM VFX supervisor Pablo Helman. The crew worked at 2K resolution on high-contrast film scans from negatives treated with a partial bleach bypass.

The people who created these effects believe they could not have achieved them two years ago- at least, not on this schedule. Because they could, they gave director Sam Mendes (American Beauty) and cinematographer Roger Deakins [see sidebar] unprecedented freedom and creative control.

“[Mendes] embraced visual effects as quickly as any director I’ve worked with,” says Helman, who supervised visual effects for War of the Worlds, Master and Commander, Terminator 3, Star Wars: Episode II and other films. “He had never really worked in visual effects, so he didn’t know what visual effects as a tool could do. But when we got into post and he realized he could change things that weren’t working for him, he got it right away. He found out what Steven Spielberg and George Lucas found out a long time ago: You can change the environment, you can change performances, and you basically have a lot of control.”

An example: A “friendly fire” sequence called for soldiers to walk through the desert as American planes flew toward them and began shooting. ILM’s job was to add more planes and remove mountains to create a desert plain continuing into the distance.

“We wanted this endless flat desert with no relief in landscape,” says Deakins, “but the only place we could shoot had mountains around it, so we needed to get rid of the mountains. I remember having a conversation with Pablo telling him that when the characters were shelled, there would be debris and smoke in the air. He asked if we could do blue screens, but once he stood [on location] with us, he knew there was no way we could have a two-mile-long blue screen. I didn’t want to lock off the shots; I wanted to shoot handheld. Pablo, bless him, said, ‘I know what you’re going for. We’ll just deal with it.’ That really blew me away. I think this was the first time I worked on a film and didn’t have to think about altering the way I was shooting something.”
Once Mendes saw the sequence, though, he wanted it changed. “The soldiers were walking through the desert and there was nothing around them,” says Helman. “He realized there was no pressure, so he said, ‘I want chaos.’” Therefore, in addition to removing the mountains from behind the debris and the soldiers, ILM added planes, explosions, smoke, burned vehicles, and military berms, fitting the soldiers among the added elements.

“Every visual effects shot needed rotoscoping because so many things had to be inserted or removed,” says Beth D’Amato, digital paint and rotoscope supervisor. “I supervised more rotoscoping for this one show than any other, including Star Wars: Episode III. As we delivered and the director saw what we could do, he wanted more.” At one point, D’Amato had 12 people on her crew, up from a planned four or five.

Moreover, because Deakins shot most of the film handheld, with a camera on his shoulder, this shot, along with many others, could not be painted and composited in 2D.

“The shots had very subtle parallax that we’d lose if we had one flat plane,” says Grady Cofer, digital production supervisor. Instead, match-movers working in ILM’s Zeno (a new proprietary pipeline the company developed in collaboration with LucasArts; see “ILM’s Game Theory” in this issue) tracked the camera, and compositors working in 3D space in Inferno and Shake layered in the elements. For the background and ground plane, the compositors projected images onto geometry shaped like a dome and then placed cards with fire, smoke and burned car elements into the 3D space.

“We had those elements because we had shot them during principal photography,” says Cofer. They shot the elements because Helman and VFX producer Jennifer Bell had anticipated Mendes’ future requests.

“We were all protecting him, so we covered ourselves in every location,” says Helman. “I even had pictures of soldiers so I could map them onto geometry, although he hadn’t asked for that. It’s my job to be prepared. Three months into production, if he asks for an element, I don’t want to say, ‘Oh, we can’t do that.’”

Because the filmed elements and digital stills were shot during principal photography, the results matched the lighting in the plates.

Sometimes the crew projected filmed elements onto 2D cards, sometimes they projected the photos onto 3D geometry, and sometimes they used the photos to create 3D geometry. “We could quickly create low- and medium-resolution models from the photographs and then slap the photos back onto the model,” says Cofer.

In one sequence, for example, the artists at ILM transformed a white commercial helicopter into a dark military helicopter. To create the military helicopter, they projected photos of a correctly painted helicopter onto 3D geometry and tracked it into the shot.

Similarly, they converted a location in California into a Saudi airport. In the plate, soldiers walk down the exit stairs of a gray United Airlines plane onto an airfield where several soldiers mill around a few other planes. In the final sequence, the soldiers exit a white TWA airplane and walk onto a tarmac filled with soldiers and airplanes. Because everything in the frame was constantly moving, the perspective was changing, so a simple 2D paint job on the body of the plane behind the exiting soldiers wouldn’t work.
“The camera was not locked,” says Helman. “It’s on the shoulder, and the soldier is walking down a ramp. We had to track the plane in Zeno, photograph a model of the new plane, and project the photos onto new geometry. Then we added more planes in the back and more troops.”

The models were small replicas created for the desks of airline industry personnel. “When I first saw the models, I was shocked,” says Brett Northcutt, lead digital matte artist, “I smiled at Pablo, but I was thinking that this will never work. I was blown away by how good
they looked.”
To create the shot, painter Joshua Ong photographed the model planes from every angle represented in the sequence and projected the photographs onto simple 3D geometry. Then, using PhotoShop from within ILM’s Zenviro, a camera-mapping software that allowed him to see the camera view in every frame, he lined up the replacement planes with those in the plate, changed windows and added panel lines and grime.

Welcome to the Suck

But as tricky as these shots were, they were only a warm-up for sequences in the burning oil fields. As before, the camera was handheld; everything in the frames was constantly moving.

Cofer describes the sequences: “First, Swoff, that’s Jake’s character, spots the oil fires on the horizon, distant plumes that become their beacon, and the soldiers begin walking toward them. The plumes collect into a canopy, like an encroaching storm, as the sequence evolves. Oil rains out of the sky and covers Jake and the other characters, and he says, ‘The earth is bleeding.’ Then we cut to the soldiers in the middle of it, surrounded by oil and fire and smoke, digging foxholes into the ground, covered with oil.”

The crew created all the oil-rig fires from photographs of one practical fire. “They had only one fire, so every time we rolled a take, I was there with a camera,” says Helman. “We shot it from different distances and with different speeds to get a library of oil fire elements.”

On location, the production crew had blackened a 150-square-foot area in the desert and sprayed vegetable oil onto the characters. Deakins moved among the actors, camera on his shoulder. Once the plates arrived at ILM, the effects artists blackened the sand farther into the distance, inserted oil-rig fires and layered in smoke.

“There were 100 frames with 50 walking soldiers who never left the frame,” says D’Amato. “Every soldier in the foreground and many others needed to be completely articulated — cut out like a cookie cutter and animated — so we could insert fire and smoke behind. They’re wearing helmets and goggles, some have backpacks, some have radios, and every single piece needed to be accurate. We couldn’t have a smooth shape on an arm where the camouflage was wrinkled and bumpy. So, a lot of them needed to be cut out in every frame. No one will know what we did, but it gave the director flexibility.”

To create the illusion that the soldiers were surrounded by fire, the artists painted fire reflections in the soldiers’ goggles and in the oil on their clothes and on the ground. Toward the end of the sequence, a horse walks into the burning oilfields at night. The air is so filled with smoke and fire that everything seems black and orange. This shot was filmed on Stage 12 at Universal Studios, which was filled with practical smoke.

“We had a little bit of fire, which gave the DP interactive light that lit up the smoke, but it was out of scale so we had to replace it,” says Helman. “He also had giant banks of lights that acted as fire.”

In post, ILM painted ribs on the horse to make it look anemic, replaced the banks of lights with fire elements and added more smoke using a 3D match-move to place layers in different depths of field. Although the lights on the set helped the compositors place the fire elements, they made the rotoscoping more difficult.

“As the soldiers passed in front of the lights they were silhouetted, but the light created dimples in their profiles,” says D’Amato. “If we wanted to extract a soldier to insert the fire and place him back over it, we had to paint out the dimples and reconstruct the shape of his
head as he moved and turned. It was fun. Doing reconstruction gives us a chance to flex our creative muscles.”

At the end of the sequence, the soldiers march, on stage, out of the nightmare. Digital matte artists transformed the set into a desert scene with oil wells, electrical towers and various vehicles. Also, after several takes of marching on stage, the soldiers’ march had created white tracks in the blackened sand, so the painters filled the tracks with black sand.

All told, the painters created around 35 matte shots for the film, according to Northcutt – establishing shots to give wide views, out-the-window shots, such as one showing Swoff’s buddies on the horizon as if they were a mirage, and some with interesting combinations of the two.

At one point, for example, Swoff and Troy (Peter Sarsgaard) watch as an air control tower concealing Iraqis is bombed. For this shot, Northcutt created a matte painting, complete with planes and bunkers, that extended a wall into an Iraqi airfield. In addition, he layered a filmed element of a practical explosion – a miniature plate – into a broken windowpane. Swoff looks out the window; the element looks like a reflection behind his head.

“This is the opposite of what a director would do in a visual effects movie,” Helman says. “In a visual effects movie, the director would have shot the explosion; the money would have gone into the miniature. This director didn’t do that. He wanted to see the reflection and that way he got the actor’s reaction, too. The work is so subtle.”

Helman believes the tools in ILM’s Zeno pipeline, which encourage artists to be generalists, made it possible for the team to create five times the effects originally slated on the same schedule. “The reason we were able to do this much complicated work in a short time was
because of Zeno,” he says. “Before, someone would do a match move that would go to a TD and then to compositors. Now, the tools let people jump from 3D to compositing to 3D paint and back into compositing.” This way of working is becoming especially important, he notes, as directors learn how they can use visual effects to control final shots and as post schedules tighten.

“It started with War of the Worlds for us,” he says. “It’s starting to be a pattern.”

DP Roger Deakins on the Beauty of the Ugly

Roger Deakins, director of cinematography for Jarhead, five-time Oscar nominee and two-time ASC Award winner (The Man Who Wasn’t There, The Shawshank Redemption) talks about
getting Jarhead in the can.

F&V: Is the camera handheld for the entire film?

Roger Deakins: Just about the whole film. We made that decision quite early on. Sam wanted a sort of immediate feeling to it, of someone being there, observing it and grabbing it. But it’s steady handheld, not gritty. Some people won’t even know it’s handheld.

F&V: The images are so beautiful, but the content is often horrible.

Deakins: There is something very strange about it, a certain surreality. This is a true story, but the situation is surreal. That’s the dichotomy. The thing about the oil fires — we were watching the Werner Herzog documentary [Lessons of Darkness] shot after the Kuwait war. In the Herzog documentary, there were miles of huge fireballs and lakes of black oil reflecting the flames. We only scratched the surface creating what we did. But that’s what we were trying for, this vision of people in a world totally alien, so surreal, so outside normal experience. In the “Highway of Death” sequence, there are corpses on the road. It was strange shooting it. The bodies are not real, but it’s so realistic, it felt odd being in that situation. I was a stills photographer for a while and admire war-zone photographers. The photojournalist Don McCullin said he knew it was too much when he found himself rearranging dead bodies to make a more beautiful composition. It’s interesting that the most ugly things have a certain beauty.

F&V: Why did you do a bleach-bypass on the negative?

Deakins: I wanted a stark, contrasty, grainy look. I did tests, but I didn’t think I could get there in DI. You have to affect the negative to bring in the grain, so I did a partial bleach-bypass on the neg. The DI is not a be-all and end-all; it can help and enhance. What’s on the neg is what’s on the neg. For Pablo [Helman, ILM VFX supervisor], that was a challenge. We tested and Pablo felt ILM could do it, and they did. We roughly pre-timed the file we sent to Pablo and he matched the texture.

F&V: Did you scan at 4K resolution?

Deakins: We scanned at 2K double-scanned, downresed to 2K and then upresed. I wish everything could have been 4K all the way. The effects are some of the best I’ve seen, but it makes a slight difference, especially on the wide day scenes.

F&V: Why do you think the effects are so good?

Deakins: Pablo was in sync with what we were doing because he was there every day. He used film we shot of the oil fires quite a bit. I think there’s quite a difference between photographing things for real and trying to paint them. That’s what I like about the look of the film. It was a good blend. The visual effects are servicing the story; they aren’t bombs sliding through the air crashing around. The trick with all the compositing, matching the lighting we did, it works because it’s all inherent within the scene. I think even the hardest scene — bivouacking under the [raining] oil — is quite breathtaking.

F&V: Did you change the way you worked to accommodate the visual effects?

Deakins: That was the great thing. We had the freedom to be able to shoot handheld and leave lights in the shots. We didn’t shoot anything differently. It was amazing how ILM could retain an image, even with smoke in it, and replace the background. Many of the visual effects were duplication of vehicles and characters, but there was also a huge amount of mountain removal. It’s fantastic that they could take away an element and give us a different shot.

F&V: How did you light shots with oil fires?

Deakins: The night shots, the scenes with the horse, were all shot on a stage with smoke. I lit it with three or four bat strips – 12-foot wooden batten strips with 500-watt mushroom bulbs on each bat that were clustered in a circle six feet across. I could shoot into it and it wouldn’t flare into the lens because of the atmosphere. When Pablo’s crew took the lights out and put in oil fires, the light was natural and flickering because it was coming from light actually doing that. On top of that, he put in more smoke elements and power lines and stuff to give a sense of depth and perspective. For the scene of the characters digging in the sand, bivouacking under the oil, we managed to get a lot of smoke in the sky from an oil fire in the middle of the shot. All the characters are real, but the background was bright-day desert beyond the black smoke. It was all handheld going in amongst the guys, the camera tilting up and down. Pablo did an amazing job extending the black into infinity and adding smoke and atmosphere. The big end-of-war celebration, when the soldiers are sort of lost in the desert, was lit by oil fire light. We lit it with flickering lights that Pablo’s crew replaced with flames. It was a fantastic way of working. It gave us so much freedom to shoot completely toward our lights knowing that our lights would be replaced.

F&V: Would you change anything?

Deakins: We need a better system for looking at images remotely. [Deakins was filming The Assassination of Jesse James by the Coward Robert Ford in Calgary, Canada, while Jarhead was being color-timed.] It’s hard for me to communicate what I want — especially for some of the interiors where I don’t want the windows outside to be the same density, relative to the rest of the shot. It’s hard to keep going back to Efilm, but we’re all over the country trying to look at the image remotely. It’s hard to say, “I don’t like that, but maybe this,” and know we’re all looking at the same image.”