With War for the Planet of the Apes, Dan Lemmon, VFX supervisor at Weta Digital, received his fourth Oscar nomination. Last year, he received an Oscar and a BAFTA award for The Jungle Book. He has also received Oscar and BAFTA nominations for Dawn of the Planet of the Apes and Rise of the Planet of the Apes, four VES awards (for War, Dawn, Rise, and King Kong), a VES nomination for outstanding environment for Avatar, and a VES award for best visual effects in a commercial.

Lemmon’s first credit was as a digital artist for the 1997 film The Fifth Element. At Digital Domain, he was a character artist on Titanic, a digital artist for Fight Club, and a TD on The Lord of the Rings: The Fellowship of the Ring and A Beautiful Mind. He moved to New Zealand to work as a sequence lead for The Lord of the Rings: The Two Towers and never left.

War for the Planet of the Apes finds many of ape army leader Caesar’s tribe killed or imprisoned and Caesar, out for revenge, wrestling with his darker side. As the films have progressed from Rise to Dawn to War, the apes have evolved from being in a film centered on humans to this film in which the humans are secondary to the apes. The apes are the stars of War for the Planet of the Apes and, as always, they’re always CG.

Dawn director Matt Reeves, also directed War for Twentieth Century Fox. In addition to the VFX Oscar and BAFTA nominations, War has received multiple additional awards and nominations, including VES awards for outstanding visual effects, animated character, effects simulation, and compositing. It achieved a 93 percent positive critical rating on Rotten Tomatoes, and earned $347 million at the worldwide box office.

Also receiving visual effects Oscar nominations for this film are Senior Visual Effects Supervisor Joe Letteri and Animation Supervisor Daniel Barrett from Weta Digital, and Special Effects Supervisor Joel Whist.

“Making History” featurette covering the film’s mocap character animation techniques.
Youtube/Twentieth Century Fox

Studio Daily: Why do you think your colleagues voted for War for the Planet of the Apes to receive Oscar nominations for Best Visual Effects?

Dan Lemmon: This is a really strong year with very different films, all with high quality effects. I can only guess maybe what resonated was that we continue to push boundaries with realism and emotional connection to the performance of digital characters — characters that not only look realistic in terms of material and appearance, but feel emotionally realistic as well. We pushed the envelope in the way our light transport works, our rendering engine, snow effects, and in growing an environment. At the end of the day, though, it’s a character piece. It’s about whether the audience believes those characters.

How many shots in the film had apes and visual effects?

From start to finish, this film is 100 percent about the apes. Even scenes with humans are from the point of view of the apes and there are apes among them. There were 1,290 visual effects shots, but we had long close-ups for 20, 30, 40 seconds. With Matt using his elegant, long, emotionally engaging long shots, the count comes down. But, the difficulty doesn’t. The movie is two hours and 20 minutes and all but 15 were visual effects shots.

L-r, Karin Konoval, Terry Notary, Andy Serkis and Michael Adamthwaite.
Twentieth Century Fox

Did you make any changes to the apes for this film?

We’ve learned a lot from the previous two movies and other films, as well — what sorts of things we can do to make an ape face communicate emotionally with a human audience. We had a good understanding designwise about how to make the ape faces feel like they’re experiencing relatable human emotions — to put on an ape face some version of all the wrinkles and folds and the things you see come and go on a human face.

We’ve been working on Caesar for eight years. But the emotional stakes were so high in this film. Caesar is wrestling for his soul, fighting his dark urges, his desire for revenge, and his need to take care of his tribe. You see that in Andy’s [Serkis] performance. We’ve done work with Andy in the past and seen some pretty amazing performances, but this was so conflicted and emotional that we had to look at our rig again and figure out how to translate his performance to Caesar’s anatomy.

We changed the way his brow furrows so we can see sad and angry at the same time; we rebuilt those shapes to more closely reflect what Andy was doing. The preservation of volume is more accurate in the cheek area when he snarls and when his mouth is quivering. And the way the wrinkles piled up and pushed the flesh around, the way they came and went, was more realistic.

This scene features Bad Ape (Steve Zahn) and Maurice (Karin Konoval)
Youtube/Twentieth Century Fox

Were any other ape stars tricky to work with?

Caesar was definitely the most difficult. But Bad Ape was a challenge as well. He doesn’t look like Steve [Zahn] at all. We couldn’t use our tricks like slipping an actor’s face on the character design to make the shapes match more. We had to lean on our facial animation leads to make Bad Ape’s facial expressions match Steve’s, even though their faces don’t look anything like each other.

And in the last movie, the only thing [orangutan] Maurice said was, “Run.” So we knew we’d have to contend with his dialog in this one. We did a funny test. We took Karen’s [Konoval] dialog from a press interview and lip synced it onto Maurice. It felt cartoony, like an Aardman animation. The mouth shapes were hyper-articulate and moved quickly from one frame to the next. It was a lot of fun, but it wasn’t what we wanted for this movie. Maurice’s head is so big, his mouth is so big, we had to make sure he wouldn’t look cartoony as he spoke. And Karen did a good job of slowing things down and making him weighty when she was actually performing as Maurice.

Karin Konaval as Maurice
Twentieth Century Fox

Steve Zahn as Bad Ape
Twentieth Century Fox

How many apes were in this film?

There were about 45 distinct individual characters. We had several species and ages, [including] toddler orangutans and gorillas for the first time. We had seven or so hero characters signing or with speaking parts for the first time in the franchise. Then lots in the mid-ground as well. Many shots had hundreds of apes. Some of the big shots had on the order of 1000 apes. For deep background, we tend to use the same 20 over and over.

Did the way you motion-captured the actors playing the apes change for this film?

Well, it’s the first time I’ve had to dig motion-capture cables out of ice. We were shooting on tops of mountains in full Canadian blizzards. One day we had to shut the production down because the storm was so intense. But the motion-capture gear held up great. That’s the beauty of this system. We can take the tools anywhere. The actors on a mountain in a blizzard are reacting to the environment and to each other. Later we have all the best possible data to create digital characters and make them look as close to what the actors were doing as possible.

Apes in snow was a new challenge for this installment of the franchise.
Twentieth Century Fox Film Corporation

So you had to put snow in the apes’ fur.

Yeah. That was a big challenge. It was one of the first things Matt Reeves talked to me about. He said, “First of all, apes in snow. Are you scared?”

I had this image of Ceasar on a horse in driving snow like in a Western. Really exciting. But it’s hard. The apes roll through the snow. Caesar is out in the elements. We studied mountaineers, the way snow sticks to their hair and beards, and saw that the snow closest to the skin tends to melt and get slushy. Then, farther away, it refreezes into harder chunks, and then farther out it’s powdery. So, we rewrote our fur tools. When snow is close to an ape’s skin, the snow deep in the volume turns into water and then when it’s out, it’s powdery. There is ice, too, but we don’t see it turn to ice over the course of a shot. There’s nothing that would have prevented us from doing that, it just wasn’t required. It would have happened between in the cuts between shots.

An ape archer.
Twentieth Century Fox

You also mentioned a technical advance in rendering …

When we looked at reference photography of apes, we noticed that our backlit fur didn’t have the same specular breakup. It was almost waxy, glassy, and uniform. So we dug into the details. Apes have a specific unique structure to their hair fibers. We’d been modeling hair like solid glass tubes. But apes’ hair has a spongy hollow medulla center, which is surrounded by a pigmented cortex, and a scaly cuticle surrounds that. They all combine to bounce light in a specific way that’s important to make it realistic. So the shader writers and rendering team figured out how to simulate that multi-layer hair model. We define the hair as a curve and then when our rendering engine Manuka hits a hair fiber, it says OK, this is what I need to do optically in my light transport calculations to simulate those different layers of the hair. It works really well, but you have to start with accurate lighting that matches the plate photography, and that is where Physlight comes in.

What is Physlight?

Physlight is a suite of tools we’ve created to accurately record the physical light on the set and model within Manuka the way the light behaves in real-world units, in terms of lux and lumens and the correct spectral distribution of light. That’s something new for this movie. One important component of this is Physcam.

Physcam?

That’s our digital model of the film’s physical camera. For this film, we profiled the Alexa 65 camera sensor, and we also recorded and tracked all the camera data from the shoot, so we could later use the camera’s photographic settings in our renders. The T-stop, the focal distance, the camera’s ISO and color temperature settings, all of that information fed into Manuka along with the Physlight data that mapped the actual number of photons flying around in the scene and all of their specific wavelengths. Using Physcam, we can match the way the production camera recorded those photons. It means less guesswork in terms of exposure and color space and depth of field for the compositors, all those things that were a little nebulous in the past.

Karin Konoval, left, and Amiah Miller.
Twentieth Century Fox

You also mentioned an advance in growing the environment.

That was one of the biggest steps forward. We wrote a new ecosystem simulator that we call Totara after a native New Zealand tree. Our older Lumberjack program was great for growing a single tree, but what makes a jungle or a forest realistic isn’t just one tree, but how the trees relate to one another. One tree’s shadow may influence another tree to push higher. The limbs on the lower part die off and fall away. Shade-tolerant plants thrive in the understories. These things are very laborious for a human to go in and build and prune manually, and it’s a challenge to get them to look right.

How does Totara grow trees?

We set up growth rules for species of plants and for what environment each species thrives in. Then we throw a bunch of seeds together, a mix of species, and distribute them across, in this case, a mountain terrain where there is a concentration of resources. As the seeds start to grow, they compete with each other for sunlight and nutrients. They orient to the sun path. Some grow faster. Some can take colder temperatures and can grow at higher altitudes. We run the simulation over 80 years or more. As the trees grow, they influence each other. The distribution and arrangement of trees and pockets emerge naturally as the trees influence each other. As the altitude rises, the species change. We get correct negative spaces and undergrowth where branches won’t grow.

Can you art-direct the simulation?

We can freeze the simulation, shift trees, and make adjustments. We grow it in patches — grow one mountain and then another. The simulation tends to finish overnight. The simulations are pretty manageable.

Did you have other simulations?

We had a fair few. We had to problem-solve the waterfall — making it feel right in volume, scale, and participation between air and water. We wanted to make it feel like all the elements were coupled — the water particles drag elements in the air volume and the air affects the mist. We did the same thing with explosions.

Because of the run-up — we usually have to let a simulation run for as many frames as are in the shots — and because of the long shots, rather than overnight simulations, the big waterfall shots could take two days.

Woody Harrelson
Twentieth Century Fox

What was the best thing about working on this film?

It’s a pleasure to come back to these characters and put them in situations where they have great new performances. And it’s really neat to push the environmental tools to the next level. It’s an amazing time to make visual effects. The hardware is fast enough that suddenly these things we knew we had to cheat because of cost and render time are opening up and we can start to do things the right way, the way we’ve always wanted to do them.

The hardest thing?

Caesar’s performance. Getting him to feel 100 percent believable and faithful to Andy Serkis but still look like an animal and still have that human emotional connection. It came down to the caliber of the animators. There were countless other technical challenges. The avalanche was a big one, to make it feel real but have an element of danger and drama. How and when to set off charges to have the avalanche nipping at Caesar’s heels without breaking the rules of physics. It took a lot of art direction.

What about that last scene in the film?

It was an emotional moment on the set. It was the very last scene shot in principal photography. After three films of working with Andy, and the last two with Matt, it was a big moment. Of course Karen and Andy were wonderful. People were crying real tears on the set. In post, you get distracted by the challenge of the process and by digging into the details. But, then we’d play the scene in dailies with that Michael Giacchino music and there wasn’t a dry eye in the room.

You’ve won many awards for movies with apes in them — the Apes series, The Jungle Book, King Kong. Would you do another?

I’m ready for a different challenge, but never say never. There are reasons why we find these characters so interesting and emotionally engaging. They’re genetically so close to us. So there will continue to be stories that feature apes. As for this series — there are threads left open. When they discover Bad Ape, Maurice tells Caesar that he didn’t know there were others. There’s a big world out there where humans have gone into serious decline, and it could be full of these amazing animals that have the evolutionary advantage. I’m excited to see if they decide to pursue that.