Making the Hulk, Faking New York, and Going Splat

Jeff White’s nominations for a visual effects Oscar and BAFTA award for his work at Industrial Light & Magic on The Avengers are his first, but they follow five VES nominations and a VES award for Outstanding Visual Effects in a Special Venue Project. White’s first film credit was as a digital artist on the stop-motion feature, The Nightmare Before Christmas. He joined ILM soon after as a creature technical director on the 2004 film Van Helsing. By 2007, he was a digital production supervisor on Transformers, then associate visual effects supervisor on the second Transformers: Revenge of the Fallen, and co-vfx supervisor on the 2011 Transformers: Dark of the Moon. He solo’d as a visual effects supervisor for Transformers: The Ride – 3D. White shares his Oscar and BAFTA nominations with overall visual effects supervisor Janek Sirrs, Weta Digital visual effects supervisor Guy Williams, and special effects supervisor Daniel Sudick. In addition to the Oscar and BAFTA nominations for visual effects, animators at ILM received two Annie nominations, and the ILM environment and modeling teams each received VES awards. Directed by Joss Whedon, the Marvel Studios production released by Walt Disney Studios has earned more than $1.52 billion at the box office. We talked to White on the release of The Avengers last year, and circled back for his Academy Award nomination.

Studio Daily: Why do you think your peers picked The Avengers for visual effects Oscar nominations?

Jeff White: I think it’s the breadth of the visual effects work, and how well it supports the story. It’s a captivating story with great dialog. Joss [Whedon] did a great job of managing the story. It wasn’t overwhelmed by the action sequences. And then I hope it’s for the Hulk and New York City. Creating a digital character is one of the hardest things in the industry, and comic book fans are a tough audience.

Do you think it pushes the state of the art of visual effects?

It absolutely does. We had to rebuild our digital human from the ground up. To make Hulk happen took not only tremendous artistry, but also an incredible amount of software development. To pull off a believable shirtless human talking, emoting, [and] stealing scenes is quite an accomplishment. It was the hardest thing I’ve ever undertaken.

This is hardly the first time you’ve been asked about this film. How many Avengers interviews have you done now?

[laughs] It’s up into the hundreds. It’s been an incredible experience. It’s great that people are so crazy about the film and want to know about the visual effects. You don’t always get that about the work we do, so that has been fantastic.

What do people most want to know about?

Our approach to the Hulk.

There have been several Hulks before. Why was this Hulk so successful?

I think the character was successful for a number of reasons, not the least because Joss gave us great things to do with him. It isn’t often that you go to the theater to watch your work and the entire audience breaks out in laughter — and they’re laughing so hard they miss the next line. In visual effects, there’s a kind of work we do that’s invisible, like creating New York, but nothing is as gratifying as getting a good audience reaction.

I’ve read that to create Hulk the crew worked from 4,500 images and photographs of Mark Ruffalo.

At least, if not more. As soon as we started into the Hulk it was an art project. We worked with Joss, and we grounded everything around Mark Ruffalo. We photographed everything we could to create a digital Mark Ruffalo. We did Light Stage scans, dental molds, everything.

If we had just said that we were going to create the Hulk, we would have modeled, textured, and rendered [a character]. Then we would have seen a number of things that didn’t look right and we’d have tried this and this to fix what was wrong. Instead, our approach was that we spent a lot of time getting a digital Mark Ruffalo right and that solved a lot of the problems. The other thing about that collaboration was that we had Mark do the performances. By doing all the performances for us on set and on a motion-capture set with dual head cams, he gave us something we could always reference even if we didn’t use the performance.

You have a background in stop-frame character animation. Did animating tiny characters help you supervise work on the Hulk?

[laughs] I would call it more animation tests back then than character animation. I worked at Vinton Studios, which became Laika, which is nominated this year for ParaNorman. I used to sit with Travis Knight [now Laika CEO and ParaNorman lead animator]. I was absolutely floored when I saw ParaNorman, knowing how difficult it is. It’s so exciting to see them continue their craft. But, yes, in so many ways animation is animation. There are different techniques and approaches, but the principles are similar.

Four people on your crew received a VES award for the New York City environments in Avengers. Did the crew at ILM use the same techniques for those environments as they did to build Chicago for Transformers: Dark of the Moon?

The concept was similar in terms of using re-projection, but Avengers had a bigger dataset. For Chicago, we had several rooftops and we flew down two buildings. On Avengers, we had many streets and lots of rooftops. The other thing was the dressing of the city. Chicago has a different flavor. In New York, there are tons of taxicabs, police cars. We made 200 props for New York City set dressing.

When I first heard about Avengers, I was excited about spending a summer in New York City – I’m from New York. But we couldn’t film there. We couldn’t get a helicopter low enough. And we needed fires, explosions. So I ended up in Albuquerque. Which was nice, but it wasn’t New York.

I remember learning when we talked with you earlier that the crew took 275,000 photographs and made 1800 spheres. 

The build of New York City was the largest environment build we’ve undertaken. The good thing was that we had great previs that mapped out all the action. [See Studio's previous coverage of Avengers previs.]

Did you develop new technology to help cope with the environment?

We had a new window shader that let us add window blinds and aging and use a simple look-up for interiors. And Splat was new with this show. .

Splat?

It’s a shader. As we moved our camera, Splat would take the best of what each sphere could contribute as a projection and render out projected geometry. It was born out of the large amount of photography we had for this build. We needed to quickly get a camera in from animation and be able to render out a quick version of the environment in that area. There were several times when we put the camera where it should be in New York, but the buildings looked dull and the lighting didn’t look real. So, we’d do versions with this new Splat renderer and it knew how to properly occlude from any flight path. We could put three options in front of Janek [Sirrs] and Joss to give them some variety before we got into the labor-intensive work of creating the shot.

Were there other interesting challenges?

One of the challenges and also the great thing about this project is that there was no way to separate the film and the assets along hard lines. Once the shows are awarded, competitors have to collaborate. And I’m good friends with [Weta Digital VFX Supervisor] Guy Williams. We had Hulk on the helicarrier. Weta destroyed engine three and sent it to us, and we sent it to Scanline. It went pretty smoothly. The big thing was that Janek Sirrs oversaw everything and made sure we were all on the same page as far as the look went.

Before The Avengers, you worked on the three Transformers movies with Michael Bay.  Was working on this film a lot different?

Three Transformers and the ride, so it’s like three and a half. [laughs].

Both directors are great in different ways, and it was fun working with both of them. When I heard that Joss was involved with Avengers, that was my primary motivation for getting involved. I really loved Firefly. When I was working long nights on Lion, Witch, Wardrobe [The Chronicles of Narnia: The Lion, the Witch and the Wardrobe] we’d watch Firefly, and those 13 episodes just flew by. I always loved his characters and dialog, and I think bringing that to a huge visual-effects film panned out really well. He’s so good with the actors, and that translated naturally with the animators.

Do you have a favorite shot in The Avengers?

My favorite shot is still Hulk slamming Loki, because of the audience’s emotional reaction.

Also, I like the long shot where all the Avengers are working together — a continuous shot from Black Widow to Iron Man to Captain America to Thor to Hulk. We started that one on the first day and finished it on the last day. Everyone was working hard across all the disciplines. It was definitely our hardest shot. What I love about it is the idea, to show the bickering Avengers now working as a team. It was a great moment. Unexpected. The perfect emotional release after a huge action sequence.

How does it feel to receive an Oscar nomination for your first film as a visual-effects supervisor?

Beginner’s luck. There was so much incredible visual-effects work at the bakeoff. To be in the list of five – I feel honored.

When I heard about the nomination, it was early in the morning and I was just getting up with our twins. My wife told them to say congratulations to Daddy, and Piper said, “No. I don’t want to.” [laughs] They keep me pretty grounded.

Do you know what film you’ll work on next?

I’ve started pre-pro on Transformers 4. [ILM visual effects supervisor] Scott [Farrar] is back, so we’ll work on it together. We have a great time every time we do those pictures together. It’s just so much fun.