On Analog Previs, the Future of Virtual Cinematography, and the Value in Trusting His Instincts

Rob Legato’s visual effects Oscar nomination for The Jungle Book is his fourth. He has received two Oscars — for supervising visual effects in Titanic and Hugo — six BAFTA nominations, and two BAFTA awards. The BAFTA awards bookend his career — one for Apollo 13 in 1996 and the second for The Jungle Book this year. Legato also received two VES Awards this year for The Jungle Book: outstanding visual effects in a photoreal feature and outstanding virtual cinematography in a photoreal project. They add to his total of eight VES nominations and, now, five VES awards. Among the 27 films to his credit are six feature films with Martin Scorsese for which he was visual effects supervisor and second unit director. He conceived the virtual production for James Cameron’s Avatar. For The Jungle Book, he was second unit director, second unit director of photography, and visual effects supervisor.

Legato began his career in episodic television, winning Primetime Emmy awards and nominations for visual effects in Star Trek: Deep Space Nine, and Star Trek: The Next Generation.

He shares his Oscar nomination and BAFTA award with Adam Valdez, who supervised the Moving Picture Company’s work on all the environments and animals in The Jungle Book except Weta Digital’s sequences with King Louie and the monkeys; Andrew R. Jones, the overall animation supervisor; and Dan Lemmon, visual effects supervisor at Weta Digital. 

Directed by Jon Favreau, the Walt Disney Studios release is based on the series of books by Rudyard Kipling and inspired by Disney’s 1967 animated feature. The Jungle Book earned $966,550,600 worldwide in 2016, making it the fifth top box-office earner last year.

Mouseover film still above to compare to original live-action plate

Studio Daily: Why do you think your colleagues voted for The Jungle Book to receive Oscar nominations?

Rob Legato: What I got from most of them was the fact that they understood what we were doing to make every shot look like it was photographed by a cinematographer, to look very consistent, and look like a movie that was well designed, photographed, and art directed. They knew the entire film was fabricated, but it didn’t feel fabricated. It felt like we went out, photographed the movie, and did a nice job. And they know that it isn’t easy in VFX. So I think people appreciated the time and effort and loving care. Also, Jon Favreau directed a well acted, entertaining movie.

You used a process you call “analog previs” with virtual environments and animals to inform the filming and later, the post-production. What is analog previs? 

With CG previs, you normally make choices in a sterile environment. You kind of guess at it, like you guess at storyboards. But when you shoot a movie, you look through the camera at the scenery, the actors, and you start to make choices based on the input you see, the lenses, the camera height, the camera moves — that’s all input. So for this movie we took a virtual camera, walked on stage, and did exactly the same things as when we’d shoot a movie. We shoot the same thing from five different angles. We shoot dailies. 

Then later, when we are editing, we can make analog choices. I can say, “Let me cut to a close-up," or "Let’s go wide with this one and insert a new line of dialog.” If we had predetermined chunks without live action, if we didn’t have a close-up or wide shot overlapping the same action, we wouldn’t make that call. Instead, we could have inspirations while we were editing. That’s difficult to do in advance, and that’s why I’m not a fan of storyboarded sequences. They’re written in stone on paper. People bid on them. There is only one pre-chosen angle, not different angles for the same action. Storyboards can be pretty good, but they don’t offer the same continual inspiration one has when photographing for real.

Who creates the analog previs?

We have the artists who will do the movie actually create the shots, the shot selection, and the edits. We don’t have a previs company do our previs. We built analog tools so I, Bill [Pope], and Jon [Favreau] could pick up a camera and it responded like a real camera. Handheld, Steadicam, dolly, crane, helicopter — it looked the way it would look. So they’re just making a movie. It’s very personal. It’s their personality. Jon [Favreau] directed people in motion-capture suits. When Bill [Pope] moved his head left and right, or I did, it would be just like we were shooting for real. It didn’t look 100% real — we had game engine graphics — but it was very fluid. We could take a step forward, try a different lens, a different dolly, and get fast iterations in 30 seconds. During the entire process, we’re always responding to what’s in front of us. 

Can you see lighting in these virtual worlds?

Yes. I don’t want to oversell — it’s never as good as final renders. Finals sometimes took 20 or 30 hours. But the intention was clear. The height of light, the shadows. It’s like when you’re shooting a movie and it’s before the right time to shoot. You know where the sun will be [so] you set up waiting for that moment to occur, knowing what shot you’ll get. You can see an indication of shadows. It’s not that different. You can make choices. That’s where the art comes in. We know that it will look right when we render. It just isn’t photoreal. 

And is that why the movie looks as if it had been photographed, even though nearly everything except the actor was CG?

As a cameraman, when you design a shot, lighting is part of why you pick an angle — the way the lighting hits a subject. By choosing the camera angle and lenses in previs, the way we placed the [actors playing] animals, it all stacked up to predetermine the beauty of rendering. Weta Digital and MPC did beautiful work on the animals and the environments. And we used a full-on ray tracer that mimics what light would do — all those things that the eye is used to seeing. Before ray tracing, that had to be faked, and it’s hard to keep up the faking. This way, once we pick the right time of day and the right light, we’ve made the artistic choice. It renders exactly like three o'clock in the jungle. And you get happy accidents. You can put the camera down low, tilt it, and get a nice lens flare, then recompose and get stuff you otherwise wouldn’t get.

Tell me about the environment.

The CG jungle was really hard to do. There are millions of things going on. Mosquitoes, dew on the plants, wind. There were so many factors. Audrey Ferrara led a team at MPC that produced the environment in a loving way that was better than I had even hoped. I thought their work and Weta Digital’s was superb. It made a huge contribution to the film.

How did MPC make the talking animals believable — and Weta Digital with the monkeys?

Once we had set up a real environment and put the animals in the right dramatic place to speak like actors, we chose early on that they would speak only as God made them. We wouldn’t make their throat or jaw do anything they wouldn’t normally do. For example, think about a parrot that talks. Even when we speak, sometimes you see lip sync and sometimes you don’t. You can believe these animals talk because it does happen in nature. Andy Jones and the animators studied what nature does and didn’t violate what nature does. They behave only as animals. Nothing is exaggerated. Nothing tells you it’s unnatural. The light is real, the camera movement is real, they’re in the right dramatic spot.

How might “analog previs” — virtual cinematography — influence visual effects and filmmaking in general?

A lot of times I don’t know what I want until I look through a camera, the same as other cameramen and directors. Without these tools, you’re kinda removed. It’s like trying to sculpt by telling someone else how to do it. You don’t get the nuances unless you’re doing every brushstroke, every molding of a sculpture. So this was invented for me to do my work better because that’s the way I like to work, with a live-action mentality. I think to do well you need to recreate the happy accident feeling. I appreciate that sometimes things that are not planned, that come from observing, are greater than I thought, that I didn’t anticipate. 

I’m also excited that this allows someone who has been directing a movie for 20 years to direct artificial elements, that future filmmakers who are not fluent in this can impart their art into a project again and not be removed from it and only get 60 or 70 percent of every shot. That’s what Jim Cameron responded to it when we worked on a system for him.

Do you think this previs process reduces the artistic impact that visual effects artists can have in post-production?

No. It makes them filmmakers. It will empower visual effects artists. Visual effects is one title that incorporates many disciplines — camera, art director, editor, hairdresser, costuming, set construction, lighting, set decoration. So many disciplines are lumped into one and called visual effects. Yeah, it’s visual effects. But look at all the art required to make a scene look like a scene in a movie. 

Right now, VFX artists aren’t cameramen. They don’t look through a viewfinder. They don’t see light. I became a cameraman, an editor, and a director because I have a tool I can practice on. Now, I have my own point of view. It’s my vision. Some visual effects people will become great cameramen, some art directors, some great actors/animators. This will be a bonus, a boon to the industry. They will have the freedom to be the artists. Instead of being short-order cooks waiting for input, they can do it themselves.

Rob Legato

What did you learn from working on The Jungle Book?

I learned to trust my instincts more about filmmaking. I thought we could pull it off and, apparently, we did. I adore the fact that if you feel and think something it kind of magically gets put into the shot and the audience gets what your intention is. It leaches out of you and into the film. That’s what I respond to when I see films I love. You feel something about the personality of the filmmaker on every frame. Something seemingly artificial is not artificial when it fools the eye that it’s real. I learned, yes, that can happen. And now we can do something more.

What are you working on now?

Lion King. I can’t say any more about it than that.