Session Is Packed to the Gills as Company Makes the Case for the Production and Post Virtues of Its Massive Light-Field Movie Camera

In an NAB session crowded with around 600 attendees, many of them stacked up two or three deep in the aisles, Lytro CEO Jason Rosenthal took the wraps off the Lytro Cinema camera for the first time.

As previously reported, Lytro’s aim is to change the very nature of cinematography, bringing light-field technology to film production. By capturing information about the direction of light, not just its intensity, a light-field camera can be used to acquire a volumetric image of a scene — one that identifies the depth of objects in the camera’s field of view rather than just their position in the frame. That allows the plane of focus, depth of field, shutter angle, frame rate, and more to be dialed in after the fact rather than captured in camera, yielding the same effects as extreme and impossible f-stops in traditional camera hardware. And it’s thought to have special implications for VFX work, eliminating the need for green screens, for example, with what Lytro calls a “depth screen,” which isolates figures from the background computationally rather than visually.

When the covers were lifted from the camera prototype in a corner of the room, the immediate reaction from the crowd  was a mixture of gasps and laughter. The camera is big — it’s about as long as a small car, and has enough girth to fit a human being in the casing lengthwise. (In fact, it resembles a coffin on a tripod, which is maybe too on-the-nose an image to describe the impact Lytro hopes it will have on the traditional motion-picture camera market.)

Rosenthal assured the audience that work is already underway to miniaturize the hardware, but there’s a lot going on inside the camera housing. It holds “many millions of lenses,” which focus the light on what Lytro’s head of light-field video Jon Karafin said was “the highest-resolution video sensor ever designed” with 755 megapixels of resolution and the capability to capture image information at up to 300fps. A second “creative preview lens” allows cinematographers to see their work on set, locking in focus, shutter angle and other decisions as desired via metadata.

The most interesting possibilites from a VFX standpoint emerge from Lytro’s work to integrate the camera’s data with off-the-shelf compositing software, starting with The Foundry’s Nuke. That would allow visual effects artists to work with live-action footage and 3D CG elements using the same toolset for focus, precise camera positioning, relighting and more. “It’s a complete merger between the CD world and the live-action world,” Karafin said.

A short film from director Robert Stromberg (Maleficent) and David Stump, ASC, was designed to highlight the technology. It mainly emphasized some elaborate visual composites that would require arduous rotoscoping work or very careful pre-production planning to be executed using traditional processes. For instance, one shot featured a cloud of confetti that had been composited in from a separate plate without the benefit of green-screen photography. Instead, the hundreds of bits of confetti were computationally isolated and pulled into the shot using the camera’s depth data. In another shot, a couple was captured against the drab background of a parking lot, then isolated using depth data for placement against a different background.

The million-dollar question, of course, is when the camera will be productized, and whether there is a real demand for it in Hollywood. Rosenthal said camera tests will take place through this summer with a target to make the technology “production-ready” later this year or early in 2017 and imagined an eventual transition “from the legacy 2D digital format to the volumetric 3D format.” Stump agreed with him, predicting that in the not-too-distant future his colleagues would wonder, “How did we ever make a movie without these things?”

Notably missing from Rosenthal’s presentation was any announcement of big studio customers (the ultimate audience for a rental-only camera with rates in the hundreds of thousands of dollars), leading some in the room to wonder if Lytro Cinema would find a market in 2017 or run out of funding and end up as vaporware. While the overall mood was positively charged, this camera technology will face a lot of scrutiny in the near future — and Lytro still has a lot to prove.