How Autodesk Flame and Maya Added Another Dimension to the Video Artist's Original Vision

If you've ever been to New York's Standard Hotel, you've seen the work of Marco Brambilla, a New York-based video artist originally from Milan. Civilization, the first in the three-part [Megaplex] video series that continued in 2010 with Evolution [Megaplex] 3D, loops continuously on a screen in the elevator as you ascend or descend to your floor. The final installment in Brambilla's trilogy, Creation [Megaplex] 3D, is a four-minute loop of some 1500 interwoven stereoscopic video elements that Brambilla likens to an orgiastic purgatory, "a spiraling trajectory that begins with a big bang and continues through embryonic inception, idyllic Eden-like bliss and decadences, and culminates in annihilation, only to then interminably re-invent itself."

At a recent preview screening of Creation 3D we attended inside New York's St. Patrick's Old Cathedral, Klieg-lit at its entrance as if for a Hollywood premiere, live music from the dark, outer aisles surrounded the audience. Illuminated only by small round bulbs attached to their folders, the Young New Yorkers' Chorus performed composer Christopher Cerrone's score, an Omen-esque reinterpretation of Prokofiev’s Cinderella Waltz that set the dream-like, vaguely sinister scene inside the Gothic church. Creation 3D is a multi-layered canvas, an animated mashup of well-known and evocative video clips and memes from popular film and culture just like the others in Brambilla's series. This time the clips, signifying different themes in the creation process, swirl slowly into a double-helixed vortex that could also be, depending on your mood, the void, the tunnel of love or the birth experience in reverse. 

Fresh from the experience we caught up with Brambilla's post collaborator Nate Robinson, above, co-founder and creative director at Ntropic in San Francisco, and asked him via a series of emails to detail the evolution of the technical architecture beneath such a complex compositing feat. Ntropic is predominantly an Autodesk shop with 10 Flames in its three offices in San Francisco, Los Angeles and New York. The facility also regularly uses Blackmagic DaVinci Resolve, The Foundry Nuke, Adobe After Effects, Maxon Cinema 4D, Avid Media Composer, Apple Final Cut and Krakatoa, among other packages. With the addition of its newer Color Division two years ago, Ntropic now offers a full 360-degree post pipeline.

StudioDaily: When did you first meet Brambilla?

Nate Robinson: When we first opened up our New York office I was introduced to Marco by our rep there. I was familiar with his work and was excited about the opportunity to collaborate with him. Also this project was uniquely interesting as it was the last one in the trilogy and the first one to be conceived in stereoscopic. I remember our first couple of meetings. It was almost like dating, seeing if there was chemistry and a similar sensibility. There was a little bit of Marco wondering, "Hey, do I trust this guy to realize my vision?" and of me thinking "Wow, this piece is f—ing crazy—can I pull it off?!"

So how did you do it? And more specifically, how did you build your rotoscoping pipeline to manage and animate Brambilla's video clips?

We built our pipeline using [Imagineer Systems] Mocha and [SilhouetteFX] Silhouette. There were simple shapes we made in Flame, but 75% was done in the other packages. First, we used our internal shot-management system called NIM to log and assign tasks to our rotoscoping team. Next, we categorized all our plates into the different vignettes — "Fertilization," "Birth," "Garden," et cetera. I then took all of the assets into [Autodesk] Flame and began a rough layout to establish camera timing, convergence, roll, art direction, and so on. Initially I used still frames of all of the plates in order to construct everything in one giant setup and still have some sort of interaction.

As roto assets became available, I started replacing layers with the newly looped clips. As the setups became more and more complicated, we broke them into sections to allow for interactivity. Once the key iconic characters like Arnold, Julie [Andrews from The Sound of Music], and Beetlejuice were established, I exported the Camera via FBX to [Autodesk] Maya so we could begin constructing geometry and use [Autodesk] Mudbox to paint textures for all the landscapes. We then brought that geometry and the textures back into Flame which immediately aligned because of the shared 3D camera rig. At the end of the day the tools we used were Flame, Maya, Silhouette, Mocha, Mudbox, [Autodesk] Smoke and [Autodesk] Lustre.

Photo courtesy of Ntropic and Marco Brambilla.

How ever did he get the clearance to use so many iconic film clips? Is it because it's art and not commerce?
That's probably more a Marco question, but when I have asked it in the past, yes, it's because it's truly an art piece, and not a commercial or branded project of any kind.

What techniques from your previous work did you employ here? Can you cite one or two specific examples?
Projection-mapping was used extensively in the creation of this piece. While we had done projection-mapping on numerous projects before, using it in a stereoscopic environment and using it to create this 2.5/3D world was new to us. I wasn't sure at the beginning if we could achieve the desired effect without building out a complete 3D world, but we found a great balance between 2.5D and 3D that gave us something much more unique than a pure 3D world would have been. While the project is an enormous technical/creative challenge, the magic was when the art direction all came together. That's when we knew we had something special, and it made us work even harder to realize the project vision.

Marco Brambilla's Creation 3D premiered in lower Manhattan's St. Patrick's Old Cathedral as part of Nuit Blanche New York earlier this month. The screening featured a live performance of Christopher Cerrone's Prokofiev remix by the Young New Yorkers' Chorus. Photo courtesy of Marco Brambilla.

Exactly how did you work with Brambilla as the project evolved?
There was a lot of creative intensity and back-and-forth as we were trying to come up with the design. Once we had the visual language solved, we would work for a week and show dailies to Marco. During the process, I would come to New York and work for a couple days and then he came out to SF for a couple weeks. The nice part about working with Marco is that I had a very clear understanding of what he was looking for. This helped establish a trust and unified sense of purpose that allowed me to contribute in a genuinely creative way.

Beyond the creative exchange, what was the most fascinating aspect of this project for you?
There were so many! I'd say my main point is that this was truly an art project. So much of the artful work we do has a commerce component. But this project really became a living entity that grew into its own and evolved over the course of a year. It still amazes me that we netted out where we did. You work on all these sections and figure out how to make them all work together, and the real effect of the sum of all of its parts is truly spectacular. I also still love the fact that I see something new every time I watch it — and I've seen it a lot!

The most challenging?
The art direction and rendering were the most challenging aspects. I should elaborate a bit here. The two previous pieces traveled visually either vertically, or horizontally. Those x- and y-axis movements allowed your "camera" to leave one section, and move on to the next. In the case of Creation the elements are on for the full four minutes! This added a huge layer of rendering and complexity. It also made the art direction a royal pain. I probably redesigned this piece 20 times! I would move something in one section and it would have a trickle-down, or cascading effect on the elements before and after. So needless to say, every time we would make a big shift in the design it required a major re-work.

What do you most admire about Brambilla's creative vision and/or process?
Nothing! Just kidding. Marco is a very thoughtful person and has an extremely cultured, considered sensibility as an artist. I mean that in the way where he understands deeply the cultural and societal context of his art, and he is able to express himself clearly in his work. His ability to tell stories in a unique and inventive way definitely drew me in. We created a bond around the project very early on and I appreciated the fact that he trusted me enough to help realize his vision. While he had strong opinions about what he was looking for he always listened to my input, which was rewarding.

Are you at liberty to say where the next Creation [Megaplex] 3-D screening will be? At Cannes, perhaps, or at another upcoming film festival?
After the buzz from the last screening, we have been approached by quite a few organizations that have expressed a lot of interest. Stay tuned!