VFX Supervisor Everett Burrell and Weta Digital on Animating the Show's Talking Chimp Character

There’s a new superhero family streaming through the airwaves, and they are unique.

Developed for Netflix by Steve Blackman and Jeremy Slater, who wrote or co-wrote many of episodes, Dark Horse Entertainment’s superhero series The Umbrella Academy opens with a concert violinist playing music from Andrew Lloyd Webber’s Phantom of the Opera. The violinist is Number 7, aka Vanya (Ellen Page), the only one of seven siblings with, apparently, no superpowers. Based on the Dark Horse Comics series of the same name by Gerard Way and Gabriel Bá, The Umbrella Academy centers on these seven siblings, all born October 1, 1989, when 43 not-pregnant women went into labor and gave birth simultaneously.

An eccentric billionaire, Sir Reginald Hargreeves (Colm Feore), adopted seven of the children, gave each a number, and created a team he named the Umbrella Academy. The kids wore prep school uniforms and Batman masks, fought crime, and became famous — until they became estranged and scattered. In episode one, Hargreaves dies and the six living siblings return home, still estranged. Remaining at home are their “mother” (an android) and Dr. Pogo, a talking chimpanzee who was Hargreaves’ butler. Sibling Number Five (Aidan Gallagher), who has time-traveled into the near future, provides fuel for the story by warning that an apocalypse will come in five days unless the siblings use their superpowers to stop it.

Season one takes place during eight days, perfect for binging: Netflix released all 10 episodes on February 15 and, during its first month on air, The Umbrella Academy captured the attention of 45 million households. In April, Netflix announced it would renew the show for a second season.

Roll Call: The Show’s VFX Roster

Supervising the visual effects for season one was associate producer Everett Burrell, who will return for season two. Burrell had previously worked with Blackman on the series Altered Carbon. In addition to acting as executive producer on Altered Carbon, Blackman was a co-executive producer and writer on Fargo, for which he received an Emmy nomination. Blackman is also onboard for The Umbrella Academy’s second season.

For season one, visual effects artists at the studios Weta Digital, Spin VFX, Folks VFX, Method, Marz, Deluxe Toronto, Digital Film Tree, and Lola provided the superpower effects and digital environments. Toronto and Hamilton Ontario provided locations.

Spin tackled the apocalypse, changed winter footage to spring with help from Folks, did set extensions, created a teleporting effect, and masterminded Ben, sibling Number 6, who is dead.

“We began the series in winter and then, in spring, the trees bloomed,” Burrell says. “But the story happens in eight days, so we had to add trees and leaves. Ben was harder. Originally, he was never to be fully seen; he was behind frosted glass. Then Steve wanted him to materialize subtly. We gave that mission to Spin.”

Marz created establishing shots of the city and the Grace (android mother) charging effect. Method broke off a chunk of the moon and hurled it to Earth to crash and destroy. Digital Film Tree, Marz, and Deluxe handled 2D compositing, with Deluxe doing 3D comps plus fix-its and cleanup. Weta Digital artists created Dr. Pogo, the talking chimpanzee who is Hargreaves’ butler.

“The first problem I needed to tackle was Dr. Pogo,” Burrell. “Would it be practical? CG? A hybrid? A friend at Fox said, ‘Weta has solved all the problems you have encountered.’ So I gave them a call.”

Calling the Ape Experts

Weta Digital, known for its high-end feature film work, including the digital chimpanzees and other apes starring in the Planet of the Apes films, agreed to take on the television project.

“I think they were excited about creating a supporting character that had to blend with the cast,” Burrell says. “But we had to be very prudent about how we used Pogo. He’d enter a room, talk for a bit, and exit. He couldn’t hang out with the family all the time. We needed to keep the shot count down.”

Dr. Pogo in The Umbrella Academy
Netflix

Creature designer Miles Teves provided the first Dr. Pogo drawings and then collaborated with Weta Digital on details and modifications needed for a talking ape.

“I felt that Dr. Pogo could utilize that tactile feeling from traditional artistry,” Burrell says. “I could feel the character in the drawing.”

In the drawing, Dr. Pogo has one hand in the pocket of his pants; his other hand holds a pipe. He wears a shirt, tie, vest, and a long loose jacket. Teves later sculpted a full-sized Pogo head that was used on set.

“We’d wheel that Pogo [head] out after every take, to help Weta see how to light his hair and skin and to help the actors and crew,” Burrell says. “We didn’t want a tennis ball on a stick.”

Sitting in on Set

On set, actor Ken Hall played Pogo, saying the character’s lines as he interacted with the other actors. Later, Adam Godley provided the voiceover and facial expressions.

“Ken [Hall] was the right height, around four foot six, but he doesn’t have the posh British accent,” Burrell says. “So Adam [Godley] came in.”

In the end, the artists at Weta Digital based Dr. Pogo’s performance more on Godley than Hall.

Ken Hall playing Pogo on set. Inset: video reference of Adam Godley’s matching ADR performance.
© Netflix 2019

“It was good to have Ken Hall on set working with the other actors and for his eyeline and interactions,” says Chris White, visual effects supervisor at Weta Digital. “He was the right height. So we built Pogo to match him, and Pogo fit right in. Everything lined up perfectly.”

To achieve that perfect alignment, the artists first built a digital version of Hall and rendered that 3D model into the scene.

Courtesy Weta Digital
© Netflix 2019

“We first made sure Ken was integrated and that the lighting worked well before we replaced him with Pogo,” White says. “Because we were working on the next episode as we were finishing the previous one, we needed to be efficient. When the first plates came in, we could put our 3D Ken in for a rough match and have lighting start. Our on-set team would capture the lighting on set with HDRI. We also had reference photography and lidar scans, so we knew where the lights were in the scene. We replaced the digital model of Ken with Pogo where we could begin beauty lighting [configuring the lights to favor the subtle details of Pogo’s face shape, skin quality, and expressions]. Once Pogo’s animation was final, the lights could be used with his final animation. We built Pogo to the same level of scrutiny as [our characters] in feature films.”

But because Dr. Pogo stands more upright than the apes Weta had previously created, modelers weren’t able to use typical ape anatomy for the character.

“We needed to pull the hip position up and give him longer legs,“ White says. “This gave him a slightly more human proportion, which was necessary for his refined movement.”

Keyframing Instead of Performance Capture

Although Weta Digital artists had famously used motion-capture to help animators apply performances from human actors to digital Caesar and other CG characters in the Apes films, they chose not to capture performances by Hall or Godley.

“When we looked at Adam’s performance, we concluded we could recreate it through intricate keyframing,” White says. “Motion-capture is valuable, but this case was different. We weren’t capturing an actor performing on set; Adam was in a studio in a constrained space.”

Moreover, animators needed to marry Pogo’s body with Godley’s head movements, so motion-capture of Hall’s on-set performance wouldn’t have helped. The crew did, however, have Godley wear a motion-capture cap so when he tilted his head they could have a true orientation. And to give the animators data to work with, they motion-captured an actor hired to create a performance based on Hall’s performance that incorporated Godley’s movements.

Courtesy Weta Digital
© Netflix 2019

“Hall influenced Godley’s performance, and then the motion-capture actor looked at both,” White says. “He did a great job. Pogo is older now. Although toward the end he runs a bit, he doesn’t move very fast. Mostly, he gives advice that helps the story progress.”

Because Dr. Pogo has a good amount of dialog, riggers made sure his facial system was up to the task for the animators. Once animated, the creature simulation department ran a muzzle simulation that softened the surface and added a little jiggle as the character spoke.

“Apes don’t have the folds around their mouths that we have, so they need a slightly different characteristic,” White says. “Also, they use their hands differently — the finger proportions are different. They don’t grab tools with their thumbs the same way we do. Having worked on several ape projects over the years, we’ve learned a bit about their gestures.”

Suit Up

Technically, the most interesting thing about Dr. Pogo is his costume. The digital costume team had to match practical clothing created by the series’ costume department. To accomplish that in the past, the team typically used textures to represent fabric weaves. For Dr. Pogo, though, they created the fabric with procedural threads. Weta Digital has developed a physically-based cloth model and the talking chimpanzee is one of the first to wear clothes made with the sophisticated system.

The weavers for this digital cloth began by putting fabric samples from the costume department under a microscope and looking at it with a magnifying glass. They wanted to determine what base fibers made the cloth. Then, they defined the weave pattern.

“We sculpted the fabric by weaving the fibers into a particular type of material,” White says. “As the fibers replicate across the surface, the fabric grows. We could create cotton, canvas, any type of fabric and it would render correctly.”

Creating cloth in this way means that each fabric woven with the procedural yarn can have different properties and can react differently to light. Procedurally woven burlap and silk, for example, will reflect and absorb light differently. It also means a piece of fabric can have an infinite amount of detail since it is no longer limited by texture resolution. Artists can also paint variations such as stitching into the fabric.

“That sounds similar to a texture solution, but because we have something making each thread procedurally at render time, we can still zoom in and see every little thread,” White says. “We’d photograph the fabric from the costume department and then render our CG fabric under different lighting conditions to make sure they matched.”

To create clothes from the digitally woven fabric, Weta’s tailors worked from patterns laid out flat. Once cut and stitched into costumes, modelers would add peach fuzz and piling to the fabric.

Dr. Pogo in The Umbrella Academy
Netflix

“In scenes where Pogo is backlit, you can see light hitting the fuzz along his edge,” White says. “Otherwise you’d wonder why he doesn’t fit with the other characters. You can tell if it isn’t there. Bad Ape [in War for the Planet of the Apes] had more hair in his beanie than over his whole body.”

For the hair and fur on Dr. Pogo’s body, the Weta Digital artists used evolutions of systems implemented for previous feature films. Throughout the series we see Dr. Pogo at different ages, so groomers started with the oldest version of his hair and fur and then reduced the amount of salt to create a younger character with more pepper in his hair.

To render Dr. Pogo, the Weta Digital team relied on their proprietary software, Manuka, a spectral renderer. Rather than restricting values to red, green, and blue values, Manuka takes into account all wavelengths of visible light to produce imagery, thus better reproducing the lighting on set.

Ready to Go Bigger

In season one, Dr. Pogo was arguably the most difficult effect. That’s not likely to be true for season two.

“My instinct initially was to go more Marvel superhero excitement, to go bigger,” Burrell says. “But Steve wanted a slow burn for season one as the family united. Now that they have come together, we’ll see what’s going to happen to them. I think because we wished for it, the time has come to have more dynamic stuff. We have to have all the things we loved about the first one and go bigger, to outdo ourselves. But we’ll still have to live by the same rules — the same budget and time.”

And that will be the next challenge for the superhero visual effects teams.