Filmmakers rarely rely on visual effects to tell humble stories, but the gentleness of the new film version of Charlotte’s Web depended largely on the artists who made the talking live-action and CG animals believable.

Director Gary Winick brought the adaptation of E. B. White’s classic children’s tale to the screen for Paramount Pictures; visual effects supervisor John Berton wrangled the effects crews. Six VFX studios worked on the film, with Rhythm & Hues, Tippett Studio and Rising Sun Pictures taking the lead.

The circle-of-life story in Charlotte’s Web revolves around a young girl, a young pig, a motherly spider, and a bevy of barnyard residents. Rhythm & Hues created lip synch and some facial animation for Wilbur, the pig, and most of the barn animals including Ike the horse, Betsy and Bitsy the cows, and Samuel the Sheep. Tippett Studio created Templeton the rat in CG and also digitally enhanced the mostly live-action crows. Rising Sun got Charlotte, a spider who is always CG.

“We wanted to keep each of the characters in one house so they could develop the personality for the character, understand it, and constantly and consistently put the character on screen,” says Berton. “That caused us to go into the morass of compositing these shots in different places with someone in the center knowing how to take the blocking from one character and send it to another studio. But we decided to bite that bullet to get consistent performances.”

The filmmakers shot dozens of real pigs playing Wilbur, along with other barnyard animals, though not all at the same time. It turns out that real animals aren’t as companionable as in the bucolic children’s tale. “Geese attack sheep,” says Berton. “We previs’d the whole movie to understand the relationships and what the shots were like. Without that previs, we could have been standing on the set going, ‘The geese are going to do what?'” Proof handled the previs. Tippett visual effects supervisor Blair Clark handled the second unit.

“It was so time consuming,” Clark says. “We had to do multiple motion control passes – the cows didn’t like the horse and the geese had to be on their own. And we had big sweeping camera moves into the barn. We had to film the animals in order to get the shadows right.”

Rhythm & Hues, which would make most of the live barnyard animals talk, hadn’t yet signed onto the film during most of the principal photography. “We put in requests for the kind of information we needed, though, while we were bidding,” says Todd Shifflett, visual effects supervisor. “But not getting to go through our usual process for measuring the set and the animals was one of our biggest challenges.”

Usually the studio, which won an Oscar for making Babe talk, takes stereoscopic images of the animals from two cameras simultaneously, front, top and sides. They use the stereoscopic images to build a model in 3D and to get precise measurements.

“One of the most difficult parts of the process is doing the modeling and match move for the facial tracking of the animals. That sets up the foundation for the entire effect,” says Shifflett. “It’s hard to hold a ruler up to an animal, which we also do, of course.”

Because they hadn’t taken 3D stereo images, the crew pored through outtakes looking for two pictures they could sync up to create substitutes. The outtakes proved a valuable resource in other ways, as well. “We like to get footage of the animals looking up, from each side, lighting reference and materials we can use to extract textures from,” says Shifflett. “So we looked for angles that would be useful to us.”

One of the most invisible parts of the talking-animal process is background replacement. Artists must remove an animal’s jaw before adding the talking muzzle because in a profile shot, for example, a remnant of an old muzzle might become visible when the animal opens its mouth. Thus, artists patch in a new background, sometimes using a simple garbage matte, but not always. “The sheep were particularly difficult when they were talking on top of one another because that put another sheep in the background,” Shifflett says, “and sheep wool is a difficult texture to paste back in.”

To do the lip sync, the studio creates the animal’s head and neck in CG to make it easier for match movers to line up the cg animal with the real animal. Animators work with the entire face ‘ eyes, nose, cheeks and jaw ‘ and occasionally even the neck. Rhythm & Hues models in Maya, but otherwise works with proprietary software.

“Essentially, this effect is a complex 3D morph,” Shifflett says. “We line up the animal in the original space, then move it, pushing and pulling the texture around to make it look like the animal is talking. We’ve done a lot of these talking animal projects, so we’re always looking for new things. What I found interesting on this project was that we pushed the technology in ways that allowed animators to make more subtle movements.”

The new technology is more realistic CG hair, fur, and shading models. When Rhythm & Hues worked on Babe, for example, they didn’t have enough rendering power to put thousands of strands of hair on the pig’s face, and subsurface scattering didn’t exist in CG.

“We used tiny little texture-mapped cards for Babe,” says Shifflett, “painted maps for the strands of hair ‘ tiny little sprites. We’d line them up along the edge and then smear in color from the surrounding footage to look like the right color for the animal. Now, we can really color the fur, not just use background texture. We can have an accurate shading model for the hair.”

Shading models such as subsurface scattering, which added luminescence to skin, gums and teeth, also meant the team could more effectively blend the animated CG surfaces into the photography. “One of the most difficult things is to transition from live-action photography used as a texture to the CG mouth, and that happens somewhere along the lip,” says Shifflett. “You wouldn’t see the subtle motion created by the animators if we only used textures from principal photography. When you’re pushing and pulling texture around, you don’t get a lot of shading change.”

The horse provides a good example. To make it look like the lips are moving, the big broad fleshy areas of the mouth must move up and down, but without shading changes on the malleable surfaces it looks fake. “The surface is a mixture of the original background and a CG texture,” says Shifflett. “That allows us to introduce our own CG lighting, which is different from the set lighting, to create the the shading and shadow that should happen when the muzzle changes shape.”

When a muzzle stretch exposed part of an animal’s face not caught in principal photography, artists in the lighting department created a 3D patchwork quilt of textures to fill the spots. “They might have 20 or 30 spots on any shot that they have to paint and blend together,” says Shifflett. To do this, they paint a series of mattes and then reveal textures they know will work through those mattes. The tiny textures come from different frames in the film ‘ the outtakes. “Think of using the little clone tool in Photoshop,” Shifflett says. “It’s like using it on a 3D surface across time.”

Although compositors might have handled this process, the studio decided that, because the lighters focused on lighting the eyes and inside of the mouth, it was better to have them stitch the patches.

Anyway, the compositors had enough to do. They had the finishing touches ‘ the specular highlights for the eyes, the color blending, and more. “It wasn’t as simple as getting this thing from the lighters and comping it on top,” says Shifflett. “They had to maintain the grain of the film. Some of the pieces in the patchwork on the face are individual frames that we locked down or stuck onto the creature for the shot, and some are patches of animating textures. Mapping the textures onto the animal affects the grain structure, so we had to blend those or you’d see floating blobs on the screen. There’s no way to do that automatically.”

Berton believes that the work Rhythm & Hues did for Charlotte’s Web pushed beyond that for talking animals created in the past. “In the past, you’d try to get away without creating fur for the pig, but you can’t, and we didn’t,” says Berton. “The fur rendering on the pig was incredibly important. It was a technical achievement. We had a subtle story to tell, and our characters could tell it in a way that fit the tone of the film. You can’t do that without complex surfaces and people who can manipulate those surfaces.”

Berton believes the most difficult animal in the film, though, might have been Charlotte the spider, who saves Wilbur, the spring pig, from becoming Christmas dinner by spinning words into her web. “At the end of the movie, when she says, ‘The miracle is you,’ in full close-up, she has to be to bring it home,” says Berton. "We worked very hard to make Charlotte perfect for that moment.” Charlotte had to be endearing, but she was still a spider in a live-action film.

That challenge fell to Rising Sun Pictures in Adelaide, Australia. There, visual effects supervisor John Dietz led a team of more than 60 artists who convinced audiences that spiders, or at least this particular spider, could be endearing.

“It’s a dream job to have a title character,” he says. “We had worked with the filmmakers before, and we did a good test that I think convinced them we understood the book and the nature of her character.” Also, taking this role would be an important step for the studio, which had not yet handled a lead character in a film. “The filmmakers knew we wouldn’t let Charlotte not turn out as good as she could be.”

To create Charlotte, Rising Sun used Softimage XSI for the modeling, rigging, and animation of the eight-legged creature, and for grooming her fur. For rendering, they use 3Delight, a RenderMan compliant renderer; for compositing, Shake; for tracking, Boujou with Hype software from Visual Appliance. “It’s a cool piece of software that removes lens distortion so you can work in an undistorted environment,” says Dietz of Hype. “It helps Boujou get better solves on the tracks.” Cinesync software from Rising Sun's sister company allowed them to share QuickTime versions of their dailies over the Internet with Berton once he returned to the States after filming in Australia.

The biggest challenge for Charlotte, Dietz believes, was her design. “At first we tried to make her fluffy, but she had to be real so we moved into more of a lifelike design,” he says. “But photoreal was too much.” To soften her creepy-crawly look, the artists gave her humanistic eyes, moved her secondary eyes above her “brows,” made her face heart-shaped, and developed a smile line from her fang lines. When the camera moves close, she becomes a performer; when the camera moves away, she becomes a spider.

But Charlotte wasn’t the biggest challenge for the crew. Her webs were. The studio created the webs using three different materials, each with different tensile strengths, and proprietary software that works inside XSI. “Nothing off the shelf could take those three materials, turn on dynamics for the combination, and make them behave properly in the wind,” says Dietz. The web also had to react to the spider’s sticky feet. A dramatic sequence, during which the spider first weaves words into a web, is entirely CG.

Fortunately, Charlotte often appeared on camera separately from the other characters, but in a few sequences, she needed to share the spotlight with other animals. “We determined who would be the lead vendor on a sequence-by-sequence basis,” says Dietz. “If Charlotte and Templeton [the rat] worked together, we’d share gray-shaded stuff.”

Tippett Studios created the always-CG rat Templeton. While Clark was on set during principal photography, co-visual effects supervisor Joel Friesch managed the rat work back in Berkeley. As with Charlotte, the challenge was in making a scary animal not so scary.

“He had to look like a real rat because he plays against real animals,” says Friesch, “but when he’s true to real, he could go scary easily.” Moreover, this rat had to act in ways real rats don’t. In one shot, for example, the rat rolls gloriously on his back in buttermilk, which a real rat wouldn’t do, and in another, drinks out of a little wax bottle. To soften the rat’s rodent demeanor, Tippett played him nose down to avoid looking at his big, yellow teeth. They also turned his eyes in a bit and made his “hands” act more squirrel-like than ratty.

Templeton stars in the film’s action scenes ‘ racing through his little tunnel (a practical set) and scampering through the crowds at the county fair. “Our goal was to have it look like a trained rat was precisely hitting his marks,” says animation supervisor Todd Labonte who worked with a team of as many as 18 animators to create walking patterns and behaviors for the rat based on dialog from voice actor Steve Buscemi.

Working in Maya, animators blocked out the performance in low-res, then moved to temp animation for approvals. In low-res, they could toggle a shell to estimate the shape changes once Templeton was furred.

For fur, Tippett uses proprietary software they call “Furocious,” and for rendering, RenderMan. Tippett also animated the two crows by match-moving and tracking the live crows’ beaks.

During one shot, the Tippett animals all appear together: Templeton lures the crows into an arcade and into crashing into a scarecrow. In this shot, the crows were sometimes elements shot in Los Angeles, sometimes CG; Tippett filmed the scarecrows in Berkeley. For feathers, the studio modified its fur tools. “Fortunately, the crows are black,” says Friesch, noting that these shots arrived after they were well into production.

Two other studios, both in Australia, contributed to the visual effects. Fuel International in Sydney created baby spiders and beak replacements for the geese. Digital Pictures: Iloura in Melbourne created Wilbur’s stunt double. “They did something along the lines of 50 shots,” says Berton. “Anytime you ask, ‘How did they get a pig to do that?’, he’s probably a digital pig.”

With so many studios working on the effects, Berton masterminded a novel method for dealing with color-space issues. “We picked a digital image from each scanned sequence as a target,” he says. “As long as the studios could match that picture, I knew everything would match. It didn’t matter if that was the right color; it only mattered that it was the same color.” As a result, a color-correction that worked for one shot worked for all the shots, which sped up the DI process at the end.

For Berton, this project was a labor of love. “Charlotte’s Web was the first book I read that didn’t have mostly pictures. The story of friendship and sacrifice blew me away. It was a big moment for me.”

Sadly, though, the charming film wasn’t chosen to be a contender for the visual effects Oscar. Berton thinks that’s probably fitting. “It reminds me of the scene where the big pig gets the blue ribbon,” he says.