By IAN FAILES
The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.
Winner of two prestigious 2018 Folio Awards for excellence in publishing.
By IAN FAILES
You may have noticed that 2019 has been an epic year in terms of the release of Disney films based on 2D animated classics. The Lion King, Aladdin and Dumbo are among the major films from the studio that have brought the original animated films into a new dimension, either as live-action or fully photoreal features.
With more such films in the works from Disney including Maleficent: Mistress of Evil, Lady and the Tramp and Cruella – all of which likely have a large CG component – visual effects studios are being called upon to help translate what had only been seen previously as ‘flat’ animation into enriched characters and worlds.
So, what does that involve, exactly? What are the challenges of taking often highly expressive 2D animated characters into the photoreal 3D world? Do VFX studios need to specifically retain certain parts of the 2D performance and the film? VFX Voice asked artists from The Lion King, Aladdin and Dumbo about how the translation process worked from their point of view.
One of the first questions faced by visual effects teams on these 2D to ‘live-action’ films is, ‘How will the main characters be realized?’ Visual effects studios regularly work from production designs, concept art and filmmaker notes. It’s a collaborative effort to bring these characters to the screen, and in the case of characters such as Simba, Genie or Dumbo, audiences also know them very well. That means that any major departures from the original have to be thought through significantly.
In the case of 2019 Aladdin’s Genie (voiced by Will Smith, who also drove the performance of the character via facial capture in the Guy Ritchie film), ILM was tasked with crafting the character in his blue form completely in CG. A predominant trait of the 2D character in the original film from 1992 was that Genie (voiced by Robin Williams) often changed forms.
“In 2D animation, one of the easiest things to do is change a shape,” notes ILM Animation Supervisor Steve Aplin, who says in 3D it is usually harder to go ‘off-model’ and do so many shape changes. “The way we approached it in the new movie was not so much shape changing, but costume changing. We did do experiments with stretching limbs, but we found we came away from the physicality, which you need for a live-action film. We couldn’t quite push Genie around and deform him as much, although we do it in some places.”
Similarly, the character of Abu, a capuchin monkey, is a character with extreme expressions and movement in the original Aladdin. But in the new film, ILM felt that Abu had to appear as much more photoreal, so that the audience would ‘buy it’ and not actually question whether it was a real capuchin.
“The way we approached it in the new [Aladdin] movie was not so much shape changing, but costume changing. We did do experiments with stretching limbs, but we found we came away from the physicality, which you need for a live-action film. We couldn’t quite push Genie around and deform him as much, although we did do it in some places.”
—Steve Aplin, Animation Supervisor, ILM
“Abu started off probably a bit more human, a bit more character-ful,” states ILM Visual Effects Supervisor Mike Mulholland. “And then he gradually steered toward some rules that we established, which were basically that every performance, every action, needed to be based on a real-world action. We found we couldn’t replicate what was in the 2D animation, and so then we had to draw on other sources, like real-world footage, to get something that was believable within our new environment for the film. A lot of it ended up being about finding the right reference and beginning to stick a performance together using real monkey footage as a guide.”
A similar challenge lay ahead for the visual effects team from MPC on Tim Burton’s Dumbo, a re-imagining of the 1941 film. “You’ve got this highly emotive 2D cartoon animation that you have to re-realize in a photorealistic way, using the technology we have,” discusses Production Supervisor Richard Stammers, who also hails from MPC. “Once you start taking that hugely expressive 2D animation into the photoreal world, you’re suddenly becoming a lot more limited in the range of expressions you have. So getting to that point where you can actually still capture the feeling of the original cartoon can be hard.”
For MPC, that ultimately meant that their baby elephant was – compared to a real elephant – quite different. He remains in the new film a caricature, but, as Stammers notes, “feeling as real as possible.” The CG Dumbo retained a cartoon-quality cute factor and was somewhat unusually proportioned. To alleviate the mis-match between that and the real world, the filmmakers also crafted a slightly expressionistic world for him to live in. “I think that was an important thing, trying to get the film to have this sort of coherent look, while still being photoreal, but not hyper-real,” adds Stammers.
The cute factor – partly achieved with an oversized head and large eyes – initially wasn’t something MPC was going to replicate, but after looking to real baby elephants, which are incredibly wrinkly and hairy, they noted that simply creating a photoreal baby elephant was not going to work. Still, the team did visit zoos and collect countless amounts of reference for Dumbo, but then pulled back on the detail in their CG model.
“Tim wanted more of an idealized elephant,” remarks MPC Visual Effects Supervisor Patrick Ledda. “So, we essentially ended up creating texture maps almost from scratch, painted by hand, where we removed so much of the real, organic detail to create this idealized version. That made it very difficult – we had to look at the micro-detail of their skin to try and get something photoreal, but without having all of these other imperfections that real elephants have.”
MPC was also behind the characters in The Lion King, and again faced the challenge of translating what were highly expressive characters in the 1994 animated feature to photoreal versions for Jon Favreau’s film. The director and MPC had previously tackled The Jungle Book, which had some similar challenges. For the new Lion King, however, Favreau was “determined that we were even more documentary in feeling than The Jungle Book,” relates Visual Effects Supervisor Adam Valdez from MPC. “He wanted to find a balance in an even more naturalistic way.”
The difficult part of striking that balance came when the characters needed to launch into musical numbers in the Favreau film, since singing is obviously something real animals do not do. In the 2D animated film, the musical numbers were incredibly theatrical. This wasn’t an option for the new Lion King. “What we had to really do,” says Valdez, “was, in everything from camera work and lighting, to the design of the world and set design, just make sure we had understated realism, but also really carefully composed images. We had to allow for light to be dramatic, and allow for the animals to be emotive so that you never felt emotionally disconnected.
“So,” adds Valdez, “when the film rose up into a musical passage, it didn’t feel like you were literally cutting from something like a documentary to a Broadway feeling, or something extreme. It’s kind of a delicate tonal balance that starts with acting, I would say, and then goes on to set design and lighting and then ultimately music.”
The live-action re-imagining of these 2D animated films do not exactly replicate the originals in terms of characters or shot design, but there are times when similarities occur or are necessary. For instance, the magic carpet in the original Aladdin and in the new film are relatively similar. That’s because, suggests ILM’s Aplin, “you’re already stretching the boundaries of reality so much with that carpet. I mean, it’s a living, breathing carpet, so I don’t think audiences are going to judge whether, ‘Oh, does it look like a real carpet, could a carpet do that?’ It’s already out of the box, it’s not real as soon as it starts moving. It works so well in the 2D animated film, so we thought if we could get some of that character into our CG version, then we were going to try to hold on to it.”
For Dumbo, MPC’s Stammers notes there was not a conscious decision from the filmmakers to craft scenes exactly like the original. But he does point to a scene from the 1941 cartoon where Dumbo visits his mother when she is locked up in the train carriage, where the design and the compositions were made to be similar to those shots. Other than that, animators at MPC often referred to the cartoon for how to pose a baby elephant.
“I’d say to the animators,” recalls Stammers, “‘here’s what the cartoon did, and here’s the nearest thing to the reality of a baby elephant doing something similarly cute. When we had the model of Dumbo as a sculpture that Tim Burton liked, the first thing we did with that was to start posing it, and using the cartoon as reference.”
Meanwhile, Valdez identifies the gorge scene in the new Lion King, when Mufasa saves Simba, as a key one that saw the filmmakers make significant reference to the original. “It’s a good example of where the animated film is a masterpiece,” Valdez says. “It’s really about rapid cutting and it’s got some of my favorite 2D squash-and-stretch moments in it, if you look at the frames where Mufasa gets hit and tumbles to the ground. He just stretches three times his length, it’s incredible! But half the things in it we couldn’t do because they’re just physically implausible. The way he climbs the cliff, and the way he leaps from nowhere out of a running herd of wildebeest. Some of that ‘super duper’ stuff we don’t do because it breaks the believability somehow.
[“For the new Lion King] what we had to really do was, in everything from camera work and lighting, to the design of the world and set design, just make sure we had understated realism, but also really carefully composed images. We had to allow for light to be dramatic, and allow for the animals to be emotive so that you never felt emotionally disconnected.”
—Adam Valdez, Visual Effects Supervisor, MPC
“We had to say, ‘What’s our movie version of that?’ Jon Favreau and the Animation Supervisor Andy Jones broke it down beat by beat, and they really worked on it for many months, analyzing everything. They’d look back at the original film, saying, ‘Could we do that? Should we do that?’ It was sometimes yes, sometimes no – a very, very iterative process. And that original film is loved by so many people. When you’re constantly concerned about how people are going to take your choices compared to something they love, it’s definitely extra work. A lot of extra work.”
While Jon Favreau’s The Lion King utilized a raft of the latest filmmaking and visual effects techniques – including virtual cinematography, VR scouting and real-time rendering tools – in order to be made, one of MPC’s major challenges in ‘selling’ the photoreal characters to the audience was allowing viewers to relate to them even though they were not smiling, frowning or otherwise behaving as they had done in the 1994 2D animated film. Luckily, MPC already had some experience with this for The Jungle Book.
“The basic trick we learned on that previous film,” says MPC Visual Effects Supervisor Adam Valdez, “was that people project emotions onto inanimate objects – the weather, animals, other people, all kinds of stuff. It’s something we naturally do, so it’s not hard to believe an animal is having feelings when you see it doing things that you perceive and project onto it as either sad or happy, lazy, slow, tired or hungry. Those basic physical and mental states are really easy to project onto animals, and you just have to find the things that make you connect for whatever intention you have.”
Valdez equates that with the many YouTube videos out there of cats or dogs that appear to look guilty when they’ve done something wrong. “You project all the human emotion onto those things,” he says. “That means we as VFX artists can really look through lots of reference, and study our subject, and observe the moments that we think look like focus, hunger, danger, or when an animal is giving you the blank stare of a predator and they look scary. Then we think we might know how an audience might react, and we use it.
“It’s kind of like reverse engineered method acting,” suggests Valdez. “And you realize that a little bit of brow movement just right, from a completely human point of view, with the same animation that you would use on a human – is something that you can borrow in some cases. Then it’s a matter of the animation department trying it on and tuning scenes until it just kind of feels right. You’re using the way that humans project how they must be feeling onto other living things to get the emotion.”