By TREVOR HOGG
By TREVOR HOGG
Images courtesy of 20th Century Studios.
Fulfilling his promise to explore the various biomes and cultures of Pandora with each of the sequels, James Cameron takes to the air and walks in the volcanic wastelands for Avatar: Fire and Ash where Jake Sully and Neytiri are reeling from the death of their son Neteyam and the lunar conflict goes beyond the human-operated Resources Development Administration to include intertribal warfare.
“The evolution in Fire and Ash is dramatic in regard to where the characters go and what confronts them,” notes James Cameron, Producer, Writer and Director. “We don’t have to go out and slay a giant every time we make an Avatar movie. We’re trying to future-proof ourselves. In the meantime, generative AI has come along with all of its threat and promise, so that means everything that we developed and thought we would advertise across the four films we are probably going to reconsider if we do move forward with more sequels. There will probably need to be a mini-evolution or mini-revolution at the very least between where we’re ending Fire and Ash and where we start on Avatar 4 and 5, maybe a year from now.”
“Generative AI has come along with all of its threat and promise, so that means everything that we developed and thought we would advertise across the four films we are probably going to reconsider if we do move forward with more sequels. There will probably need to be a mini-evolution or mini-revolution at the very least between where we’re ending Fire and Ash and where we start on Avatar 4 and 5, maybe a year from now.”
—James Cameron, Writer, Director & Producer
“There is an individual algorithm that gets created for each character, and there is some machine learning that goes into the process as well,” Cameron remarks. “We try to get the animators to be as hands-off as possible when it comes to facial. The animators have to get involved when it comes to muscle tension in the neck, which is a surprising amount of performance. I always say that the face ends at the clavicles because there is so much intensity that can be shown with the tendons in the neck. We get coarse data on the hands, so the hand work is done by keyframe animators to get exactly the finger pressure, bend back, contact, tendon tension and hand rotation, and then you’ve got procedural muscle simulations. Often you go in and do shot-specific sculpting on top of the procedural stuff to get the right muscle tension, tendons firing, the IT band and quads; we look at all of this. There is still a lot of human in the loop work that brings the body fully to life. The data is fine enough coming out of capture that we can see individual breaths, but sometimes that informs neck tendons and body tension. It’s this interesting dance between the capture and the final character that emerges. The characters are running around with not a lot of clothing on, but, of course, we have to have cloth simulations, and that has become a more developed artform than what it was.”
Starting off as concept art directors on Avatar and taking over the role of production designers on the sequels are Ben Procter and Dylan Cole. “We each have a specialty,” Procter explains. “Dylan handles Pandora, everything to do with the look of the moon, flora, fauna and tribal cultural designs. The RDA [Resources Development Administration] and all of the human technology is my realm. There’s a production-related pragmatic difference in that. Because I’m doing human stuff, I have a lot more of the live-action sets, which is not to say Dylan doesn’t have sets. He’s got tons of them. And it doesn’t mean that I don’t have a lot of visual effects-related tasks to oversee as well.” The conceptualization and execution process does not change in the digital realm. “Whether it’s pixels or plaster, it is still production design,” Cole states. “I care about what’s on the screen, not how it got there, because it’s all design and world-building. We build giant proxy sets because as we learned way back on the first Avatar, jungles aren’t flat, so you can’t run around a flat soundstage. We’ll have undulating terrain and logs to jump over and duck under; that’s our simplest stuff. For Fire and Ash, we meet a brand-new culture called the Wind Traders, and they essentially have these giant-gondola-floating-pirate-ship-type things. We built that as a one-to-one proxy set with multi-deck rigging, and it filled a stage.”
Building a world bigger than one movie has always been the goal. Procter remarks, “Knowing I needed to expand the palette of the RDA, I leaned into a concept I call high-performance cruelty. There’s got to be something cool and aspirational about the RDA designs because they’re not just meant to be intimidating or oppressive. I definitely introduced more color, which you find in a lot of heavy-duty maritime industrial vessels from oil rigs to giant ships.” Landscape and culture are entwined. “For the Ash People, we wanted a striking visual difference, both in their culture and landscape, and to suggest why they’re the way they are,” Cole states. “Their home was taken out by a volcano, so they have rejected Eywa and all basic Na’vi principles. The Ash People embrace death and conquering, but at the same time they’re still Na’vi, highly skilled, intelligent crafts people. This makes them much more terrifying, especially Varang. They are a harsh people, so it’s a harsh contrast, black and white. The Wind Traders have a different way of life. They’re nomadic traders and benefit from visiting all of the other cultures. We see music and dancing, not that the other cultures didn’t do that, but it’s always a good time when the Wind Traders come to town.”
There is a major misconception that the Avatar movies are animated films with the characters being voiced by the cast. “The truth is that every character is played by an actor performing their scenes on the motion capture stage,” explains Stephen E. Rivkin, Editor, who received an Oscar nomination for Avatar. “There is even a separate face camera for each actor to record every nuance of their facial performance in order to deliver a true and accurate representation of the actor’s performance in the final CG shot. Even the large crowd scenes have real actors playing the background characters. But we still have to build them in a mosaic because they’re not all captured at the same time. We may capture the principal characters in one pass and then do crowd passes where we fill in secondary characters and sections of crowds captured to a playback of the principal actors. If you imagine it like overdubbing in recording audio, we would have to overdub and add layer upon layer of more characters in a large scene with many CG characters. This is not an animated film. Everything you see is based on an actor’s performance.”
Sometimes, facial performances were replaced. “There may be an instance where Jim wants to change a particular character’s line for story purposes, or have something said in one scene opposed to another,” Rivkin remarks. “We can bring the actor in for what we call FPR [Facial Performance Replacement]. They put the head rig on, and we have them say a different line that would be incorporated into their performance from before.” Avatar: Fire and Ash was more complicated than Avatar: The Way of Water because of the increased number of shots that required the integration of the live-action Spider character with CG cast members. “We did our usual performance capture process capturing Spider as a CG character first, then put together a performance edit using the best takes from everyone, and prepared the files for Jim to shoot the virtual camera scenes in the usual way. The scene was then edited in this template form. This would provide a blueprint for each shot in the edited sequence that Spider needed to be replaced with his live-action element. When shooting the live action, Jim could see the other CG characters [minus the CG Spider] played back in the camera to direct the shoot. We used computer-programmed, repeatable camera moves to match the CG camera moves, but this time with live-action Spider in the shot; that way we could do multiple takes. Ultimately, when finished, there is a seamless blend of live-action and CG characters,” Rivkin says.
Going from Avatar to Avatar: The Way of Water was a major technological leap. “We rebuilt the entire pipeline,” remarks Ryan Champney, Virtual Production Supervisor, who previously was the Simulcam Technical Director for Giant Studios on Avatar. “In the course of making The Way of Water and Fire and Ash, it was about scale. Day one, it was like, ‘Brand new technology. It’s working great. Yeah!’ But after that, it’s, ‘Now add 10 more characters into the scene live. I want to have fire onstage.’ And anyone who understands mocap knows that fire is a huge emission of IR radiation, and everything we’re capturing is infrared. We tackled that problem. Then it was the underwater problem. Every bubble looked like a marker. It was this constant evolution of, ‘How do we tackle these problems?’ There’s no rule book or reference to look for how to do those things. Even on the performance capture stage, Jim wanted to performance capture someone taking a shower, but he wanted the real believability of them under the waterfall. We created a waterfall, but that water reflects all the infrared light and distorts the markers, so we had to create filtering and all sorts of other tools for those pieces.”
Fire is a significant atmospheric that also happens to be a lighting source. “You have to be judicious about how you use it,” observes Richard Baneham, Executive Producer and Visual Effects Supervisor, who won Oscars for Avatar and Avatar: The Way of Water. “For us, it has always been a matter of sculpting with light. If you watch Jim’s movies, traditionally, he lights in layers. When you introduce a chroma, like fire, and that bioluminescence, which is sitting in that cyan range, you get to sculpt characters and images, not unlike a Maxfield Parrish painting.” Careful attention was paid to the life and death cycle of vegetation to ensure the believability of environments. “We have a reservoir of Pandorian fauna and flora. But it goes a little further. We have to understand growth patterning and the relationship of plant to plant, which animals feed on which plants, and what kind of destruction would be there. You’re replicating the idea of how real-world environments come to be.”
Physics had to be properly represented. “You’ll often see in big creature movies where a dragon will go up and down based on a flight,” Baneham remarks. “The truth is that’s not sustainable. Flight works because you ascend, plateau, ascend, plateau or you would never be able to fly up. Then you would glide down and preserve energy. Being able to employ that with all our Ikran [Na’vi for Mountain Banshees] is a way to establish a reality. But we had some interesting challenges on this one, which was finding the appropriate vocabulary for the Medusoids and Windrays, which are essentially a ship and a tugboat. It’s a great symbiotic relationship that the Wind Traders managed to set up. When you start to look at how complex nature is and how hard it is to understand and replicate that; that’s the stuff where we get to play and be like a kid again.”
Wētā FX, which won Oscars for both Avatar and Avatar: The Way of Water, returns once again with ILM contributing some character work on Jake, Quaritch and Neytiri. “What Fire and Ash has allowed us to do is to spend much more time on the creative,” remarks Joe Letteri, VES, Senior Visual Effects Supervisor at Wētā FX. “And a lot less of, ‘Why is this thing not working? How do we fix it? You know, what’s the problem here? Is it fundamental?’ This has been a lot more about, ‘Are we getting through creatively what we need?’ That’s been great to see. Everyone’s feeling that way, and if I could speak for Jim, he’s feeling that way, too. We’ve spent a long time developing this whole process; the idea of performance capture, and evolving that from the stage to underwater, and his method of building a template to work from, that the cut is based on and that our work is based on. This idea of unifying the whole flow, even including the live-action shoot. It’s all interwoven, and it’s all come together to the point right now where we’re able to look at what it needs to be creatively, and spend our time thinking about that.”
“Because we’ve had three movies now, we understand how the Na’vi act, the performances and subtleties that we were learning along the way, like the ears and tails, all the extra things that bring these characters to life,” states Eric Saindon, Visual Effects Supervisor at Wētā FX. “With all of this extra time we’ve had with these characters, we’ve refined that, and the characters come to life so much more.” Tools have been refined from The Way of Water. “One of the things we had on this was not for animation, but for presentation and carrying facial puppets in the scene,” remarks Dan Barrett, Senior Animation Supervisor at Wētā FX. “We had real-time representation of the work that the artists had done. The puppets themselves were working. They were the usual animation puppet that wouldn’t run in real-time, but anything that had been done, we could bank that in our scenes and had real-time for that. It made life a lot easier for animators seeing everyone together in a scene performing.”
Varang is distinguished by her physical, not facial, performance. “It might have been Richard Taylor [Founder, Wētā Workship] who said, ‘The Na’vi that you are used to carry themselves with this upright grace,’” Barrett states. “You look at the way Oona Chaplin embodied Varang and the way that she moves. She’s pelvis forward and sinewy. It’s an incredible thing for an animation team to work with, but also the physicality of her body is quite something.” There is marriage between animation and live-action. “We capture the actors doing these things, but a lot of times because of the proportional differences of the bodies, there are subtle changes,” Letteri notes. “When Oona moves as Varang and you’re putting that on a body that’s got longer legs and torso, it still needs to convey that same sense of movement, such as the head movements relative to the body. All those things have to be looked at and dialed a little bit so you’re not looking at the mathematical translation from one to the other. In the end, you’re looking at the emotional translation.”
“When you think about it, Fire and Ash is 3,500 shots, so it’s all about having scalable processes,” Cameron observes. “It’s about maintaining the highest possible quality at mass production rate; that was a challenge on Avatar, The Way of Water and now on Fire and Ash. How do we keep the quality absolutely top of the line on every single one of 3,500 shots? You need robust workflows and technology to do that.” The production is always on the cutting edge of technology. “Sometimes, the initial tool rollout will almost be like a laboratory experiment,” Champney reveals. “In cases of things like depth comp and our current generation of Simulcam, it literally rolled out four days after we started live action. We weren’t sure it was going to work, and it was literally those last few days where everything came together. Over the course of the years, as we’re filming, those processes become more like a regular tool pipeline. We’re not thinking about holding it together, and don’t need all the engineers down there waiting for it to explode!” There have been a multitude of challenges. “The Way of Water had the water which was a unique challenge,” Baneham notes. “In Fire and Ash, it was representing the full swath and breadth of the world. Our job is to represent the multidimensional world and invite people to come on a journey.
