VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


June 04
2019

ISSUE

Web Exclusive

How Framestore Fleshed Out ‘Smart Hulk’ for AVENGERS: ENDGAME

By TREVOR HOGG

Machine-learning software was trained to capture micro-moments that made for a more believable facial performance. Front is Chris Evans, who plays Steve Rogers/Captain America. (Images courtesy of Framestore and Walt Disney Pictures/Marvel Studios.)

Along with ILM, Framestore was responsible for creating Smart Hulk, a peaceful coexistence between scientist Bruce Banner and his volatile, green alter ego that makes his first appearance in Avengers: Endgame.

“ILM started developing Smart Hulk probably during Infinity War,” states Framestore Visual Effects Supervisor Stuart Penn. “Before the beginning of Endgame they turned over the base model, textures and some of the face shapes. From that, we developed our own muscle system, added to the face shapes, built our own face rig, and were provided reference by the client of turntables and facial workouts that ILM had done to make sure our facial performances were matching.”

Stuart Penn, VFX Supervisor, Framestore. 

Smart Hulk was the most complex facial rig created by Framestore. The two key areas to translate were the movement of Mark Ruffalo’s mouth and the way his eyes moved. 

 

“It was down to the level where [Mark Ruffalo] has specific muscle structures in his face, and it was translating those movements into Hulk. The two key areas were the movement of his mouth and the way his eyes move. It was the essence of Mark’s performance coming through on quite a different face.”

—Stuart Penn, Visual Effects Supervisor, Framestore

Machine-learning software was trained to achieve certain keyframes in specific shots.  “That allowed us to generate a quick first-pass animation for the entire sequence,” explains Penn. “We would quickly render it out and send it back to the client so they had something quickly for their edit. We then moved to keyframe animation on the facial stuff to get what we weren’t getting from the solves. The solves also gave us a micro-movement pass that we could extract from the machine-learning data.”

Because Smart Hulk is a hybrid of Hulk and Bruce Banner, there was a heavier reliance on the performance capture of Mark Ruffalo. “It was down to the level where he has specific muscle structures in his face, and it was translating those movements into Hulk. The two key areas were the movement of his mouth and the way his eyes move. It was the essence of Mark’s performance coming through on quite a different face.”

Smart Hulk is slightly less green in Endgame, as the desire was to have a more human subsurface skin coming through underneath. At right is Scarlett Johansson, who plays Natasha Romanoff/Black Widow, and left is Chris Evans.

“It was the most complex facial rig that we’ve ever done. It had many more shapes and additional controllers for the animators to get that fine level of performance, and then all of the dynamic stuff and machine-learning micro-movement that we then plugged in on top.”

—Stuart Penn, Visual Effects Supervisor, Framestore

“It was the most complex facial rig that we’ve ever done,” continues Penn. “It had many more shapes and additional controllers for the animators to get that fine level of performance, and then all of the dynamic stuff and machine-learning micro-movement that we then plugged in on top. The body rig was similar to other rigs that we’ve done before, but we didn’t have a full, working muscle system in there, which we needed.  Marvel was keen that he’s muscly and quite bulky, so when he moves and flexes his arms you actually see those muscles moving and sliding underneath the costume. We had to overdo the muscles to read them through the cloth of the costumes.”

Smart Hulk needed to be able to convey a wide range of emotions. “The hardest thing was that he was doing comedy performances quite a bit. Comedy is all about timing and subtle changes in expressions. We would always go back to watch what Mark did to see what it was about his performance told you what his emotional state was.”

A more finesse approach was adopted for the animation of Smart Hulk since he is capable of graceful and delicate movements. At right is Chris Evans.

Smart Hulk (Mark Ruffalo) was rigged to believably handle a wide range of emotions, including comedy, like in this moment with Chris Evans.

“Marvel was keen that he’s muscly and quite bulky, so when he moves and flexes his arms you actually see those muscles moving and sliding underneath the costume. We had to overdo the muscles to read them through the cloth of the costumes.”

—Stuart Penn, Visual Effects Supervisor, Framestore

Unlike his unruly predecessor, Smart Hulk is capable of graceful and delicate movements. “You wouldn’t have caught him picking up a pencil and poking the keyboards, but now he is,” states Penn. “These are precise actions, rather than Hulk smashing, grabbing a car and throwing it. It was a different level of animation. It was much more finessed.”

Having Smart Hulk interacting with the console was straightforward. “We found and filmed people with chunky hands to see the way all of the muscles and tendons moved and simulated that on top as well. We had him eating ice cream as well. We were trying to work out what sort of instrument does he use? Does he use a big or little spoon? His hands are disproportionately big to his mouth. It was tricky finding the right size spoon for him to hold in his fingers without it looking uncomfortable.

The entire hangar environment was built in CG along with the exterior view of the Avengers’ compound. 

“You wouldn’t have caught him picking up a pencil and poking the keyboards, but now he is. These are precise actions, rather than Hulk smashing, grabbing a car and throwing it. It was a different level of animation. It was much more finessed.”

—Stuart Penn, Visual Effects Supervisor, Framestore

The time-travel suits were a digital creation. 

“Our animators love to perform and filmed each other acting out things so they could then use that as reference beyond what we had,” adds Penn. “We got the designs of the costumes and built real copies of them. We dressed up the biggest and chunkiest in the office and filmed them to give us reference as to how the costume might move. Mark isn’t anywhere as heavy as Hulk and doesn’t move in the same way. His footsteps are nowhere near as big as Hulk, so it’s a definite interpretation of Mark’s body performance into something much bigger. Trying to keep it alive and weighty was definitely a challenge.”

Smart Hulk is slightly less green in Endgame, according to Penn. “There was a desire that we feel the human subsurface skin coming through underneath. We put a pinkish, subsurface glow underneath. We pushed that quite a lot, especially in the cheek area and around the ears, to give him more of a human, fleshy surface underneath the green that’s on top.”


Share this post with

Get the Latest VFX News

Receive notifications when WEB EXCLUSIVE content is posted on VFXVoice.com

Select List(s)*

Most Popular Stories

The Miniature Models of <strong>BLADE RUNNER</strong>
02 October 2017
Exclusives, Film
The Miniature Models of BLADE RUNNER
In 1982, Ridley Scott’s Blade Runner set a distinctive tone for the look and feel of many sci-fi future film noirs to come, taking advantage of stylized production design, art direction and visual effects work.
The New <strong>Artificial Intelligence</strong> Frontier of VFX
20 March 2019
Exclusives, Film
The New Artificial Intelligence Frontier of VFX
The new wave of smart VFX software solutions utilizing A.I.
THE PEARL: THE SUPER ALIEN MODELS OF<strong> VALERIAN</strong>
02 August 2017
Exclusives, Film
THE PEARL: THE SUPER ALIEN MODELS OF VALERIAN
Among the many creatures and aliens showcased in Luc Besson’s Valerian and the City of a Thousand Planets are members of the Pearl, a beautiful...
Converting a Classic: How Stereo D Gave <strong>TERMINATOR 2: JUDGMENT DAY</strong> a 3D Makeover
24 August 2017
Exclusives, Film
Converting a Classic: How Stereo D Gave TERMINATOR 2: JUDGMENT DAY a 3D Makeover
James Cameron loves stereo. He took full advantage of shooting in native 3D on Avatar, and has made his thoughts clear in recent times about the importance of shooting natively in stereo when possible...
2018 – Year of the Rapidly Expanding <b>VFX Moviemaking Vocabulary</b>
03 April 2018
Exclusives, Film
2018 – Year of the Rapidly Expanding VFX Moviemaking Vocabulary
Industry leaders discuss trends and issues.