23 January 2024
Exclusives, Film
By IAN FAILES
By IAN FAILES
A world in which humans and puppets co-exist is the world created in Brian Henson’s The Happytime Murders, an adults-only detective romp that takes puppeteering – and visual effects – to a new level.
Stargate Studios’ Sam Nicholson oversaw VFX on the film as Visual Effects Supervisor, with work ranging from classic rod and puppeteer removal, to intricate compositing of greenscreened puppets, to the inclusion of fully CG puppets on screen.
VFX Voice asked Nicholson how he managed the work for the Jim Henson Company production.
“It’s some of the most complex work that we’ve ever done because, in one shot you could have a combination of principal photography done at 8K and then puppets on greenscreen being puppeteered by four puppeteers for each puppet. So if you have six puppets in a scene, you’ve got 24 puppeteers operating them, and these are all people dressed in green suits.”
—Sam Nicholson, Visual Effects Supervisor, Stargate Studios
VFX Voice: This seems like a very different kind of visual effects assignment. What was involved in it from your end?
Sam Nicholson: This was a big visual effects film. We’re delivering over 750 shots for the show. It’s some of the most complex work that we’ve ever done because, in one shot you could have a combination of principal photography done at 8K and then puppets on greenscreen being puppeteered by four puppeteers for each puppet. So if you have, say, six puppets in a scene, you’ve got 24 puppeteers operating them, and these are all people dressed in green suits. And then you have all the rod removal of those puppets, and then you have avatars or CG puppets that are standing right next to the real puppets.
VFX Voice: What would determine whether a puppet would be achieved as a greenscreen element or as an avatar, or in some other way?
Nicholson: The general rule of thumb was that everything that was possible to shoot waist-up we did practical, but even then we tried to make it easy on the puppeteers. So rather than have a puppeteer lying down on the ground and operating a puppet walking next to a human being, we’d just let them be in frame and get a much better performance that way. That meant we had a lot of puppeteers in green outfits with rods and everything just all over every shot. Of course they had to be removed, and everything had to be reconstructed behind them.
“We had a lot of puppeteers in green outfits with rods and everything just all over every shot. Of course they had to be removed, and everything had to be reconstructed behind them.”
—Sam Nicholson, Visual Effects Supervisor, Stargate Studios
There are a couple of really big shots with 20 or 30 puppets all in the scene at Santa Monica beach in broad daylight with avatars and actors. We shot it all wild on a Technocrane, analyzed that move, then scaled that for motion control, and shot the greenscreen puppets that needed to go into the scene on a separate set using scaled motion control that would match the full-scale scenes. There was a lot of complicated programming that went into that.
VFX Voice: How did you approach scenes that required CG puppets, or avatars, as you call them?
Nicholson: Anytime you see a full-height puppet, it’s generally an avatar. That avatar has to be an exact match to the look and feel and animation of the real puppet itself. You have hair and fabric, and you need to match the way the light absorbs into felt. Skin is actually pretty easy compared to puppet felt!
The close-up avatars were scanned and had a complete texture model done, and others were done with photogrammetry. Some of the puppets on set that had their heads explode or get shot, we’d just do a bunch of photogrammetry of it so that we have a digital duplicate to match to the live action.
“The close-up avatars were scanned and had a complete texture model done, and others were done with photogrammetry. Some of the puppets on set that had their heads explode or get shot, we’d just do a bunch of photogrammetry of it so that we have a digital duplicate to match to the live action.”
—Sam Nicholson, Visual Effects Supervisor, Stargate Studios
We also did some motion capture of the puppets on set with Xsens MVN sensors on the puppets. And we had a puppeteer who was operating the main character, Phil Phillips, wearing a suit so that we could capture his leg movements. I wanted to capture all of Phil’s movements because, keep in mind that the shadow reconstruction is very important here.
Generally, the floors are all built up four feet high on a puppet set because puppets are operated above people’s heads. So even when you do something as simple as walk through a door into a room, the door is split in half so that the puppeteer can articulate his body through the door, but you have only the upper part of the door. You have the upper part of the torso, the puppet grabbing the doorknob, but then everything below the door knob is puppeteers and junk. So sometimes you have three people trying to fit through a doorway at the same time. It’s split, and there is no floor at all, so you have to reconstruct the floor, reconstruct the door, and then take the motion capture that we got on the day and add it and blend it into the puppet.
“Generally, the floors are all built up four feet high on a puppet set because puppets are operated above people’s heads. So even when you do something as simple as walk through a door into a room, the door is split in half so that the puppeteer can articulate his body through the door, but you have only the upper part of the door.”
—Sam Nicholson, Visual Effects Supervisor, Stargate Studios
VFX Voice: Puppets move in a certain way because of the way they’re built – how did you match that with any CG movement?
Nicholson: Yes, one of the interesting things is that, when you think about it, if a puppet is on your wrist or on your arm and you rotate your wrist one way, your wrist only rotates one way for a certain amount. So when a puppet turns its head, depending on which hand the puppeteer’s using, the puppet can look around 180 degrees in one direction, but it might only be 30 degrees in the other direction. So we really had to match that kind of movement.
VFX Voice: What was your toughest shot?
Nicholson: There’s some shots in there with multiple puppets in full daylight and the camera’s moving, and you’ve also got to remember the whole point of the scene is that it needs to be funny. One of the funniest scenes is in the porn shop – the octopus milking a cow has got to be one of the classics of all time. But I mean, you’ve got to make that look spontaneous. Where do you find that in the rule book? I just don’t know.
;
