VFX Voice
The definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Subscribe now to the VFX Voice Quarterly Print Edition.

SUBSCRIPTION & SINGLE ISSUES

Full Access to VFXVoice.com website is available FREE for a limited time only. Exclusive web content will soon be available only to VFXVoice subscribers and VES members.


September 04
2018

ISSUE

Web Exclusive

The Puppet Masters Behind THE HAPPYTIME MURDERS

By IAN FAILES

All images copyright © 2018 STXfilms.

A world in which humans and puppets co-exist is the world created in Brian Henson’s The Happytime Murders, an adults-only detective romp that takes puppeteering – and visual effects – to a new level.

Stargate Studios’ Sam Nicholson oversaw VFX on the film as Visual Effects Supervisor, with work ranging from classic rod and puppeteer removal, to intricate compositing of greenscreened puppets, to the inclusion of fully CG puppets on screen.

VFX Voice asked Nicholson how he managed the work for the Jim Henson Company production.

Watch the RESTRICTED trailer for The Happytime Murders.

“It’s some of the most complex work that we’ve ever done because, in one shot you could have a combination of principal photography done at 8K and then puppets on greenscreen being puppeteered by four puppeteers for each puppet. So if you have six puppets in a scene, you’ve got 24 puppeteers operating them, and these are all people dressed in green suits.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

VFX Voice: This seems like a very different kind of visual effects assignment. What was involved in it from your end?

Sam Nicholson: This was a big visual effects film. We’re delivering over 750 shots for the show. It’s some of the most complex work that we’ve ever done because, in one shot you could have a combination of principal photography done at 8K and then puppets on greenscreen being puppeteered by four puppeteers for each puppet. So if you have, say, six puppets in a scene, you’ve got 24 puppeteers operating them, and these are all people dressed in green suits. And then you have all the rod removal of those puppets, and then you have avatars or CG puppets that are standing right next to the real puppets.

VFX Voice: What would determine whether a puppet would be achieved as a greenscreen element or as an avatar, or in some other way?

Nicholson: The general rule of thumb was that everything that was possible to shoot waist-up we did practical, but even then we tried to make it easy on the puppeteers. So rather than have a puppeteer lying down on the ground and operating a puppet walking next to a human being, we’d just let them be in frame and get a much better performance that way. That meant we had a lot of puppeteers in green outfits with rods and everything just all over every shot. Of course they had to be removed, and everything had to be reconstructed behind them.

Greenscreen puppeteers work on a greenscreen stage to provide elements for a street scene.

“We had a lot of puppeteers in green outfits with rods and everything just all over every shot. Of course they had to be removed, and everything had to be reconstructed behind them.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

There are a couple of really big shots with 20 or 30 puppets all in the scene at Santa Monica beach in broad daylight with avatars and actors. We shot it all wild on a Technocrane, analyzed that move, then scaled that for motion control, and shot the greenscreen puppets that needed to go into the scene on a separate set using scaled motion control that would match the full-scale scenes. There was a lot of complicated programming that went into that.

VFX Voice: How did you approach scenes that required CG puppets, or avatars, as you call them?

Nicholson: Anytime you see a full-height puppet, it’s generally an avatar. That avatar has to be an exact match to the look and feel and animation of the real puppet itself. You have hair and fabric, and you need to match the way the light absorbs into felt. Skin is actually pretty easy compared to puppet felt!

The close-up avatars were scanned and had a complete texture model done, and others were done with photogrammetry. Some of the puppets on set that had their heads explode or get shot, we’d just do a bunch of photogrammetry of it so that we have a digital duplicate to match to the live action.

Phil Phillips is puppeteered on the greenscreen set.

“The close-up avatars were scanned and had a complete texture model done, and others were done with photogrammetry. Some of the puppets on set that had their heads explode or get shot, we’d just do a bunch of photogrammetry of it so that we have a digital duplicate to match to the live action.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

We also did some motion capture of the puppets on set with Xsens MVN sensors on the puppets. And we had a puppeteer who was operating the main character, Phil Phillips, wearing a suit so that we could capture his leg movements. I wanted to capture all of Phil’s movements because, keep in mind that the shadow reconstruction is very important here.

Generally, the floors are all built up four feet high on a puppet set because puppets are operated above people’s heads. So even when you do something as simple as walk through a door into a room, the door is split in half so that the puppeteer can articulate his body through the door, but you have only the upper part of the door. You have the upper part of the torso, the puppet grabbing the doorknob, but then everything below the door knob is puppeteers and junk. So sometimes you have three people trying to fit through a doorway at the same time. It’s split, and there is no floor at all, so you have to reconstruct the floor, reconstruct the door, and then take the motion capture that we got on the day and add it and blend it into the puppet.

Puppeteer Bill Barretta (left) and director Brian Henson on the set of The Happytime Murders.

“Generally, the floors are all built up four feet high on a puppet set because puppets are operated above people’s heads. So even when you do something as simple as walk through a door into a room, the door is split in half so that the puppeteer can articulate his body through the door, but you have only the upper part of the door.”

—Sam Nicholson, Visual Effects Supervisor, Stargate Studios

VFX Voice: Puppets move in a certain way because of the way they’re built – how did you match that with any CG movement?

Nicholson: Yes, one of the interesting things is that, when you think about it, if a puppet is on your wrist or on your arm and you rotate your wrist one way, your wrist only rotates one way for a certain amount. So when a puppet turns its head, depending on which hand the puppeteer’s using, the puppet can look around 180 degrees in one direction, but it might only be 30 degrees in the other direction. So we really had to match that kind of movement.

VFX Voice: What was your toughest shot?

Nicholson: There’s some shots in there with multiple puppets in full daylight and the camera’s moving, and you’ve also got to remember the whole point of the scene is that it needs to be funny. One of the funniest scenes is in the porn shop – the octopus milking a cow has got to be one of the classics of all time. But I mean, you’ve got to make that look spontaneous. Where do you find that in the rule book? I just don’t know.

Another greenscreen element featuring Phil Phillips.

Share this post with

Get the Latest VFX News

Receive notifications when WEB EXCLUSIVE or VFX WEEKLY content is posted on VFX Voice

Name *

Email *

Interested groups *
Web Exclusives
VFX Weekly

Most Popular Stories

The Miniature Models of <strong>BLADE RUNNER</strong>
02 October 2017
Exclusives, Film
The Miniature Models of BLADE RUNNER
In 1982, Ridley Scott’s Blade Runner set a distinctive tone for the look and feel of many sci-fi future film noirs to come, taking advantage of stylized production design, art direction and visual effects work.
THE PEARL: THE SUPER ALIEN MODELS OF<strong> VALERIAN</strong>
02 August 2017
Exclusives, Film
THE PEARL: THE SUPER ALIEN MODELS OF VALERIAN
Among the many creatures and aliens showcased in Luc Besson’s Valerian and the City of a Thousand Planets are members of the Pearl, a beautiful...
Converting a Classic: How Stereo D Gave <strong>TERMINATOR 2: JUDGMENT DAY</strong> a 3D Makeover
24 August 2017
Exclusives, Film
Converting a Classic: How Stereo D Gave TERMINATOR 2: JUDGMENT DAY a 3D Makeover
James Cameron loves stereo. He took full advantage of shooting in native 3D on Avatar, and has made his thoughts clear in recent times about the importance of shooting natively in stereo when possible...
2018 – Year of the Rapidly Expanding <b>VFX Moviemaking Vocabulary</b>
03 April 2018
Exclusives, Film
2018 – Year of the Rapidly Expanding VFX Moviemaking Vocabulary
Industry leaders discuss trends and issues.
<b>AVATAR: FLIGHT OF PASSAGE</b>: A Cinematic, Multi-Sensory 3D Experience that Soars
24 June 2018
Exclusives, Film
AVATAR: FLIGHT OF PASSAGE: A Cinematic, Multi-Sensory 3D Experience that Soars
How the sci-fi fantasy gem became a 3D thrill ride.