Image courtesy of Peacock.
Image courtesy of Peacock.
While known for creating raunchy, boundary-testing comedies, Seth MacFarlane is equally willing to push technological boundaries to realize his storytelling ambitions. Nowhere is that more apparent than in the Ted franchise, where the fully CG title teddy bear has headlined two feature films and two seasons of TV on Peacock. For the eight-episode sophomore season of Ted, Co-Visual Effects Supervisors Blair Clark (Hellboy) and Hoyte Yeatman (The Fly) faced an eclectic mix of visual effects challenges, including producing a fantasy world for a Dungeons & Dragons-themed episode, recreating a period-appropriate Bill Clinton using machine learning and making Ted fluffier than ever.
“This season we went in a bunch of different directions. There were a lot of effects that went outside of the realm of doing a Ted,” Yeatman notes. “Seth MacFarlane wanted to play mid-1990s Bill Clinton, and the studio had forbidden us from using any AI.” The rights for news videos and presidential portrait photographs were acquired from the William J. Clinton Presidential Library and used to create an accurate ZBrush sculpt, which revealed that the head shapes of the former U.S. president and MacFarlane were entirely different. It became clear that a CG head replacement was needed. Gradient Effects was hired to use the proprietary software program Shapeshifter to create a 3D mesh of MacFarlane’s head, which was then applied to the Clinton rig based on the ZBrush sculpt. The approach was an improvement, but it was challenging to avoid the uncanny valley. Yeatman continues, “Typically, most AI is generated by a text prompt or an image and text prompt, not actually having to deal with photography and a plate where lighting, performance and lensing are established. We found Deep Voodoo in Venice, California, which is a visual effects house that does visual effects with an eye toward AI.”
“The advantage of using AI over CG is that it understands human speech well. This means you can look into the mouth and see the tongue moving. In animation, you can’t capture that, or the little nuances around the eyes and squints; AI does that perfectly. AI does all the heavy lifting that is not seen by the viewer but is felt by them. That’s the last 5% or 10% that you struggle with in traditional CG. AI was another tool in the visual effects arsenal.”
—Hoyte Yeatman, Co-Visual Effects Supervisor, Ted Season 2
Founded by South Park creators Trey Parker and Matt Stone, Deep Voodoo relied on an application that was under development at the time. “They were literally trying to do what we do in CG, which is to lop the head off somewhere through the neck and replace the whole head with CG, including the hair, ears and the back of the head,” Yeatman explains. “You get AI that understands the human face, and in this case, understands speech well. We had to train it to understand what Bill Clinton’s face looks like, but then this model had to study the plate and [account for] Seth’s performance, lensing, camera tracking and lighting. It rendered a 1024 x 1024 window that went over the top of Seth’s head, and that was the replacement part. The advantage of using AI over CG is that it understands human speech well. This means you can look into the mouth and see the tongue moving. In animation, you can’t capture that, or the little nuances around the eyes and squints; AI does that perfectly. AI does all the heavy lifting that is not seen by the viewer but is felt by them. That’s the last 5% or 10% that you struggle with in traditional CG. AI was another tool in the visual effects arsenal. It wasn’t a magic pushbutton that turned it on. This generated element went through a standard visual effects pipeline, was composited, went through several approval processes and the color pipeline like everything else.”
Visual effects worked closely with the art and costume departments for the Dungeons & Dragons quest in Episode 203. “There were certain rules we had to follow with Ted,” Clark shares. “We never put shoes on him because they look like an orthopedic contrivance that’s ridiculous. The art department did a whole Unreal Engine previs of the transition from the big vista of the scene down through the forest to our actors walking on a path. The forest part around the path was built on a stage, and we had bluescreen around that.”
The practical forest build was enormous. “The tree branches were less than eight or nine inches away from the bluescreen, which is typically a big no-no,” Yeatman laughs. “You need at least 36 to 40 inches away so you can have a chance of lighting it. There were over 3,000 linear feet of bluescreen that went 40 feet into the air. You want to have God rays coming through the branches, as that gives it a magical feel. But even with a large stage at Universal, you can’t get a single light source like the sun far enough back to get columnated rays to produce that naturally onstage. We also wanted to add little particulates floating in the air. We LiDAR scanned the set so we had a volumetric representation of where all the plants, trunks and rocks were; that is used in the digital arena where we track the camera in 3D. We can fill that CG area with atmosphere and artificially put the key light in and light that to get the God rays. Those rays were done in layers, and we can sandwich them like a clamshell between the foreground, the actors, the midground and the background.” Digital set extensions carried the forest beyond the practical build, ensuring the environment never revealed the limits of the stage. Prosthetics were used for the demon Dra’hul, who appears in the episode. “We added wings, slimmed down the waist and did a treatment on the skin that was otherworldly or ethereal,” Clark reveals. “Other than that, it was straight makeup. It looked great.”
A fluffy Ted comes out of the dryer in a running gag that would eventually become a popular meme. “That was one of the many fun things that we did with him,” Clark notes. “Framestore in Melbourne did all the Ted work in this season. It was a continuation of a labor of love. The same team has been on the show since the first feature, so they had the recipe for his movements and mannerisms. Coming out of the dryer was the fluffiest he had ever been! That was artistic license. We had some where he was puffed up, but Seth wanted a full sphere of fur.” The slick-talking teddy bear also coughs up furballs. “The success in that was mainly the pantomime,” Clark states. “It wasn’t a struggle, but part of the trick was to keep Ted grounded in subtlety with his mannerisms and not going cartoony. Just go in enough so that people forget he is a CG character and buy him as another member of the cast.”
“Framestore in Melbourne did all the Ted work in this season. It was a continuation of a labor of love. The same team has been on the show since the first feature, so they had the recipe for his movements and mannerisms. Coming out of the dryer was the fluffiest he had ever been! That was artistic license. We had some where he was puffed up, but Seth wanted a full sphere of fur.”
—Blair Clark, Co-Visual Effects Supervisor, Ted Season 2
Ted gets thrown around too. “Most of that was the talent of Scott Grimes being able to throw something and make it feel believable, like a football,” Clark remarks. “We had a stunt stuffie that had detachable arms, legs and head. Scott practiced with the torso [nicknamed ‘the egg’] to get a sense of the volume and mass. Then for the plate where Scott scoops Ted up and throws him, it’s all Scott pantomiming; he nailed it.”
ViewScreen remains crucial for animating Ted in real-time on set. “It was a good reference,” Clark observes. “We integrated it as much as possible with Framestore’s capabilities because it had to be seamlessly worked into their pipeline.”
Beating the stuffie is hard. “Basically, on set, Blair is Ted,” Yeatman explains. “In rehearsals, Blair goes in like one of the other cast members. Seth will direct Blair where Ted needs to be, what he needs to do and the pacing. The cameramen are figuring out what they’re seeing. ViewScreen is helpful for some cameramen to see, but the character may not be moving exactly as we rehearsed. It does nothing for the cast because they’re in front of the camera performing. We did use ViewScreen in the forest as it allowed us to see beyond what we had physically on set. We had an opening shot where the camera moves 24 feet, but it was actually a 64-foot crane in the Unreal Engine animatic that we had. It gives everyone an idea of where we have to be and where the camera has to be pointed. For the sequence in the ceiling of the temple where spikes come down, it was also helpful to see framing – where the set extensions would be and the speed of the spikes. It was more of an environmental use. It’s harder on the character because it’s so dynamic on set. It’s like the difference between jazz and classical music. In classical music, every single note is written down, but here you know the tune and melody and just get in there, and it organically happens. That’s what’s nice about Ted. These guys have been working [together for] so long on the features and episodes that they have it drilled down. That’s why it goes as smoothly as it does because everyone knows how it goes.”
Alongside Framestore, Studio Blackbird, Rising Sun Pictures, Deep Voodoo and Pitch Black contributed to nearly 3,000 visual shots, with extensive environmental work transforming the backlots of Universal Studios Hollywood into Boston. “Seth was keen on the trees,” Clark reveals. “He would be like, ‘Those look like West Coast trees.’ Seth was very specific.”
Maintaining the illusion of 1990s Boston presented its own set of challenges. “Big chunks are driving in a car, which is more of a traditional greenscreen,” Yeatman notes. “We have three cameras running all the time, so we have to deal with multi-camera shoots, which is another part of the challenge of doing visual effects. We used plates from the previous season, and those were difficult because they were actually shot in Boston. As we drove by, there would be a car from the wrong era that needed to be changed out. For the scenes at the mansion, we had eight camera arrays, which gave us complete freedom on the stage. You can point anywhere you want to, and we can make it all work. A video wall would be nice, but it’s a little bit much. For the speed that we’re working, with three cameras blazing, this works quite well. There were some areas where we had snow on the ground, so they added powder paper to those parts of the Universal backlot; however, we have to take care of the whole neighborhood. We had angles looking down the road.”
Over the course of their careers, Yeatman and Clark have witnessed digital technology evolve at a remarkable pace. “Blair and I stay up on the latest greatest [tech] and make sure it works before we put it into practice,” Yeatman notes. “We want to give Seth the greatest amount of freedom that he can have and still achieve a good look. Sometimes we do stuff that looks crazy. But with the technology that we have in our back pocket, we can pull it out, make some shots that look cool and make the day, which is probably the biggest thing that we’re all working on.”
