By TREVOR HOGG
Images courtesy of Netflix.
By TREVOR HOGG
Images courtesy of Netflix.
Going from the political intrigue of Westeros to an impending extraterrestrial invasion of Earth to a presidential assassination are Executive Producers David Benioff and D.B. Weiss, who partnered with Creator and Showrunner Mike Makowsky for the Netflix production of Death by Lightning, which recounts the murder of American President James Garfield by the delusional Charles J. Guiteau. Not as fantastical as Game of Thrones and 3 Body Problem, the four-episode historical drama miniseries has some stunning environmental work, especially the 1880s Republican National Convention held in Chicago at the Interstate Exposition Building, the warehouse of the Army Medical Museum in Washington, D.C. and virtual production train shots.
“We set up the convention hall in Unreal Engine and toured it with a virtual reality headset. Rainer Gombos was able to set it up in a way where we could point a laser pointer to a piece of the environment, and you’d be instantly transported there, turn around and be able to see the vast scope of the space. There were no such things as LED volumes, so we didn’t have the benefit of putting our ship in front of a LED wall with our background projected as we did with the train.”
—Steve Kullback, Visual Effects Producer
“Right in the beginning we didn’t think about doing full city shots on the backlot,” states Rainer Gombos, Visual Effects Supervisor. “That came later on in production. It was about choosing the right locations that would allow for the easiest set extensions and worked with the historical reference like the exterior of the convention hall. That, as a specific design, we had to find in Hungary and were lucky to find at the Heroes’ Square ice rink. We were also looking for the amount of visibility. Sometimes these locations are obstructed by modern architecture or trees. Other than that, we were supporting the art department with their dressing and extending it.”
The subject matter harkens back to an acclaimed HBO miniseries John Adams. “The content and approach were similar, but one of the major differences was that we had some tools at our disposal that didn’t exist back at the time when we did John Adams,” recalls Steve Kullback, Visual Effects Producer. “We did a bunch of previs in Unreal Engine. We set up the convention hall in Unreal Engine and toured it with a virtual reality headset. Rainer Gombos was able to set it up in a way where we could point a laser pointer to a piece of the environment, and you’d be instantly transported there, turn around and be able to see the vast scope of the space. Also, there were no such things as LED volumes, so we didn’t have the benefit of putting our ship in front of a LED wall with our background projected as we did with the train.”
“Unreal Engine has become the platform where you can bring things together. You can bring in the assets as well as the scans that you do, such as Gaussian splats or photogrammetry and some of the set builds. In our case, we had data from the art department, so we were able to bring that in, add our things to it, and offer different camera positions or allow the DP and director to explore the scenes. Unreal Engine is generally being used currently in the industry for all of these visualizations.”
—Rainer Gombos, Visual Effects Supervisor
Unreal Engine is a versatile tool. “Unreal Engine has become the platform where you can bring things together,” Gombos notes. “You can bring in the assets as well as the scans that you do, such as Gaussian splats or photogrammetry and some of the set builds. In our case, we had data from the art department, so we were able to bring that in, add our things to it, and offer different camera positions or allow the DP and director to explore the scenes. Unreal Engine is generally being used currently in the industry for all of these visualizations. We were be able to go to a location in the morning, create a Gaussian splat, bring it in and have a scene ready in the afternoon. This quick turnaround was really useful. Mostly, it helped us to assess locations and make more informed decisions about where bluescreens would be placed and where set extensions would start.”
Virtual production was not simply used for visualization but for in-camera effects. “There were a couple of scenes where our heroes were on a train,” Kullback explains. “It was 1880s America, moving from the Midwest back to Washington on a rural American environment. We thought we could shoot it in front of a greenscreen, but one of the things that was a concern to us, in addition to the number of shots, was having a lightning environment that would interact with the train car. The train had leather seats and a highly reflective wood tabletop, plus there was a dining car scene. Lawrence Jones, who was our Virtual Production Creative Director, created an environment that was rendered to play in real-time so that Rainer could manipulate the perspective of the environment and adjust the lighting. It turned out that we shot in a train museum in Budapest in front of two quite large LED screens that were about 25 feet tall by 80 feet long. It looked authentic, and those LED screens provided 100% of the lighting for the scene. Adriano Goldman, our DP, was pleased with the results, and out of roughly 48 shots, we only touched four of them in post to make some minor adjustments.”
“[Director] Matt [Ross] always had this idea that when James Garfield would enter this big hall, he would be impressed by the scope of it, and we wanted to milk that a little bit. We wanted to draw the viewers attention into the background. We had a CG flag raised just at the right time. It was an extension using digital doubles, but also actors were shot in little groups that were added to the background.”
—Steve Kullback, Visual Effects Producer
Pulling off the interior of the conventional hall was a mammoth task. “We had the added benefit of this being an actual historical event, so we had drawings, etchings and even a couple of photographs from the real-world environment,” Kullback states. “We discussed at length how to approach this, and it was decided that a set would be built and then extended.” Numerous sequences take place within the set. “One of the big items on the show was that we make it as grand as possible,” Gombos remarks. “There was always a plan to add some greenscreens or bluescreens in the back to extend it. But how do we maximize the impact of the foreground and show the most believable crowd shots? We had around 500 extras, and early on the art department knew what they were going to build. We took that and brought it into Unreal Engine and offered it up to Matt Ross [director] and Adriano Goldman to find shots in the sequence. Then we were thinking about, ‘How do we add the most mid-ground and background to these scenes?’ Matt always had this idea that when James Garfield would enter this big hall, he would be impressed by the scope of it, and we wanted to milk that a little bit. We wanted to draw the viewers attention into the background. We had a CG flag raised just at the right time. It was an extension using digital doubles, but also actors were shot in little groups that were added to the background.”
El Ranchito, Important Looking Pirates and Clearcut FX were responsible for the visual effects work. “This was a fairly small project,” Kullback notes. “In the end, we did about 604 shots compared to approximately 1,550 for Season 1 of 3 Body Problem, and our last season of Game of Thrones was over 3,000 shots. We didn’t have as big a team as we ordinarily have. Sometimes we have a cast of dozens on the visual effects team, but on Death by Lightning it was me, Rainer, a data wrangler and an assistant. That was us on set.” The goal was to get as much production value without drawing too much attention to the digital augmentation. “For the cities, we wanted to have the foreground, more mid-ground than the background, but a little bit more background,” Gombos states. “The Budapest streets look good; however, they were too narrow. The U.S. streets were larger. These cities were more modern and had more space. We tried to create more open spaces and add grander buildings to it, even though Budapest is quite grand. Budapest had lots of trees, but Chicago didn’t have that many trees. Gemma Jackson’s [Production Designer] Garfield farmhouse looked great, and what she did in the foreground looked amazing. We did simple stuff, like paint fixes.”
Stock footage was digitally enhanced for the exterior train shots. “To work around our budget constraints, we wanted to use the most difficult-to-achieve thing from the stock footage,” Gombos explains. “We went through a whole lot of stock footage and found something that would work in the best way with the smallest number of changes. Often, it was what’s in the center that should somewhat work. The smoke and engine should be fine, some of the train characters should be roughly the same, and then we would apply changes, like altering the landscape. Generally, we use the smoke to have the maximum amount of realism in the shot. It had to be up res’d. We had some lakes in the background and hikers walking around, so things had to be painted out and make it work with the rest of the sequence.”
Prosthetics were utilized for the autopsy of Charles J. Guiteau. “There was a complete prosthetic of Guiteau’s body having been molded from Matthew Macfadyen,” Kullback reveals. “They did the incision and lifted out the brain. It was quite graphic, disgusting and authentic. That was completely in-camera, which was a tremendous credit to our prosthetics team. There was no digital augmentation. I have been doing this for awhile and I don’t think that I’ve ever broken down a script that didn’t include a fair amount of assistance to prosthetics work. But not here. These guys were quite extraordinary.”
A partial set surrounded by greenscreen was created for the opening warehouse sequence, which takes place in 1969. “We wanted to have something you wouldn’t expect to see in the rest of the episodes, which are set in the 1880s,” Gombos remarks. “It had to be a grand setting and establish the tone. We worked with ILP in trying to add as much depth and variety to the shapes, details and architecture. There were some concept art and references for it. The art department built some of the shelves and we went in and did Gaussian splats of the unfinished set. Because we have this quick turnaround, we could bring it into Unreal Engine and use the unfinished set to layout possible extensions. We would duplicate these Gaussian splats of the unfinished set in the foreground to give us scope and play with some camera moves. The foreground extras were shot again as sprites and cards for the background.”
Establishing shots had to tell a story. “You only have that many establishing shots in a production like this, and we wanted to cram as much information into them as we could,” Gombos notes. “They should be like, ‘This is the world of 1880.’ The world looked completely different in those days. There were more children on the streets, people were younger, and there were amputees because there was the Civil War before that. How do you make the city lived in? The city was still under construction. There was a lot of dirt in the air. There was a lot of industry, so there were smokestacks. All these things made the shots aesthetically more interesting. We tried to put as much as possible into these few shots that we had without drawing too much attention to all of this work. That was our goal and being as historically accurate as much as we can be, but also continue the foreground into the background as seamlessly as we can make it.”
“We talked a lot about what are we going to see out of the window,” Kullback states. “Often times, if it was a big concern we’d ask, ‘Are we going to put greenscreen out of the window? Are we going to have to create an environment to go out the window?’ One of the things Adriano came up with was that we would blow them out. The interiors were candlelit. There were some locations that actually had electricity like the Fifth Avenue hotel, so we were able to play it with incandescent lighting. But for a lot of the interiors, they are muted and soft. Many of those environments are set pieces that played beautifully in-camera, such as the White House, George and Franny Scoville’s house and the farm. A lot of the windows are blown out to be accurate photographically. It was a tremendous collaboration between quite a few departments that brought their A game and produced a piece of work that I’m incredibly proud of.”
