ZOIC STUDIOS AND THE LESSONS LEARNED FROM ITS LED WALL

By IAN FAILES

An overhead lighting rig is installed on the LED wall stage.

If there’s one thing changing the way that physical production and visual effects interact right now, it’s the rise of LED walls, also known as LED volumes.

LED walls form part of the shift to virtual production and real-time content creation, and have gained prominence by enabling in-camera VFX shots that don’t require additional post-production, offering an alternative to bluescreen or greenscreen shooting, and helping to immerse actors and crews into what will eventually be the final shots.

Several production houses and visual effects studios have built dedicated LED walls and associated workflows. Some rely on bespoke LED wall setups, depending on the needs of the particular project. One company, Zoic Studios, even embarked on a project to install a LED wall inside existing studio space and use it to further the crew’s understanding of virtual production.

How did Zoic do that exactly? Here, Zoic Creative Director and Visual Effects Supervisor Julien Brami breaks down the process they followed and what they took away from this new virtual production experience.

A HISTORY OF VIRTUAL PRODUCTION AT ZOIC

Zoic’s foray into LED walls is actually part of a long history of virtual production solutions at the visual effects company, which has offices in Los Angeles, New York and Vancouver. In particular, some years ago the studio developed a system called ZEUS (Zoic Environmental Unification System) that could be used to track live-action camera movements on, for example, a greenscreen stage and produce real-time composites with previs-like assets. The idea was to provide instant feedback on set via a Simul-cam setup, either a dedicated monitor or smart tablet.

Zoic Studios Creative Director and Visual Effects Supervisor Julien Brami reviews a real-time scene on a laptop.

“FTG [Fuse Technical Group] lent us an LED wall for about three months, and it was really exciting because they said, ‘We can give you the panels and we can do the installation, but we can’t really give you anyone who knows how to do it. You have to learn yourself.’ So it was just a few of us here at Zoic doing it. We each tried to figure everything out.”

—Julien Brami, Creative Director and Visual Effects Supervisor, Zoic Studios

When Brami, who has been at Zoic since 2015, began noticing a major shift in the use of game engines for virtual production shoots on more recent shows – The Mandalorian, for example, has of course brought these new methods into the mainstream – he decided to try out the latest techniques himself at home using Epic Games’ Unreal Engine. “I’ve always been extremely attracted by new workflows and new technology,” shares Brami. “I mostly work in the advertising world and usually the deadlines are short and budgets are even shorter, so I’m always looking for new solutions. I’d been looking to employ real-time rendering in a scene as a way to show clients results straight away, for example.

“So,” he adds, “I first started doing all this virtual production stuff in my own room at home using just my LED TV and a couple of screens. What’s amazing with the technology is that the software is free. I just had a Vive controller and was tracking shots with my Sony DSLR. As soon as I saw something working, I called [Zoic Founder and Executive Creative Director] Chris Jones and said, ‘I think we need to do this at scale.’”

Zoic Studios tests the LED wall that was set up at its Culver City, California location.

A hand-held test-shoot against an LED wall environment.

The LED wall, provided to Zoic on loan from Fuse Technical Group, gave the visual effects studio a chance to test several shooting scenarios.

The LED wall, a ROE Black Pearl 2, was 28 tiles wide and 8 tiles high.

An on-set tracking system utilizing Vive controllers allowed for camera movement and real-time re-adjustment of the imagery on the LED wall.

“The panels create light, but not strong enough to get shadows. So, you can have an overall global lighting and the value’s just right on the character, but there is no shadow for the character. That’s why we wanted to also have practical lighting. We used the DMX plugin from Unreal, which would send information for correct intensity and color temperature to every single practical light. So, when you move in real-time, the light actually changes, and we can create the real cast shadows on the characters.”

—Julien Brami, Creative Director and Visual Effects Supervisor, Zoic Studios

‘WE NEED TO LEARN IT’

With an Epic MegaGrant behind them, Zoic partnered with Fuse Technical Group (FTG) and Universal Studios to create a LED wall at their studio space in Culver City. It was a big leap for Zoic, which predominantly delivers episodic, film and commercial VFX work. Subsequently, the studio launched a ‘Real Time Group’ to explore more virtual production offerings and use Unreal Engine for visualization, virtual art department deliveries, animation and virtual production itself. “What we said was, ‘We have the time, we have the technology, we need to learn it,’” relates Brami, in terms of the initial push into LED wall filmmaking. “FTG lent us an LED wall for about three months, and it was really exciting because they said, ‘We can give you the panels and we can do the installation, but we can’t really give you anyone who knows how to do it. You have to learn yourself.’ So it was just a few of us here at Zoic doing it. We each tried to figure everything out.” The LED wall setup at Zoic was used for a range of demos and tests, as well as for a ‘Gundam Battle: Gunpla Warfare’ spot featuring YouTuber Preston Blaine Arsement. The spot sees Preston interact with an animated Gundam character in a futuristic warehouse. The character and warehouse environment were projected on the LED wall. “That was the perfect scenario for this,” says Brami.

The wall was made up of around 220 panels. Zoic employed a Vive tracker setup for camera tracking and as a way to implement follow-focus. The content was pre-made for showing on the LED wall and optimized to run in Unreal Engine in real-time. “We had the LED processor and then two stations that were running on the set,” details Brami. “We also had a signal box that allowed us to have genlock, because that’s what matters the most. Everything has to work in tandem to ensure the refresh rate is right and the video cards can handle the playback.” There were many challenges in making its LED wall setup work, notes Brami, such as dealing with lag on the screen, owing to the real-time tracking of the physical camera. Another challenge involved lighting the subject in front of the LED panels. “The panels create light, but not strong enough to get shadows. So, you can have an overall global lighting and the value’s just right on the character, but there is no shadow for the character. That’s why we wanted to also have practical lighting. We used the DMX plugin from Brami. “When you shoot on location, you may have the best location ever, but you don’t know if next year if there’s another season whether this location will be there or will remain untouched. Now, even 10 years from now, you might want a location in Germany, well, we can come back to the same scene built for virtual produc-tion. Exactly the same scene, which really is mind blowing.”

Thoughts on an LED Wall Future

Arising from his LED wall and virtual production experiences so far, Zoic Studios Creative Director and Visual Effects Supervisor Julien Brami believes there are many advances still to come in this space.

One development Brami predicts is that there will be more widespread operation of LED walls remotely, partly something that became apparent during the pandemic. “All of this technology just works on networks. My vision is that one day I can have an actor somewhere in Texas, an actor somewhere in Germany, I can have the director anywhere else, but they all look at the same scene. As long as it can be all synchronized, we’ll be able to do it. And then you won’t need to all travel to the same location if you can’t do that or if it’s too expensive.”

Another advancement that Brami sees as coming shortly is further development in on-set real-time rendering to blend real and virtual environments. “This is going to be like an XR thing. You shoot with an LED wall, but then you can also add a layer of CG onto the frame as well – that’s still real-time. You can sandwich the live-action through the background that’s Unreal Engine-based on the wall and then add extra imagery over the wall.

“Doing this,” Brami says, “means you can actually create a lot of really cool effects, like particles and atmospherics, and make your sets bigger. It does need more firepower on set, but I think this is really what’s going to blend from an actor in front of a screen to something that’s fully completed. You could put in a creature there, you can put anything your space can sandwich. I’m really excited about this.”

LED panels with a 2.8mm pixel pitch made up the wall.

Zoic Studios has ultimately established a ‘Real-Time Group’ at the studio to handle virtual production and real-time game-engine solutions to VFX work.

Filming the ‘Gundam Battle: Gunpla Warfare’ spot featuring YouTuber Preston Blaine Arsement, left.

Preston Blaine Arsement performs on the LED wall set.

An Unreal Engine view of the Gundam character for the ‘Gundam Battle: Gunpla Warfare’ commercial.

The video village, or ‘brain bar’ area of the LED wall stage, during filming of the Gundam commercial at Zoic.

“We know how to create VFX and we know how to create content and assets, we just need to get more involved in preparing this content for real-time on LED walls, since the assets need to run at high frame rates…”

—Julien Brami, Creative Director and Visual Effects Supervisor, Zoic Studios

THE STATE OF PLAY WITH ZOIC AND LED WALLS
It became apparent to Zoic during this test and building phase that a permanent build at their studio space may not be necessary, as Brami explains. “We realized a couple of things. First, this tech-nology was changing so fast. Literally, every three months there was a new development, which meant buying something now and having it delivered in three months means it would be obsolete. “Take the LED panels, for example,” adds Brami. “The panels we had at the time were 2.8mm pixel pitch. The pixel pitch is basi-cally the distance between LEDs on a panel, and these define the resolution you can get from the real estate of the screen and how close you can get to it and all the aberrations you can get. When we started shooting, 2.8 was state-of-the-art. But then we’ve seen pixel pitches of 1.0 appearing already. Everybody has seen the potential of this technology, and the manufacturers want to make it even better.”

This meant Zoic decided that, instead of buying an LED wall for permanent use, to utilize the loaned wall as much as possible for the three months to understand how it all worked. “Our main focus then became creating really amazing content for LED walls,” states Brami. “We know how to create VFX and we know how to create content and assets, we just need to get more involved in preparing this content for real-time on LED walls, since the assets need to run at high frame rates, etc.”

Already, Zoic has produced such content for a space-related project and also for scenes in the streaming series Sweet Tooth, among other projects. Indeed, Zoic believes it can take the knowledge learned from its LED wall experience and adapt it for individual clients, partnering with panel creators or other virtual production services to produce what the client needs. “So, if they need a really high-end wall, because everything needs to be still in focus, we know where to go and how to create a set for that,” says Brami. “But if they just need, let’s say, an out-of-focus background for a music video, we know where to go for that approach. “I love where it’s all going, and I’m starting to really love this new technology,” continues Brami. “The past three years has really been the birth of it. The next five years are going to be amazing.”

Share this post with