VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of two prestigious 2018 Folio Awards for excellence in publishing.

 

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


December 04
2018

ISSUE

Web Exclusive

Graham Sack Explores the Depths of Historical and Psychological VR

By CHRIS McGOWAN

Abraham Lincoln strides through the cemetery in Lincoln in the Bardo. (Image copyright © 2017 Graham Sack and Sensorium)

Virtual reality’s creative vocabulary has things in common with that of film and theatre, as well as aspects that are all its own.

“Imagine a Venn diagram that incorporates cinematic and theatrical elements and elements that are native to virtual reality,” says Graham Sack, the writer and director of the 360 VR video Lincoln in the Bardo (released in 2017 through the New York Times VR app) and of the four-part series The Interpretation of Dreams (released through Samsung VR Video in 2018).

The task of Sack and other VR directors is to understand what virtual reality has in common with other media, and to develop its particular tools.

Lincoln in the Bardo is a good example of this process. The much-lauded 10-minute piece is based on the George Saunders novel of the same name, and was produced by Graham Sack, Plympton (a literary studio), the new-media studio Sensorium, and The New York Times VR. The Molecule provided VFX work. The cinematic 360 video (with spatialized sound) takes place on a February night in 1862, during the Civil War. A grief-stricken President Abraham Lincoln pays a visit to a marble crypt in a Georgetown cemetery, where his 11-year-old son Willie Lincoln has been laid to rest. The president mourns as his beloved son’s spirit futilely tries to communicate with him. Dead souls – visible to us but not to Lincoln – offer up both piteous and comedic monologues. Everyone is in limbo – hence the title’s use of bardo, Tibetan Buddhism’s name for a transitional state between death and rebirth.

Abraham Lincoln at night in the cemetery in Lincoln in the Bardo. (Image copyright © 2017 Graham Sack and Sensorium)

“The absence of frame is a big anxiety for traditional film directors that are starting to work in virtual reality. They worry that the audience isn’t going to be able to follow a story if you can’t directly manage where they’re looking.”

—Graham Sack, VR Writer/Director

The performances in Lincoln are rather theatrical, being somewhat “elevated,” and delivered by seasoned New York theatre actors like Pete Simpson (Lincoln). And while plays use certain methods to guide the audience’s attention, VR has its own ways to cope with a 360-degree “stage.”

Sack observes, “The absence of frame is a big anxiety for traditional film directors that are starting to work in virtual reality. They worry that the audience isn’t going to be able to follow a story if you can’t directly manage where they’re looking.” This is arguably more of a concern in single-narrative VR, as opposed to interactive VR, with multiple storylines or experiences. Sack adds, “There are a lot of tools for guiding viewers around the space and gently nudging them towards what is narratively significant while still giving them the freedom to look elsewhere if they want to.”

The assembled ghosts look on as Abraham Lincoln carries his dead son in Lincoln in the Bardo. (Image copyright © 2017 Graham Sack and Sensorium)

“[Lincoln in the Bardo] feels like one shot. It unfolds, in a sense, almost like a 10-minute play, with very heavy visual effects. The way we did that, we shot these giant, continuous shots on greenscreen with all of the various characters. We were incredibly careful about the spatial positioning. It was the first time we had the whole crew dealing with greenscreen in a VR setting like that.”

—Graham Sack, VR Writer/Director

Those tools include sound elements and visual cues. “A good example is where Lincoln enters, maybe 140 degrees over the viewer’s left shoulder. With no cuing you probably would not think to turn around and watch the entrance, but we give a series of subtle signals that make turning and looking behind you automatic. One of them is simply sound. You hear his footsteps and some ambient environmental sounds that cue you, like the hooting of an owl. There are a series of things happening in that direction.

“And second, you have the attention of the characters on screen, where they’re looking. Viewers will naturally follow the sightlines of the primary characters on screen. Third, there’s the use of movement and shadow. So there was some dramatic expressionistic lighting. The way Lincoln is lit from behind by this light throws a very long shadow all the way into the foreground directly in front of the viewer. You catch the shadow and the movement of the shadow, and it cues you that something must be happening behind you.

“So there are at least three or four different elements. The result is people feeling like they’re naturally looking behind them but, of course, it’s orchestrated directorially.”

Abraham Lincoln at night in the cemetery in Lincoln in the Bardo. (Image copyright © 2017 Graham Sack and Sensorium)

Abraham Lincoln strides through the cemetery in Lincoln in the Bardo. (Image copyright © 2017  Graham Sack and Sensorium)

Sack had already done some experiments in VR when he approached George Sanders at a 2015 book event in Austin, Texas. He demonstrated the medium for the first time to Sanders, who was immediately intrigued. The two kept in touch, discussing ways to collaborate. Sack read the manuscript of the author’s new book Lincoln in the Bardo (winner of the Man Booker Prize for 2017) and found it “stunningly beautiful.” He adds, “It was set in a single location, a cemetery at night. It was the sort of immersive, transportive location I thought would both work in virtual reality and be evocative. The content was supernatural but still poignant, being about the living and the dead, particularly the living president and the ghost of his dead son. So I thought there was a lot of visual effects potential in terms of the treatment of the spirits, which would be really visually engaging in VR. So that’s sort of what attracted me to doing it.”

“It was very effects heavy,” continues Sack. The dead souls in the cemetery were given a “spectral quality and transparency” and were painstakingly gathered together with extensive greenscreen shots, stitching and layering. “Lincoln and the body of his son were the only people shot on location [at the Green-Wood Cemetery in Brooklyn, N.Y.]. Every other character in the piece was shot separately on greenscreen.

The actors had to work extensively with the green screen in “Lincoln in the Bardo”. © 2017 Graham Sack and Sensorium.

“There was some dramatic expressionistic lighting. The way Lincoln is lit from behind by this light throws a very long shadow all the way into the foreground directly in front of the viewer. You catch the shadow and the movement of the shadow, and it cues you that something must be happening behind you.”

—Graham Sack, VR Writer/Director

“If you watch the piece, it feels like one shot. It unfolds, in a sense,  almost like a 10-minute play, with very heavy visual effects. We shot these giant, continuous shots on greenscreen with all of the various characters. We were incredibly careful about the spatial positioning. It was the first time we had the whole crew dealing with greenscreen in a VR setting like that. The principals, including the soul of Willie and a speaking chorus, were shot basically independently. The giant crowd scenes were shot in layers and then composited together. We shot it by recycling the same dozen supernumeraries in different costumes, different positions, etc.”

The Molecule’s CEO, Chris Healer, adds, “A lot of times we were shooting things and we’d map things out, and we’d have people looking in the strangest directions. But we didn’t really know until post-production was complete and everything was composited together whether people would be looking in the right direction. One of the hardest things is getting the sightlines right with greenscreen in this context.”

Filming at the cemetery in Lincoln in the Bardo. (Image copyright © 2017 Graham Sack and Sensorium)

“I thought there was a lot of visual effects potential in terms of the treatment of the spirits, which would be really visually engaging in VR. And so that’s sort of what attracted me to doing it.”

—Graham Sack, VR Writer/Director

Sack has also teamed with Geoff Sobelle, John Fitzgerald and Matthew Niederhauser (of Sensorium) to create objects in mirror AR closer than they appear, a fusion of augmented-reality technology and an immersive theatre installation, presented in May 2018 at Tribeca Storyscapes, the interactive media division of the Tribeca Film Festival. The Molecule and Sensorium provided technical direction.

Tribeca also presented another VR production by Sack last May, one that was inspired by Sigmund Freud rather than Abraham Lincoln. Sack comments, “Samsung was interested in whether virtual reality could support serialized content, almost like television-style episodes. After Lincoln came out they approached me.” The result was the four-episode The Interpretation of Dreams, written and directed by Sack, who teamed again with Sensorium on the production. “I went back to Sigmund Freud’s The Interpretation of Dreams,” says Sack. “Each episode is based on one of Freud’s original case studies in dreams and the unconscious.”

Graham Sack’s The Interpretation of Dreams. (Image copyright ©  2018 Graham Sack and Sensorium)

About the challenge of utilizing traditional narrative techniques vs. developing immersive storytelling tools, Sack says, “That’s been the most fun in terms of working in the medium. What parts of the directorial vocabulary naturally carry over from other media, like film, and which ones need to be invented from scratch?”


Clare and her fiancé, who has been summoned to see the supercomputer called the Great C (from the VR 360 movie of the same name). (Image copyright © 2018 Secret Location)

Philip K. Dick Meets VR in The Great C

Whereas Lincoln in the Bardo has a lot in common with a play, The Great C would have more elements in the cinematic circle of a Venn diagram. Entertainment One’s (eOne) studio Secret Location has adapted science-fiction author Philip K. Dick’s short story The Great C into what might be termed a VR 360 movie. (Toronto-based Secret Location calls it “a cinematic VR narrative.”) Whatever the label, the Steve Miller-directed immersive experience has a compelling story and convincing “presence” (the feeling that you are really there). It is like watching a well-made 3D animated film, except you’re inside it and can take in the rich detail all around, including above and below you. There are 20 unique environments in the 30-minute-plus experience, which maintains the viewer’s feeling of presence, despite cinematic cuts and camera movements not usually seen in VR (note: it is a passive experience, not interactive). In addition, it runs at 90fps, which helps eliminate viewer discomfort during long VR 360 movies.

Dick’s highly influential fiction has provided the basis for many movies, including Blade Runner, Minority Report and Total Recall. The story is set in a post-apocalyptic world where an all-powerful supercomputer known as the Great C rules over what’s left of humanity. Each year, a village is forced to send a young person on a pilgrimage to appease the mysterious machine – a journey from which no one ever returns. Clare is a young woman whose fiancé is summoned for the pilgrimage; she must decide whether to let him go alone or accompany him to ensure his safety. The Great C is the first adaptation of a Philip K. Dick story into VR, and was scheduled to be available at select location-based venues in the fourth quarter of 2018 – and for viewing on Oculus Rift, HTC Vive and PlayStation VR headsets afterwards.

Facing the supercomputer in The Great C. (Image copyright © 2018 Secret Location)

The 360 VR film The Great C. (Images copyright © 2018 Secret Location)

Share this post with

Get the Latest VFX News

Receive notifications when WEB EXCLUSIVE content is posted on VFXVoice.com

Select List(s)*

Most Popular Stories

The Miniature Models of <strong>BLADE RUNNER</strong>
02 October 2017
Exclusives, VR Trends
The Miniature Models of BLADE RUNNER
In 1982, Ridley Scott’s Blade Runner set a distinctive tone for the look and feel of many sci-fi future film noirs to come, taking advantage of stylized production design, art direction and visual effects work.
THE PEARL: THE SUPER ALIEN MODELS OF<strong> VALERIAN</strong>
02 August 2017
Exclusives, VR Trends
THE PEARL: THE SUPER ALIEN MODELS OF VALERIAN
Among the many creatures and aliens showcased in Luc Besson’s Valerian and the City of a Thousand Planets are members of the Pearl, a beautiful...
The New <strong>Artificial Intelligence</strong> Frontier of VFX
20 March 2019
Exclusives, VR Trends
The New Artificial Intelligence Frontier of VFX
The new wave of smart VFX software solutions utilizing A.I.
Converting a Classic: How Stereo D Gave <strong>TERMINATOR 2: JUDGMENT DAY</strong> a 3D Makeover
24 August 2017
Exclusives, VR Trends
Converting a Classic: How Stereo D Gave TERMINATOR 2: JUDGMENT DAY a 3D Makeover
James Cameron loves stereo. He took full advantage of shooting in native 3D on Avatar, and has made his thoughts clear in recent times about the importance of shooting natively in stereo when possible...
2018 – Year of the Rapidly Expanding <b>VFX Moviemaking Vocabulary</b>
03 April 2018
Exclusives, VR Trends
2018 – Year of the Rapidly Expanding VFX Moviemaking Vocabulary
Industry leaders discuss trends and issues.