VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


September 24
2019

ISSUE

Web Exclusive

AR Adventures: Weta Gameshop’s Magic Leap Journey

By IAN FAILES

(Images courtesy of Magic Leap)

In the game Dr. Grordbort’s Invaders, players don a Magic Leap One and use its hand controller to shoot steampunk robots using a steampunk raygun. This happens in augmented reality (AR), meaning the imagery a player sees is computer-generated and is overlaid on the real world around them via the Magic Leap One goggles.

To play Dr. Grordbort’s Invaders, the space the user is in is first scanned with the Magic Leap One. This produces a ‘mesh’ of the area, and allows robots and other elements of the game to appear to be locked in with that space, or occluded by tables or other items in front of the player as he or she moves around.

For several years now, keen observers have been following the progress of Magic Leap as it looked to release a product centered on augmented reality and computer vision. Last year, the company’s first headset, the Magic Leap One, became available to the public, and one of its flagship experiences was Dr. Grordbort’s Invaders.

The game’s developer is Weta Gameshop, which has been in close partnership with Magic Leap for some time, all the way back to some of the first research Magic Leap made into augmented reality, which, among several other areas of computer vision, involves light-fields.

Photo-composite of Dr. Grordbort’s Invaders.

At the center of development of the Dr. Grordbort’s game is Greg Broadmore, studio director at Weta Gameshop. With a history in concept art, creature design and special effects, he invented the world of Dr. Grordbort’s, eventually ushering this world into the AR experience as it now exists for the Magic Leap One. It has been a long journey for Broadmore.

“When I first started, actually, there was no technology,” the Gameshop studio director told VFX Voice at the recent Neuchâtel International Fantastic Film Festival in Switzerland. “I flew to Florida, where Magic Leap is, met the original team, and really all it was then were ideas about a light field thing.”

However, when Broadmore returned to Florida a second time, he was able to see more of Magic Leap in action with what was known as the ‘Benchtop’. “It was also called ‘the Beast,’” reveals Broadmore. “It was basically this thing strapped to a table, a massive aluminum frame with wires poking out of it, just an explosion of science fiction.”

Greg Broadmore, studio director, Weta Gameshop.

“It was astonishing just to see people smile and run up and interact with the robots.”

—Greg Broadmore, Studio Director, Weta Gameshop

Artwork for the game.

“We’d already had a version of the [Dr. Grordbort’s Invaders] game running on a WD1, but that one couldn’t estimate head pose. The WD2 (Wearable Device Two) was pretty rubbish at estimating head pose, but it could do it, and that was the revelation. We also coupled that with our hand controller, and although it was all connected to this big cable connected to a PC, we could move around within a couple square meters. I could see what the game could be at that point.”

—Greg Broadmore, Studio Director, Weta Gameshop

What Broadmore saw was a light field display in which viewers placed their chin on a cradle and peered through two lenses. “Through that I saw five crosses, very boring crosses,” he recalls. “They weren’t even programmed, they were photos of crosses! Flat black and white images. But the important part was that they were a light field. I looked at the front one and it was in focus. And then I moved up to the back one, and the front one went out of focus. And I was like, ‘Holy shit, that looks real.’”

Broadmore was impressed with the display, but thought those crosses were “truly ugly.” With the world of Dr. Grordbort’s on his mind, he suggested Magic Leap put something more radical in the light field display. “I said, ‘Let’s put ray guns in there!’ So, on that trip I got to see ray guns in there and experiment a little bit with that.”

As research at Magic Leap continued, so too did Weta Gameshop’s development of a Dr. Grordbort’s experience that could be used with augmented reality. While the game works with the Magic Leap One headset now, at one stage it was also displayed on the large Benchtop itself. “It was a very narrow field of view,” notes Broadmore, “but you could see from the top of the robots’ heads to just below their knuckles. Still, we had the game working, and that was a ‘Eureka!’ thing for us. It was very much a hack thing, and we’d just keep on getting better with every update.”

Artwork for the game.

“We are just at the start of this. Every single performance attribute of that device is just going to keep on getting better and better. To me, the field of view is the number one thing. You can design around it, and I think we’ve made an incredible game that deals with it in a really good way, but it’s of course the thing I would like to get better.”

—Greg Broadmore, Studio Director, Weta Gameshop

The goal, of course, was to miniaturize the Benchtop experience to something that could be worn by a user as a headset. The first time Broadmore saw a working prototype of such a device, that also met some of his main imaginations for Dr. Grordbort’s, was with Magic Leap’s WD2 (Wearable Device Two). This featured several sensors that aided in head tracking and other things crucial to lining up CG imagery with the real world for AR.

“We’d already had a version of the game running on a WD1, but that one couldn’t estimate head pose,” says Broadmore. “The WD2 was pretty rubbish at estimating head pose, but it could do it, and that was the revelation. We also coupled that with our hand controller, and although it was all connected to this big cable connected to a PC, we could move around within a couple square meters. I could see what the game could be at that point.”

Finally, with the release of the Magic Leap One, Broadmore could see Dr. Grordbort’s Invaders as intended and have people move freely about to play the game. “It was astonishing,” he says, “just to see people smile and run up and interact with the robots.”

A person wears the Magic Leap One, which consists of goggles, a processing pack and a hand controller.

“It’s like we’ve made the first phone camera and it doesn’t do everything we want to do, so now we’ll keep refining it and adding to it until it’s awesome. But even that will never end, there’s always something more. That’s the fun of making technology.”

—Greg Broadmore, Studio Director, Weta Gameshop

Research continues at Magic Leap, and development also goes on at Weta Gameshop. Areas such as field of view, more accurate tracking and occlusion, and further miniaturization of the device, are things being worked on.

“We are just at the start of this,” states Broadmore. “Every single performance attribute of that device is just going to keep on getting better and better. To me, the field of view is the number one thing. You can design around it, and I think we’ve made an incredible game that deals with it in a really good way, but it’s of course the thing I would like to get better.

“And there’s 100 other things that I would like to get better,” adds Broadmore. “But that’s the fun of the job. It’s like we’ve made the first phone camera and it doesn’t do everything we want to do, so now we’ll keep refining it and adding to it until it’s awesome. But even that will never end, there’s always something more. That’s the fun of making technology.”

Watch a gameplay trailer for Dr. Grordbort’s Invaders.

Share this post with

Get the Latest VFX News

Receive notifications when WEB EXCLUSIVE content is posted on VFXVoice.com

Select List(s)*

Most Popular Stories

The Miniature Models of <strong>BLADE RUNNER</strong>
02 October 2017
Exclusives, VR/AR Trends
The Miniature Models of BLADE RUNNER
In 1982, Ridley Scott’s Blade Runner set a distinctive tone for the look and feel of many sci-fi future film noirs to come, taking advantage of stylized production design, art direction and visual effects work.
THE PEARL: THE SUPER ALIEN MODELS OF<strong> VALERIAN</strong>
02 August 2017
Exclusives, VR/AR Trends
THE PEARL: THE SUPER ALIEN MODELS OF VALERIAN
Among the many creatures and aliens showcased in Luc Besson’s Valerian and the City of a Thousand Planets are members of the Pearl, a beautiful...
The New <strong>Artificial Intelligence</strong> Frontier of VFX
20 March 2019
Exclusives, VR/AR Trends
The New Artificial Intelligence Frontier of VFX
The new wave of smart VFX software solutions utilizing A.I.
Converting a Classic: How Stereo D Gave <strong>TERMINATOR 2: JUDGMENT DAY</strong> a 3D Makeover
24 August 2017
Exclusives, VR/AR Trends
Converting a Classic: How Stereo D Gave TERMINATOR 2: JUDGMENT DAY a 3D Makeover
James Cameron loves stereo. He took full advantage of shooting in native 3D on Avatar, and has made his thoughts clear in recent times about the importance of shooting natively in stereo when possible...
2018 – Year of the Rapidly Expanding <b>VFX Moviemaking Vocabulary</b>
03 April 2018
Exclusives, VR/AR Trends
2018 – Year of the Rapidly Expanding VFX Moviemaking Vocabulary
Industry leaders discuss trends and issues.