VFX Voice

The award-winning definitive authority on all things visual effects in the world of film, TV, gaming, virtual reality, commercials, theme parks, and other new media.

Winner of three prestigious Folio Awards for excellence in publishing.

Subscribe to the VFX Voice Print Edition

Subscriptions & Single Issues


September 24
2019

ISSUE

Web Exclusive

AR Adventures: Weta Gameshop’s Magic Leap Journey

By IAN FAILES

(Images courtesy of Magic Leap)

In the game Dr. Grordbort’s Invaders, players don a Magic Leap One and use its hand controller to shoot steampunk robots using a steampunk raygun. This happens in augmented reality (AR), meaning the imagery a player sees is computer-generated and is overlaid on the real world around them via the Magic Leap One goggles.

To play Dr. Grordbort’s Invaders, the space the user is in is first scanned with the Magic Leap One. This produces a ‘mesh’ of the area, and allows robots and other elements of the game to appear to be locked in with that space, or occluded by tables or other items in front of the player as he or she moves around.

For several years now, keen observers have been following the progress of Magic Leap as it looked to release a product centered on augmented reality and computer vision. Last year, the company’s first headset, the Magic Leap One, became available to the public, and one of its flagship experiences was Dr. Grordbort’s Invaders.

The game’s developer is Weta Gameshop, which has been in close partnership with Magic Leap for some time, all the way back to some of the first research Magic Leap made into augmented reality, which, among several other areas of computer vision, involves light-fields.

Photo-composite of Dr. Grordbort’s Invaders.

At the center of development of the Dr. Grordbort’s game is Greg Broadmore, studio director at Weta Gameshop. With a history in concept art, creature design and special effects, he invented the world of Dr. Grordbort’s, eventually ushering this world into the AR experience as it now exists for the Magic Leap One. It has been a long journey for Broadmore.

“When I first started, actually, there was no technology,” the Gameshop studio director told VFX Voice at the recent Neuchâtel International Fantastic Film Festival in Switzerland. “I flew to Florida, where Magic Leap is, met the original team, and really all it was then were ideas about a light field thing.”

However, when Broadmore returned to Florida a second time, he was able to see more of Magic Leap in action with what was known as the ‘Benchtop’. “It was also called ‘the Beast,’” reveals Broadmore. “It was basically this thing strapped to a table, a massive aluminum frame with wires poking out of it, just an explosion of science fiction.”

Greg Broadmore, studio director, Weta Gameshop.

“It was astonishing just to see people smile and run up and interact with the robots.”

—Greg Broadmore, Studio Director, Weta Gameshop

Artwork for the game.

“We’d already had a version of the [Dr. Grordbort’s Invaders] game running on a WD1, but that one couldn’t estimate head pose. The WD2 (Wearable Device Two) was pretty rubbish at estimating head pose, but it could do it, and that was the revelation. We also coupled that with our hand controller, and although it was all connected to this big cable connected to a PC, we could move around within a couple square meters. I could see what the game could be at that point.”

—Greg Broadmore, Studio Director, Weta Gameshop

What Broadmore saw was a light field display in which viewers placed their chin on a cradle and peered through two lenses. “Through that I saw five crosses, very boring crosses,” he recalls. “They weren’t even programmed, they were photos of crosses! Flat black and white images. But the important part was that they were a light field. I looked at the front one and it was in focus. And then I moved up to the back one, and the front one went out of focus. And I was like, ‘Holy shit, that looks real.’”

Broadmore was impressed with the display, but thought those crosses were “truly ugly.” With the world of Dr. Grordbort’s on his mind, he suggested Magic Leap put something more radical in the light field display. “I said, ‘Let’s put ray guns in there!’ So, on that trip I got to see ray guns in there and experiment a little bit with that.”

As research at Magic Leap continued, so too did Weta Gameshop’s development of a Dr. Grordbort’s experience that could be used with augmented reality. While the game works with the Magic Leap One headset now, at one stage it was also displayed on the large Benchtop itself. “It was a very narrow field of view,” notes Broadmore, “but you could see from the top of the robots’ heads to just below their knuckles. Still, we had the game working, and that was a ‘Eureka!’ thing for us. It was very much a hack thing, and we’d just keep on getting better with every update.”

Artwork for the game.

“We are just at the start of this. Every single performance attribute of that device is just going to keep on getting better and better. To me, the field of view is the number one thing. You can design around it, and I think we’ve made an incredible game that deals with it in a really good way, but it’s of course the thing I would like to get better.”

—Greg Broadmore, Studio Director, Weta Gameshop

The goal, of course, was to miniaturize the Benchtop experience to something that could be worn by a user as a headset. The first time Broadmore saw a working prototype of such a device, that also met some of his main imaginations for Dr. Grordbort’s, was with Magic Leap’s WD2 (Wearable Device Two). This featured several sensors that aided in head tracking and other things crucial to lining up CG imagery with the real world for AR.

“We’d already had a version of the game running on a WD1, but that one couldn’t estimate head pose,” says Broadmore. “The WD2 was pretty rubbish at estimating head pose, but it could do it, and that was the revelation. We also coupled that with our hand controller, and although it was all connected to this big cable connected to a PC, we could move around within a couple square meters. I could see what the game could be at that point.”

Finally, with the release of the Magic Leap One, Broadmore could see Dr. Grordbort’s Invaders as intended and have people move freely about to play the game. “It was astonishing,” he says, “just to see people smile and run up and interact with the robots.”

A person wears the Magic Leap One, which consists of goggles, a processing pack and a hand controller.

“It’s like we’ve made the first phone camera and it doesn’t do everything we want to do, so now we’ll keep refining it and adding to it until it’s awesome. But even that will never end, there’s always something more. That’s the fun of making technology.”

—Greg Broadmore, Studio Director, Weta Gameshop

Research continues at Magic Leap, and development also goes on at Weta Gameshop. Areas such as field of view, more accurate tracking and occlusion, and further miniaturization of the device, are things being worked on.

“We are just at the start of this,” states Broadmore. “Every single performance attribute of that device is just going to keep on getting better and better. To me, the field of view is the number one thing. You can design around it, and I think we’ve made an incredible game that deals with it in a really good way, but it’s of course the thing I would like to get better.

“And there’s 100 other things that I would like to get better,” adds Broadmore. “But that’s the fun of the job. It’s like we’ve made the first phone camera and it doesn’t do everything we want to do, so now we’ll keep refining it and adding to it until it’s awesome. But even that will never end, there’s always something more. That’s the fun of making technology.”

Watch a gameplay trailer for Dr. Grordbort’s Invaders.

Share this post with

Most Popular Stories

2024 STATE OF THE VFX/ANIMATION INDUSTRY: FULL SPEED AHEAD
09 January 2024
Exclusives, VR/AR Trends
2024 STATE OF THE VFX/ANIMATION INDUSTRY: FULL SPEED AHEAD
Despite production lulls, innovations continue to forward the craft.
CHANNELING THE VISUAL EFFECTS ON SET FOR THE WHEEL OF TIME SEASON 2
23 January 2024
Exclusives, VR/AR Trends
CHANNELING THE VISUAL EFFECTS ON SET FOR THE WHEEL OF TIME SEASON 2
In the case of the second season of The Wheel of Time, the on-set visual effects supervision was equally divided between Roni Rodrigues and Mike Stillwell.
NAVIGATING LONDON UNDERWATER FOR THE END WE START FROM
05 March 2024
Exclusives, VR/AR Trends
NAVIGATING LONDON UNDERWATER FOR THE END WE START FROM
Mahalia Belo’s remarkable feature directorial debut The End We Start From follows a woman (Jodie Comer) and her newborn child as she embarks on a treacherous journey to find safe refuge after a devastating flood.
REMOTE WORK 2024: FINDING THE BEST BALANCE BETWEEN HOME AND OFFICE
17 January 2024
Exclusives, VR/AR Trends
REMOTE WORK 2024: FINDING THE BEST BALANCE BETWEEN HOME AND OFFICE
In just a few years, turbocharged by the pandemic, remote work has become widely established in the VFX industry and is now a preferred option for many visual artists.
GAIA BUSSOLATI: CONNECTING HOLLYWOOD AND ITALIAN CINEMA – AND STAYING CURIOUS
09 January 2024
Exclusives, VR/AR Trends
GAIA BUSSOLATI: CONNECTING HOLLYWOOD AND ITALIAN CINEMA – AND STAYING CURIOUS
VFX Supervisor bridges Italian cinema, Hollywood blockbusters.