Original article appears in Variety - August 14th, 2018

Metastage Opens Microsoft-Powered Volumetric Capture Studio in Los Angeles


By TODD LONGWELL


CREDIT: COURTESY OF METASTAGE


Hollywood, get ready to make holograms: Metastage, a new studio for volumetric video capture for virtual reality (VR), augmented reality (AR), and holographic experiences, opened its doors at Culver Studios in Culver City, Calif., on Tuesday.

Housed on Stage 8 of the historic studio lot, Metastage is using cutting-edge volumetric capture technology developed by Microsoft. The software giant has been working on this technology for close to 8 years, and is operating its own Mixed Reality Capture Studios in San Francisco and Redmond, Wash., as well as licensing it to London’s Dimension Studios. Metastage is the first U.S. partner to license Microsoft’s Mixed Reality Capture Studios technology.

“We’re not a technology company, we’re a client-facing production company,” said Metastage CEO Christina Heller, formerly of VR Playhouse. “Our goal is to strip the complication from this as much as possible so by the time the talent gets here, all they have to do is walk on stage and you can say ‘action.'”


Partners in Metastage include Magnopus, a production company that has created VR experiences for Disney/Pixar’s “Coco” and Warner Bros.’ “Blade Runner 2049.”

“Both Christina and I come from the world of immersive production, and that world looks a lot more like a mad scientist’s laboratory than it does content creation and creativity,” said Magnopus co-founder and CEO Ben Grossmann, who won an Oscar for his work as a visual effects supervisor on 2011’s “Hugo.” “With Metastage, the technology has been vetted, so we don’t have to focus on being the Q laboratory from a James Bond movie.”

Applications of volumetric capture — which allows a virtual camera (or viewer) to move around live performers at will — range from games and movie-like experiences to education and training.

“If you just put game engine characters in there, the closer they get to reality, the creepier they get,” Grossmann said. And with live capture, “you’re focused on directing an actor rather than a team of engineers or artists to build that performance, which can take months.”


CREDIT: COURTESY OF METASTAGE


Heller was impressed with the lack of artifacts in Microsoft’s volumetric capture as well as the company’s compression technology. The latter allows the facility to take the 10 gigabytes of data it captures per second and shrink it down to 10 megabits, allowing producers to easily import the files to Unity and Unreal game engines and use them for streaming mobile AR experiences.

While Metastage’s neighbors on the Culver Studios lot include traditional big Hollywood productions such as the ABC series “Agents of S.H.I.E.L.D.,” it’s not a typical soundstage build with freestanding walls that will be torn down immediately after shooting wraps. Designed by Cassidy Shipley and built by scenic construction company Jet Sets, it has a solid mid-century-industrial-meets-“Star Trek” look, all gray concrete walls, and blue carpet accented by recessed lighting.

Visitors entering Metastage walk down a hallway into a waiting area, where they are greeted by four small sculptures of a ballerina in motion suspended over an island in the center of the room.

“I wanted this to feel like an experience right when you walked through the door, and also make it a really fun place for people to hang out for hours and hours while we’re doing productions,” Heller said.


CREDIT: COURTESY OF METASTAGE

Metastage CEO Christina Heller.

Metastage has a green room, a wardrobe department, craft services, production offices, a rehearsal space next to the capture stage, and a conference room upstairs. Behind the stage is the server and render farm, enclosed in a concrete structure designed to resemble the power station on the cover of Pink Floyd’s “Animals,” will dual chimneys on top to help vent the heat.

The heart of the operation is the capture stage itself, which measures 35 feet in diameter, capable of capturing a space of up to 10 feet in diameter volumetrically. The stage is equipped with 106 cameras (53 RGB and 53 infrared) future-proofed with 12 megapixel sensors, even though the software currently only accommodates 4 megapixels.

“It will reveal itself over time, but our expectation is that we may get better captures right off the bat,” Heller said.


CREDIT: COURTESY OF METASTAGE

The first round of projects booked into Metastage include an AR activation by a major newspaper publisher and an immersive documentary by Springbok Entertainment capturing a performance by a ballerina living with terminal cancer.

Metastage has a one-year lease on the space, but Heller hopes to be there longer. And if the concept proves popular, they plan to open other Metastage locations internationally.

“The real ambition here is to democratize the means of production so other people around the world can spring up and make great content and overcome that barrier to entry,” Grossmann said.


Original article appears in the New York Times - September 5th, 2018

A Hologram Hits the Runway

In the Times AR team’s first rendition of video that you can walk around, Ashley Graham is “a model in motion without the filters, the Photoshopping and the angles the fashion industry is used to.”


By GRAHAM ROBERTS



Ashley Graham, surrounded by cameras, on a feedback monitor at Metastage, a brand new volumetric video studio at the Culver City Studios complex in Los Angeles.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES


Times Insider delivers behind-the-scenes insights into how news, features and opinion come together at The New York Times.

When we stepped out of an Uber at Culver City Studios in Los Angeles, the place where cultural touchstones such as “Citizen Kane,” “Beetlejuice” and “The Matrix” were filmed, it was still early enough to be dark outside. The eerie quiet of the place at that hour had us half expecting to see the ghosts of Hollywood crews still zooming between sound stages. Or at least a Pepper’s ghost.

Culver City Studios at dawn.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES

We were here, after all, to create our first holographic video recording. (It was the next evolution of our augmented reality app feature, which was introduced in February with “Four of the Best Olympians, as You’ve Never Seen Them,” pegged to the 2018 Winter Olympics in Pyeongchang, South Korea.)

Our subject was the model and activist Ashley Graham. She had a strict cutoff at noon, and we were leaving extra time for things to go wrong.

We would be using Microsoft’s new “volumetric capture” technology — which deploys advanced computer algorithms to compress the nearly 10 gigabytes of information collected every second, and would help us give a new twist to the epithet “newspaper of record.” Ashley, in this brief recorded sequence, would be endlessly available to be projected into the world.

Long a staple of science fiction, the ability to beam somebody into your space in three dimensions (think receiving an urgent message from Leia in “Star Wars”), and walk around them as if they’re actually there, is gradually becoming real.

As part of “Ashley Graham, Unfiltered — an interview conducted by Joanna Nikas, a New York Times Styles editor, that explores issues around body image — readers who have downloaded the New York Times app (iOS) onto their iPhones or iPads can project a “hologram” of Ashley into their space as she demonstrates poses and her runway walk.






This is a new video format, with a new production process. Rather than record Ashley from just a few perspectives, she is captured from all angles using more than 100 cameras positioned around her on a stage that is essentially a giant green-screen in-the-round. The images from the cameras are then composited and processed together to create a single hologram that, when viewed through the screen on your phone or tablet, can be placed into your own physical environment.

Joanna helped choose her as the subject of our first exploration of this new AR form. “Ashley has been a champion for body diversity and positivity for years. She was one of the first curvy women to be on the cover of Sports Illustrated,” she explains. “This technology allows people to see a model in motion without the filters, the Photoshopping and the angles the fashion industry is used to.”

Which is exactly the way Ashley wants it. “You see my lower belly fat, you might be seeing some cellulite, you see the curves of my arms and my butt. You’re seeing all of it,” she told Joanna during her interview. “I’m showing the real me. And I want other women to know that you can show the real you and the real you is perfect, the real you is beautiful.”



The Metastage stylist Harmony Arnold, left, oversees some last-minute legwork before a take.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES


We recorded Ashley Graham at Metastage, a brand new volumetric video studio in the Culver City Studios complex. “Having Ashley as one of our first captures was a testament to how far we’ve come with female empowerment in business,” said the studio’s chief executive, Christina Heller. “We had a strong, vivacious supermodel performing on a state-of-the-art volumetric capture stage run by women. The energy was fun and inclusive, just like it should be.”

As might be expected, the technology is not without its limitations at this early stage. The capture zone is confined to an 8-foot-diameter circular area, for instance, which required some creative problem solving — such as designing walk patterns that would fit and look natural in a small space. “It was like a math problem,” Joanna noted. Before she even stepped onto the stage, we worked with Ashley on a piece of circular carpet outside the dressing room area, drawing patterns and arrows on paper as if planning a football play.


The Styles editors Joanna Nikas and Tracy Ma work out walk patterns on the test carpet with Ms. Graham.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES


We also needed to work within specific styling guidelines from the studio. No super-skinny stiletto heels, for example, and hair needed to be contained to a unified shape, as small flyaway strands are confusing to the capture system. While Ashley came with an entourage and stylist in tow, the studio has its own stylist, Harmony Arnold, who knows how to keep the algorithms from getting angry. When Ashley emerged with her hair styled down, it was deemed “risky,” and Harmony helped restyle it into something pulled back.

There are new issues to contend with in the viewer experience as well, like controlling how close the viewer can get to the subject without violating her personal space. We’ve solved this for now by having the hologram go transparent once a certain threshold of closeness is crossed.


Tracy Ma looks at a live video feed of Ms. Graham trying the treadmill walk barefoot.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES



Ms. Graham preparing to shoot the final take in heels.
CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES



Our last shoot of the day was our most treacherous, and potentially ankle-twisting. We’d asked Ashley to walk on a special rail-free treadmill to create a walk cycle, and out of concern for her safety suggested she try it barefoot, which she did. Then, unfazed, she insisted she could do the treadmill walk in heels.

With that, it was a wrap. Shoes already on, Ashley was off to the airport for her next shoot.

To find other augmented reality projects, readers can choose the Immersive AR/VR tab in the NYTimes app.

Original article appears in the New York Times - September 5th, 2018

A Hologram Hits the Runway

In the Times AR team’s first rendition of video that you can walk around, Ashley Graham is “a model in motion without the filters, the Photoshopping and the angles the fashion industry is used to.”


By GRAHAM ROBERTS



Ashley Graham, surrounded by cameras, on a feedback monitor at Metastage, a brand new volumetric video studio at the Culver City Studios complex in Los Angeles.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES


Times Insider delivers behind-the-scenes insights into how news, features and opinion come together at The New York Times.

When we stepped out of an Uber at Culver City Studios in Los Angeles, the place where cultural touchstones such as “Citizen Kane,” “Beetlejuice” and “The Matrix” were filmed, it was still early enough to be dark outside. The eerie quiet of the place at that hour had us half expecting to see the ghosts of Hollywood crews still zooming between sound stages. Or at least a Pepper’s ghost.











Culver City Studios at dawn.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES

We were here, after all, to create our first holographic video recording. (It was the next evolution of our augmented reality app feature, which was introduced in February with “Four of the Best Olympians, as You’ve Never Seen Them,” pegged to the 2018 Winter Olympics in Pyeongchang, South Korea.)

Our subject was the model and activist Ashley Graham. She had a strict cutoff at noon, and we were leaving extra time for things to go wrong.

We would be using Microsoft’s new “volumetric capture” technology — which deploys advanced computer algorithms to compress the nearly 10 gigabytes of information collected every second, and would help us give a new twist to the epithet “newspaper of record.” Ashley, in this brief recorded sequence, would be endlessly available to be projected into the world.

Long a staple of science fiction, the ability to beam somebody into your space in three dimensions (think receiving an urgent message from Leia in “Star Wars”), and walk around them as if they’re actually there, is gradually becoming real.

As part of “Ashley Graham, Unfiltered — an interview conducted by Joanna Nikas, a New York Times Styles editor, that explores issues around body image — readers who have downloaded the New York Times app (iOS) onto their iPhones or iPads can project a “hologram” of Ashley into their space as she demonstrates poses and her runway walk.







This is a new video format, with a new production process. Rather than record Ashley from just a few perspectives, she is captured from all angles using more than 100 cameras positioned around her on a stage that is essentially a giant green-screen in-the-round. The images from the cameras are then composited and processed together to create a single hologram that, when viewed through the screen on your phone or tablet, can be placed into your own physical environment.

Joanna helped choose her as the subject of our first exploration of this new AR form. “Ashley has been a champion for body diversity and positivity for years. She was one of the first curvy women to be on the cover of Sports Illustrated,” she explains. “This technology allows people to see a model in motion without the filters, the Photoshopping and the angles the fashion industry is used to.”

Which is exactly the way Ashley wants it. “You see my lower belly fat, you might be seeing some cellulite, you see the curves of my arms and my butt. You’re seeing all of it,” she told Joanna during her interview. “I’m showing the real me. And I want other women to know that you can show the real you and the real you is perfect, the real you is beautiful.”



The Metastage stylist Harmony Arnold, left, oversees some last-minute legwork before a take.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES


We recorded Ashley Graham at Metastage, a brand new volumetric video studio in the Culver City Studios complex. “Having Ashley as one of our first captures was a testament to how far we’ve come with female empowerment in business,” said the studio’s chief executive, Christina Heller. “We had a strong, vivacious supermodel performing on a state-of-the-art volumetric capture stage run by women. The energy was fun and inclusive, just like it should be.”

As might be expected, the technology is not without its limitations at this early stage. The capture zone is confined to an 8-foot-diameter circular area, for instance, which required some creative problem solving — such as designing walk patterns that would fit and look natural in a small space. “It was like a math problem,” Joanna noted. Before she even stepped onto the stage, we worked with Ashley on a piece of circular carpet outside the dressing room area, drawing patterns and arrows on paper as if planning a football play.








The Styles editors Joanna Nikas and Tracy Ma work out walk patterns on the test carpet with Ms. Graham.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES


We also needed to work within specific styling guidelines from the studio. No super-skinny stiletto heels, for example, and hair needed to be contained to a unified shape, as small flyaway strands are confusing to the capture system. While Ashley came with an entourage and stylist in tow, the studio has its own stylist, Harmony Arnold, who knows how to keep the algorithms from getting angry. When Ashley emerged with her hair styled down, it was deemed “risky,” and Harmony helped restyle it into something pulled back.

There are new issues to contend with in the viewer experience as well, like controlling how close the viewer can get to the subject without violating her personal space. We’ve solved this for now by having the hologram go transparent once a certain threshold of closeness is crossed.







Tracy Ma looks at a live video feed of Ms. Graham trying the treadmill walk barefoot.

CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES



Ms. Graham preparing to shoot the final take in heels.
CREDIT: GRAHAM ROBERTS / THE NEW YORK TIMES



Our last shoot of the day was our most treacherous, and potentially ankle-twisting. We’d asked Ashley to walk on a special rail-free treadmill to create a walk cycle, and out of concern for her safety suggested she try it barefoot, which she did. Then, unfazed, she insisted she could do the treadmill walk in heels.

With that, it was a wrap. Shoes already on, Ashley was off to the airport for her next shoot.

To find other augmented reality projects, readers can choose the Immersive AR/VR tab in the NYTimes app.

Original article appears in Variety - August 14th, 2018

Poppy to Star in Augmented Reality Experience 'A Jester's Tale,' Premiering at Sundance 2019: Exclusive


By TODD LONGWELL


COURTESY PHOTO



'I can't wait to meet you in AR,' the android-pop star says of her collaboration with RYOT and 1RIC.


With an inescapalbe YouTube presence, two full-length albums, more than a million followers on social media and her own online "church," Poppy is already everywhere --and now, the robot-esque pop star is making her way into augmented reality.

Billboard can confirm that Poppy is set to star in the augmented-reality experience A Jester's Tale, premiering at the 2019 Sundance Film Festival as part of the New Frontier program. Viewers will put on AR headsets to be transported into the 12-minute, interactive narrative following a fictional storyline inside a child's bedroom and featuring a slew of character holograms, including Poppy.

A Jester's Tale comes together via RYOT, the Emmy Award-winning content studio under Verizon Media, and 1RIC, the augmented-reality studio led by Asad J. Malik, who recently saw his Terminal 3 AR experience premiere at the 2018 Tribeca Film Festival.

"I am excited to be a part of Sundance again -- this time as a hologram," Poppy says in a statement. "The innovative experience of A Jester's Tale is a whole new form of storytelling and I can't wait to meet you in AR." The star premiered her YouTube Red series I'm Poppy at the Utah film festival in 2018.

"To me, Poppy represents a new kind of cultural critique, one that plays with ideas of fame and manufactured perfection in ways that are unsettling yet enjoyable," says Malik. "What she stands for will bring a new layer of meaning to A Jester's Tale and vice versa. What is real and what is a mere representation has been a foundational idea in the history of media and art. We happen to be living at a time when immersive technology is allowing for literal manifestations of this idea while our social media lives are leading to an increasingly altered sense of reality."

Adds Jake Sally, head of immersive development at RYOT: "We are on the cusp of a fully enabled augmented reality world and Asad embodies the tip of the spear as the first true augmented reality auteur. A Jester's Tale represents the promise of what AR can become, not what it is today, blurring the line between realities to create a stunning reflection of our world in the digital age."



See the viewing calendar and full poster for A Jester's Tale, taking place at the New Frontier at The Ray in Park City below.

Jan. 25 (2PM-9:30PM), Jan. 26 (4PM-7:30PM), Jan. 27 (2PM-9:30PM), Jan. 28 (2PM-9:30PM), Jan. 29 (2PM-9:30PM), Jan. 30-Feb. 1 (12PM-9:30PM) and Feb. 2 (10AM-5:30PM).


COURTESY PHOTO





METASTAGE / ALL RIGHTS RESERVED