Original article appears in the New York Times - September 5th, 2018

A Hologram Hits the Runway

In the Times AR team’s first rendition of video that you can walk around, Ashley Graham is “a model in motion without the filters, the Photoshopping and the angles the fashion industry is used to.”


Ashley Graham, surrounded by cameras, on a feedback monitor at Metastage, a brand new volumetric video studio at the Culver City Studios complex in Los Angeles.


Times Insider delivers behind-the-scenes insights into how news, features and opinion come together at The New York Times.

When we stepped out of an Uber at Culver City Studios in Los Angeles, the place where cultural touchstones such as “Citizen Kane,” “Beetlejuice” and “The Matrix” were filmed, it was still early enough to be dark outside. The eerie quiet of the place at that hour had us half expecting to see the ghosts of Hollywood crews still zooming between sound stages. Or at least a Pepper’s ghost.

Culver City Studios at dawn.


We were here, after all, to create our first holographic video recording. (It was the next evolution of our augmented reality app feature, which was introduced in February with “Four of the Best Olympians, as You’ve Never Seen Them,” pegged to the 2018 Winter Olympics in Pyeongchang, South Korea.)

Our subject was the model and activist Ashley Graham. She had a strict cutoff at noon, and we were leaving extra time for things to go wrong.

We would be using Microsoft’s new “volumetric capture” technology — which deploys advanced computer algorithms to compress the nearly 10 gigabytes of information collected every second, and would help us give a new twist to the epithet “newspaper of record.” Ashley, in this brief recorded sequence, would be endlessly available to be projected into the world.

Long a staple of science fiction, the ability to beam somebody into your space in three dimensions (think receiving an urgent message from Leia in “Star Wars”), and walk around them as if they’re actually there, is gradually becoming real.

As part of “Ashley Graham, Unfiltered — an interview conducted by Joanna Nikas, a New York Times Styles editor, that explores issues around body image — readers who have downloaded the New York Times app (iOS) onto their iPhones or iPads can project a “hologram” of Ashley into their space as she demonstrates poses and her runway walk.

This is a new video format, with a new production process. Rather than record Ashley from just a few perspectives, she is captured from all angles using more than 100 cameras positioned around her on a stage that is essentially a giant green-screen in-the-round. The images from the cameras are then composited and processed together to create a single hologram that, when viewed through the screen on your phone or tablet, can be placed into your own physical environment.

Joanna helped choose her as the subject of our first exploration of this new AR form. “Ashley has been a champion for body diversity and positivity for years. She was one of the first curvy women to be on the cover of Sports Illustrated,” she explains. “This technology allows people to see a model in motion without the filters, the Photoshopping and the angles the fashion industry is used to.”

Which is exactly the way Ashley wants it. “You see my lower belly fat, you might be seeing some cellulite, you see the curves of my arms and my butt. You’re seeing all of it,” she told Joanna during her interview. “I’m showing the real me. And I want other women to know that you can show the real you and the real you is perfect, the real you is beautiful.”

The Metastage stylist Harmony Arnold, left, oversees some last-minute legwork before a take.


We recorded Ashley Graham at Metastage, a brand new volumetric video studio in the Culver City Studios complex. “Having Ashley as one of our first captures was a testament to how far we’ve come with female empowerment in business,” said the studio’s chief executive, Christina Heller. “We had a strong, vivacious supermodel performing on a state-of-the-art volumetric capture stage run by women. The energy was fun and inclusive, just like it should be.”

As might be expected, the technology is not without its limitations at this early stage. The capture zone is confined to an 8-foot-diameter circular area, for instance, which required some creative problem solving — such as designing walk patterns that would fit and look natural in a small space. “It was like a math problem,” Joanna noted. Before she even stepped onto the stage, we worked with Ashley on a piece of circular carpet outside the dressing room area, drawing patterns and arrows on paper as if planning a football play.

The Styles editors Joanna Nikas and Tracy Ma work out walk patterns on the test carpet with Ms. Graham.


We also needed to work within specific styling guidelines from the studio. No super-skinny stiletto heels, for example, and hair needed to be contained to a unified shape, as small flyaway strands are confusing to the capture system. While Ashley came with an entourage and stylist in tow, the studio has its own stylist, Harmony Arnold, who knows how to keep the algorithms from getting angry. When Ashley emerged with her hair styled down, it was deemed “risky,” and Harmony helped restyle it into something pulled back.

There are new issues to contend with in the viewer experience as well, like controlling how close the viewer can get to the subject without violating her personal space. We’ve solved this for now by having the hologram go transparent once a certain threshold of closeness is crossed.

Tracy Ma looks at a live video feed of Ms. Graham trying the treadmill walk barefoot.


Ms. Graham preparing to shoot the final take in heels.

Our last shoot of the day was our most treacherous, and potentially ankle-twisting. We’d asked Ashley to walk on a special rail-free treadmill to create a walk cycle, and out of concern for her safety suggested she try it barefoot, which she did. Then, unfazed, she insisted she could do the treadmill walk in heels.

With that, it was a wrap. Shoes already on, Ashley was off to the airport for her next shoot.

To find other augmented reality projects, readers can choose the Immersive AR/VR tab in the NYTimes app.