Kinfolk: First Augmented Reality Installation at the Museum of Modern Art

Imagine going to a museum and experiencing history, not through exhibits, but in 3D right before your eyes. That's the power of Kinfolk, an augmented reality (AR) app developed by Pariah Interactive for Kinfolk Foundation. Kinfolk was featured at the Museum of Modern Art (MoMA) in New York City as part of the “Architecture Now: New York, New Publics” exhibition from February to July 2023.

Upon entering the third floor of MoMA, there are three wooden pedestals with nothing on them. At first glance, the installation appears incomplete. On closer observation, iPads stationed within the exhibition display large monuments on the pedestals: these are Kinfolk’s digital AR monuments.

Kinfolk debuted three new AR monuments at MoMA, each of whom were an influential presence in New York City: David Ruggles, Seneca Village, and the Young Lords. Museum visitors can walk around the pedestals holding iPads, view the AR monuments from all angles, peer into digital portals, and learn about often overlooked Black and Brown history.

David Ruggles, Seneca Village, and the Young Lords monuments in the Kinfolk app

With the Kinfolk installation as the first-ever AR project at the MoMA, there was an expectation of strong execution. Luna Yixi Wu, developer at Pariah Interactive, considered those pressures. "As the first-ever AR project at MoMA, how can we set the standard of XR Art for the Museum of Modern Art?" Kinfolk's debut carried the weight of ensuring a successful launch, and paving the way for future digital art.

Overcoming Challenges, Embracing Innovation

While the Kinfolk app has been available to the public since 2021, we had to modify the app to be installation-specific for MoMA. This version would be calibrated every morning and set to display stable digital monuments in AR space for 24 hours without a docent present. A docent-free AR installation running for that long was unheard of, but we had to overcome this unique challenge within the technological limitations of augmented reality.

After extensive R&D, we developed a solution for the MoMA-specific Kinfolk app. Prashast Thapan, Pariah's studio director, describes, “By scanning the space where the monument was to be situated, then using cloud computing to create a ‘memory’ of the space for the program, the app remembered precisely where monuments were supposed to be situated in physical space.” With this new solution, the Kinfolk app passed all our internal stability tests. We were able to leave the app running 24 hours while stably displaying multiple digital monuments placed in a site-specific AR space.

The exhibition space in MoMA

Feeling optimistic with our results, we went to MoMA to check out the exhibition space, and we quickly discovered our next hurdle: the large floor-to-ceiling windows that let in abundant natural light. While the views from the window were beautiful, they also caused environmental challenges for the app’s stability because the sun would cast shadows throughout the day into the installation space, changing the appearance of the physical environment. This would confuse the app’s memory of the physical space from its morning calibration compared to the constantly changing exhibition space as the day went on. These changes in lighting would destabilize the AR monuments, as the software’s original reference of its physical space would not match as time passed throughout the day.

The sun would get so bright in the room that we had to wear sunglasses indoors for installation!

With the MoMA exhibition opening in less than a month, our team put our heads together, iterated on the code, and ran numerous QA sessions until we created a stable Kinfolk app experience for MoMA. “We iterated and added a lot of failsafe conditions to the code to make sure [the app] does not crash [or glitch] on prolonged use,” Sahil Sharma, developer, explains. The Kinfolk installation at MoMA would have hundreds of active visitors for extended periods each day, so the app needed to withstand constant usage without docent help. The unique environment of the museum was a challenge in itself, and quite unlike the controlled testing environment at the Pariah studio.

Stepping Outside Our Comfort Zone

The Pariah team, Prash, Luna, and Faith, working on the gallery floor.

Stepping outside the comforts of our studio in Brooklyn, we were now working directly out of the MoMA. We found ourselves camping out on the gallery floor, fine-tuning our code, and pushing test builds to ensure the AR monuments were placed precisely on their corresponding pedestal, while adapting to the ever-changing environment. We used every available second in the exhibition space, wrapping up minutes before the journalists came to check out the installation for the press review. At this point, the MoMA-specific version of the app was done, but we still had another hurdle to jump over: updating the Kinfolk app for the iOS App Store and Google Play Store.

Installation in process!

Crossing the Finish Line

The Kinfolk installation was unique because people were able to experience the AR monuments both at MoMA– on iPads stationed next to the pedestals– and on their personal devices. In tandem with the MoMA version of the Kinfolk app, we also needed to update the public version of the Kinfolk app, adding a layer of complexity to our production and development.

With each version of the app having a different purpose, we had to be diligent while developing and testing. The upgraded Kinfolk public app debuted a new “scan mode” feature where users can scan a real-world image to spawn a corresponding AR monument. This differed from the Kinfolk MoMA experience where users could not scan the wooden pedestals themselves; instead, the MoMA app would be pre-calibrated with the AR monuments situated on the pedestals. "Maintaining consistent quality across the two distinct Kinfolk versions was crucial,” recalls Malvika Mital, producer. “We had to…focus on user experience, functionality, and accessibility across different devices and environments.” As we worked on both versions of the Kinfolk app at the same time, it was a challenge to switch back and forth, constantly considering the different audience and user experience for each app.

More Than Just an App

Once the MoMA exhibit opened, we got to see the public interact with the Kinfolk installation: walking around the space and looking at the AR monuments. It was rewarding to hear the excited buzz around the installation that we poured so much time and energy into. We saw how Kinfolk transcends technology: touching hearts, celebrating underrepresented narratives, and connecting people.

Toussaint L’Ouverture wallwork in the exhibition space. Photograph by Robert Gerhardt (IN2515.7)

At the MoMA exhibit, there was a giant wallwork image of Toussaint L’Ouverture, a Haitian general and leader of the Haitian Revolution. A Haitian MoMA security guard felt deeply moved to see L’Ouverture featured. He returned to the installation multiple times to stand in front of the L’Ouverture’s image and take it all in. In another instance, a journalist 's eyes lit up at seeing the Young Lords AR monument because their grandmother was part of the Young Lords. The digital monument allowed the journalist to see their grandmother’s impact on civil rights be recognized.

The Young Lords monument at the Kinfolk MoMA installation

Kinfolk is more than an augmented reality app or MoMA installation. It is a vehicle for revealing and highlighting the hidden histories of those who are overlooked. Faith Zeng, producer, shares, “As a person of color, I understand the joys of feeling seen when my culture is represented and celebrated by the public, so it is really rewarding to bring the same feeling to others through this project.” Kinfolk's MoMA exhibition resonated on a profoundly human level and is a conversation about accessible history and inclusive storytelling. The Kinfolk AR experience, beyond a technical feat, became a bridge between the past and the present, fostering understanding, and strengthening underrepresented narratives.

Bonus images of the Pariah team and Kinfolk team at the opening reception for the exhibition "New York, New Publics"

Download Kinfolk Today

Try out the Kinfolk app on iOS App Store and Google Play Store.

Pariah Interactive Team * Technical Direction: Prashast Thapan * App Development: Rishabh Kukreti, Sahil Sharma, YiXi “Luna” Wu * Production & QA: Malvika Mital, Faith Zeng * Additional 3D Art Help: Stephanie Davila