“This is what your brain was doing!” a Lincoln Center staffer said to Shanta Thake, the performing arts complex’s artistic director, while swiping through some freshly taken photos.
It was the end of a recent rehearsal at Alice Tully Hall for “Song of the Ambassadors,” a work-in-progress that fuses elements of traditional opera with artificial intelligence and neuroscience, and the photos did appear to show Thake’s brain doing something remarkable: generating images of flowers. Bright, colorful, fantastical flowers of no known species or genus, morphing continuously in size, color and shape, as if botany and fluid dynamics had somehow merged.
“Song of the Ambassadors,” which was presented to the public at Tully on Tuesday evening, was created by K Allado-McDowell, who leads the Artists and Machine Intelligence initiative at Google, with the A.I. program GPT-3; the composer Derrick Skye, who integrates electronics and non-Western motifs into his work; and the data artist Refik Anadol, who contributed A.I.-generated visualizations. There were three singers — “ambassadors” to the sun, space and life — as well as a percussionist, a violinist and a flute player. Thake, sitting silently to one side of the stage with a simple, inexpensive EEG monitor on her head, was the “brainist,” feeding brain waves into Anadol’s A.I. algorithm to generate the otherworldly patterns.
“I’m using my brain as a prop,” she said in an interview.
Just to the side of the stage, level with the musicians, sat a pair of neuroscientists, Ying Choon Wu and Alex Khalil, who had been monitoring the brain waves of two audience volunteers sitting nearby, with their heads encased in research-grade headsets from a company called Cognionics.
Wu, a scientist at the University of California, San Diego, investigates the effects of works of art on the brain; in another study, she’s observing the brain waves of people viewing paintings at the San Diego Museum of Art. Khalil, a former U.C. San Diego researcher who now teaches ethnomusicology at University College Cork in Ireland, focuses on how music gets people to synchronize their behavior. Both aim to integrate art and science.
Read More on Artificial Intelligence
Which makes them a good match for Allado-McDowell, who first pitched “Song of the Ambassadors” in January 2021 as a participant in the Collider, a Lincoln Center fellowship program supported by the Mellon Foundation. “My proposal was to think about the concert hall as a place where healing could happen,” said Allado-McDowell, 45, who uses the gender-neutral pronouns “they” and “them.”
Healing has long preoccupied them. They suffered from severe migraines for years; then, as a student at San Francisco State University, they signed up for a yoga class that took an unexpected turn. “I was besieged by rainbows,” they recalled in a forthcoming memoir. “Orbs of light flickered in my vision. Panting shallow breaths, I broke out of the teacher’s hypnotic groove and escaped to the hall outside. As I knelt on the carpet, cool liquid uncoiled in my lower back … as a glowing purple sphere pulsed gold and green in my inner vision.”
This, they were told, was a relatively mild form of kundalini awakening — kundalini being, in Hindu mythology, the serpent that is coiled at the base of the spine, a powerful energy that generally emerges from its dormant state only after extensive meditation and chanting. Others might simply have dropped yoga. “For me, it was an indication that I didn’t understand reality,” Allado-McDowell said. “It showed me that I didn’t have a functional cosmology.”
What followed was a yearslong quest to get one. Along the way, they picked up a master’s degree in art and went to work for a Taiwanese tech company in Seattle. At one point, while sitting in a clearing in the Amazon rainforest, they had a thought: “A.I.s are the children of humanity. They need to learn to love and to be loved. Otherwise they will become psychopaths and kill everyone.”
Later, in 2014, Allado-McDowell joined a nascent A.I. research team at Google. When the leader suggested collaborations with artists, they volunteered to lead the initiative. Artists and Machine Intelligence was launched in February 2016 — 50 years after “9 Evenings: Theater and Engineering,” the pioneering union of art and technology led by Robert Rauschenberg and the AT&T Bell Labs engineer Billy Kluver. The connection was not lost on Allado-McDowell.
One of the earliest partnerships they established was with Anadol: first for “Archive Dreaming,” a project inspired by the Borges story “The Library of Babel,” then for “WDCH Dreams,” Anadol’s A.I.-driven projection onto the billowing steel superstructure of the Frank Gehry-designed Walt Disney Concert Hall in Los Angeles. For “Song of the Ambassadors,” Anadol said, “we are transforming brain activities in real time into an ever-changing color space.”
Anadol’s artwork also responds to Skye’s music, which alternates between periods of activity and repose. “We wanted to bring people in and out of a space of meditation,” Skye said. “I carved out these long gaps where all we’re doing is environmental sounds. Then we slowly bring them out.”
All this is tied to Allado-McDowell’s goal of testing the therapeutic powers of music in a performance setting. “Might there be policy implications?” they asked. “Might there be a role that institutions could play if we know that sound and music is healing? Can that open up new possibilities for arts funding, for policy, for what is considered a therapeutic experience or an artistic experience?”
The jury is still out.
“We know that listening to music has an immediate impact for things like mood, attention, focus,” said Lori Gooding, an associate professor of music therapy at Florida State University and president of the American Music Therapy Association. Positive results have been found for people who have suffered a stroke, for example — but that’s after individualized therapy in a medical or professional setting. The approach in “Song of the Ambassadors,” she said, is different because of “the public aspect of it.”
Wu and Khalil, the neuroscientists involved with the production, have yet to analyze their data. But at a panel discussion preceding Tuesday’s performance — and yes, this opera did come with a panel discussion — Khalil made a prediction that left the audience cheering.
“We’ve started to understand that cognition — that is, the working of the mind — exists far outside our head,” he said. “We used to imagine that the brain is a processor and that cognition happened there. But actually, we think our minds extend throughout our bodies and beyond our bodies into the world.”
With music, he continued, these extended minds can lock onto rhythms, and through the rhythms onto other minds, and then onto yet more. As for the spaces where that happens, Khalil said, “You can start to think of them as healing places.”