Is This A.I. Thanksgiving Menu Better Than a Human’s?

Recipes can be passed down by word of mouth, scribbled on index cards, published in cookbooks. But they’ve always had one thing in common: They’re made by people.

Few things, in fact, are as packed with humanity as a recipe. Mixed and folded and baked into each one are their creators’ backgrounds, stories, tastes and feelings.

People have their limitations, though. They can’t read every mashed potato recipe on the internet before coming up with their own version. They can’t analyze thousands of techniques in search of the best way to make a pie crust.

These recipes have all the components of their handmade forebears: lists of ingredients, precise measurements, step-by-step instructions and introductory notes with (fabricated) personal touches. Their advantage, in theory, is that they draw on a vast trove of online information about food and cooking.

But are they any good? And can they improve on millenniums’ worth of lived culinary experience?

As home cooks, professional chefs and food-magazine editors know, the ultimate test for recipes is Thanksgiving dinner, a sprawling, varied spread that invites high expectations.

So we decided to enlist artificial intelligence — in this case, a technology called GPT-3 — to devise a holiday menu, which we then prepared and presented to a corps of taste testers: four New York Times cooking columnists.

The results say a lot about the technology’s potential, and the very purpose of a recipe.

Before we get to the verdict, let’s explain the science. Designed by OpenAI, one of the world’s most ambitious artificial intelligence labs, GPT-3 is a neural network, a mathematical system that can learn skills by analyzing enormous amounts of data.

Some systems study images; in September, an A.I.-generated work took the top prize at a state-fair art contest. GPT-3 analyzes digital text, including books, Wikipedia articles, tweets, chat logs, computer programs and, yes, recipes. It can identify billions of distinct patterns in the way people connect words, numbers and symbols, and then use that knowledge to generate its own content — like a Thanksgiving menu of original recipes.

Artificial intelligence is poised to reshape several fields, from email marketing to computer programming. Recipe writing is not a common area of study, but a handful of researchers, including a team at the Massachusetts Institute of Technology, have begun to explore whether A.I. can master the skill.

In 2016, Janelle Shane, an optics research scientist who runs a machine-learning humor blog called AI Weirdness, began using systems like GPT-3 to create recipes, and then posted them. Early versions of the technology, she said, produced recipes that were a bit, well, weird. They called for nonsensical ingredients like “peeled rice” or “chopped flour.”

Today, she said, many A.I. recipes can seem indistinguishable from human-made ones.

“What it does really well is sound plausible,” Dr. Shane said. “So if you weren’t paying attention and someone was just reading this recipe out loud to you, you would be, ‘Oh yeah, that sounds like a perfectly ordinary recipe.’”

To create our A.I. Thanksgiving menu, we began by introducing ourselves to the GPT-3 system — in a surprisingly human way.

Mark Chen, an OpenAI research scientist, advised me, Priya Krishna, to get personal. Tell the system about yourself, he said: your family background, what flavors you like, which ingredients you tend to use often.

“The more details you provide in the prompt,” he said, “generally, the better the model performs.”

So after logging into GPT-3 on my laptop, I typed: “I am originally from Texas, and I grew up in an Indian American household. I love spicy flavors, Italian and Thai food, and desserts that are not too sweet. Some ingredients I frequently cook with are chaat masala, miso, soy sauce, herbs and tomato paste.”

Then I wrote, “Show me a Thanksgiving menu made for me.”

The first recipe GPT-3 produced was called “pumpkin spice chaat.” I was confused by the concept yet impressed by the inventiveness.

I asked follow-up questions to spur GPT-3’s creativity: Show me a few desserts tailored to my taste preferences. Show me a nontraditional Thanksgiving recipe. Show me a recipe for cranberry sauce that’s not too sweet and a little spiced.

Minutes later, I had a complete menu that seemed both plausible and intriguing: pumpkin spice chaat, green beans with miso and sesame seeds, naan stuffing, roasted turkey with a soy-ginger glaze, cranberry sauce that’s not too sweet and a little spiced (yes, that’s the full recipe name) and pumpkin spice cake with orange cream cheese frosting.

The dishes looked appetizing enough. We used DALL-E, another OpenAI system that generates images, to create photos for each one. And we asked GPT-3 to provide introductions for each recipe, written from my point of view: “This roasted turkey recipe is inspired by the flavors of my childhood.” (It was not.)

Some of the ingredient lists seemed questionable. The naan stuffing called for 32 different components, including two cups of dried fruit. Most of the recipes were suspiciously light on salt and fat. Still, I was hopeful.

Cooking and tasting the recipes all but dashed that hope.

The cake was dense and more savory than sweet. The naan stuffing tasted like a chana masala and a fruitcake that had gotten into a bar fight. The roast turkey recipe called for a single garlic clove to season a 12-pound bird, and no butter or oil; the result was dry and flavorless.

The chaat, laced with cilantro and baking spices, was a grassy-flavored mush. The green beans and the cranberry sauce were edible but unremarkable.

Our taste-tasting columnists were even less kind.

“We’re not out of a job,” Melissa Clark said. “I don’t feel anything eating this food,” Yewande Komolafe added.

Genevieve Ko summed it up best: “There is no soul behind it.”

The recipes provided few clues on what cooks should look or smell for during the process, and no reasoning for why ingredients were added in a particular order.

Even before the tasting, Dr. Shane, the optics research scientist, suggested lowering our expectations. She called A.I.-generated dishes “the recipe equivalent of hotel room art.”

Yet the technology, even at this early stage, can have its uses. Mr. Chen, the OpenAI scientist, said it could help provide home cooks with some basic inspiration.

“You’ve got something in mind you want to make, you’re not quite sure how to make it, or you have a set of ingredients and I’m not quite sure how to combine it,” he said. “This could be a very quick way to give you a lot of different ideas.”

GPT-3 could also help to personalize an existing recipe by creating a version that’s less spicy, or that incorporates specific flavors, he said.

But the cookbook author Nik Sharma, whose work intertwines food and science, worries about the inherent biases of these programs. They draw on the English-language internet, where Western recipes predominate. And when these programs do suggest non-Western dishes, he added, those may be the more generic ones familiar to Americans, like tandoori chicken.

“As food writers, our goal is to drive people into a new direction, to help challenge them, to help them understand the world,” he said. With artificial intelligence, “how do you do that?”

A.I. has already worked its way into the act of cooking. Home cooks use search engines to find recipes and ask virtual assistants like Amazon Alexa to convert teaspoons to tablespoons.

This technology is not a replacement for people, at least so far. It can nudge cooks in one direction or another. But it is still humanity — with its intuition, storytelling and warmth — that drives a good recipe.

The individual behind the dish is just as important as the recipe itself, Dr. Shane said. There is value in knowing that someone took the time to perfect a version of pie, stuffing or turkey, and share it.

“You want to know that these recipes mean something to somebody,” she said. That’s something artificial intelligence may never be able to provide.