Parents know the drill well. It’s bedtime, the kids are going to sleep, but tonight, they don’t want to just hear you read from a book. They want to come up with a story with music and visuals. Your brain starts sifting through characters they like and plausible plots avoiding dead ends. What if a new feature on a smart device could help?
Such a feature now exists on Amazon’s Echo Show devices. “Create with Alexa” uses advances in conversational and generative artificial intelligence (AI) to empower young storytellers to build unique stories with a narrative arc, colorful graphics, and fun, complementary background music. The animated stories then come to life on the screen of your Echo Show devices.
“It’s an experience where Alexa becomes your child’s co-creator,” said Nico Bishop, a UX designer whose team builds the customer experience for “Create with Alexa.”
Children can say the simple phrase, “Alexa, make a story,” follow a few prompts, and Alexa takes over—generating a unique and original narrative based on the child’s preferences. For each scene, the AI also composes an illustration, background music, and sound effects. The child’s choices of character, setting, and other variables dictate the direction the story goes—and no two stories are the same.
For example, once a child has told Alexa they’re ready to create, they select a theme, which could be “space exploration,” “underwater,” or “enchanted forest.” They then choose and name a character—say, an astronaut named Speedy or an alien named Fuzzy—choose a color theme, and pick adjectives like “silly” or “happy” or “mysterious.” With those few inputs, Alexa works behind the scenes to create a five- to 10-line story, told across five unique scenes that are complete with sound and visual effects. Even if the child chooses the same prompts in the future, the story will be different each time. They can then save the story in their personal media gallery to experience it again, and coming soon, share it with their loved ones.
Kids have long loved listening to stories, and with “Create with Alexa” they can also actively co-create stories of their own.
Amazon came up with the idea for “Create with Alexa” the same way it came up with ideas for most products; the team started with the customer and worked backwards. “We’ve always envisioned Alexa to be a trusted companion that brings joy and fun to the family,” said Eshan Bhatnagar, head of product for Alexa AI. “And we created this experience to further that vision.”
The act of creating stories has become part of the regular bedtime routine for Bhatnagar and his 6-year-old son, Reyaansh. “He was one of our early and most demanding beta testers,” Bhatnagar said. “He’s helped us uncover bugs and, at the same time, impatiently demanded more themes and characters to be introduced—‘Pa, I want dinosaurs next’ my son tells me.”
As a father, Bhatnagar knew it was essential to make the experience safe for young kids. “From the get-go, we used carefully curated data sources to train AI models,” he said. “We have multiple guardrails such as content filtering and curated prompts to ensure this experience is both delightful and safe.” He said privacy is also key. As with any child-directed Alexa feature, “Create with Alexa” requires verifiable parental consent before it’s enabled on a device.
“The moments my son and I spend together creating a fun story are priceless,” Bhatnagar said. “They inspire me not just to improve the ‘Create with Alexa’ experience but, more broadly, to bring the power of generative AI to unlock future possibilities.”
The entire experience is made possible by innovations in conversational and generative AI that help build the narrative, scenery, and sound. A language model trained on written stories generates the text of the story. It is trained to build a cohesive story by generating text from just a few key inputs.
A scene-generation model illustrates the story by selecting a background image created by AI or human artists. It also adds in relevant objects from the narrative (e.g., pirate, jellyfish, seaweed) to locations, orients gestures (e.g., waving hands), and sets facial expressions (e.g. laughing, crying).
Finally, Alexa’s AI sound and music generator produces relevant sounds and complementary background music. The model draws from a large library of instrumental parts, including artist-created chord progressions, harmonies, and rhythms. An AI musical-arrangement system ensures that all the pieces fit together.
To help build and improve the “Create with Alexa” experience, the researchers sought feedback from parents and kids in the target age group, ages 4-8, who provided discerning feedback. “Everything they said, we took very seriously,” Bishop said. “The parents and their kids had very interesting and innovative ideas, and they were not shy to share them.”
As the team worked to make the “Create with Alexa” experience a reality, they continued to improve the technology and experience. For example, “At first we didn’t think we would have contextual music generation,” Bishop said. But, she said the Alexa science team invented on behalf of the customers to enable each story to have unique AI-generated background music based on the thematic elements of the story. Learn more about the science behind it.
According to Bishop, this kind of AI-assisted story creation is just scratching the surface of what the technology can do. “Kids will want to be able to say, ‘I want to see a wall of chocolate,’ and soon we expect we’ll be able to do that,” she said. She added that voice recognition competency is improving all the time, along with improvements in graphic latency, sound effects, and music capabilities.
Bhatnagar said that “Create with Alexa” started at the crossroads of creativity and AI. “We envision a world where anyone can bring their ideas to life in the form of digital creations just by using a few spoken words,” he said. “And Alexa will be right with them, assisting them as their co-creator.”
Families can start creating with Alexa on November 29, with a variety of free templates. “Create with Alexa” is available in English on supported Echo devices throughout the U.S.