Each month, Future Tense Fiction—a series of short stories from Future Tense and ASU’s Center for Science and the Imagination about how technology and science will change our lives—publishes a story on a theme. The theme for October–December 2019: artificial intelligence.
“I haven’t told them yet,” Shristi said, looking out over the park, “but I think we’re going to have to replace Naneen.”
“Oh, no!” Karen responded, with obliging dismay.
“Maybe it’s for the best,” Blaise said. She was digging around for something in her enormous purse. “You can get a new one. A little more advanced.” She glanced up, anti-wrinkle spritzer in hand. “How long have you had Naneen anyway?”
Shristi smiled. “Decades. She was my nanny first.”
The other two gasped appreciatively. “She was yours?” Blaise asked. “I always thought she was just, kind of, retro-styled.”
Shristi shook her head. “My brother is 10 years younger, so she took care of him, and she stayed with him for a long time, helping him with his homework well into high school, and then … Well, my parents shut her down for a while, but they never got rid of her, just kept her in storage. And then when Esme was born my dad, you know, he used to work on cooking A.I.s? So he was able to tweak some of the newer software to make it compatible with her operating system.”
“That’s amazing,” Karen said, holding her hand out for a turn with Blaise’s spritzer. “So sweet.”
“Mmm.” Shristi said. Now that she had told the story, she wondered if it sounded like they had been cobbling together a frankennanny to save money. “I had such fond memories of her, you know? And maybe because my brother is so much younger, so she was with us a really long time … ”
“It makes sense, too,” Blaise said. She sounded supportive, although Shristi knew that her kids’ nanny, Satella, was the absolute latest model and got updated every month. “Naneen remembering how you grew up must be useful in raising your children.”
“Kind of. I mean yes, there are some very sweet moments.” Naneen posing Esme in exactly the same pose, in the same dress that Shristi had worn 28 years earlier. Naneen reminding Shristi’s parents about the snacks she used to like so they can prepare the same for Esme and Sonal. “But lately there have been a few times when … ” It felt disloyal to talk about this, even with her friends.
“Well … little things at first. We’ve replaced her snack drawer three times because it kept getting stuck, and finally they told us not to bother anymore, because the problem is in her chassis, it’s just slightly warped and—anyway. And she’s slower than the new models. I worry about whether she could make it across the playground in time to catch them, if one of them fell.” Blaise and Karen were nodding; these were all solid reasons for nanny replacements. “But also,” Shristi sighed, and admitted it. “She’s gotten confused. She forgets that they’re not me.”
“Wow. I didn’t realize that could happen to nannies.” Karen leaned back against the bench. “I mean … they don’t get old or sick, right?”
“Just because they’re not human doesn’t mean they don’t age,” Blaise said. “I mean, their components get older and, and, worn out. Right?”
Shristi nodded. “Yeah. We looked at upgrades, but my dad’s not up to doing it anymore, so … in any case, they said at this point replacing that much would fundamentally change her personality.” And cost nearly as much as a new nanny.
“But she still works, right?” Blaise tucked a strand of hair—smooth with the latest shimmer mousse—behind her ear. “You’re not worried about—”
“Oh no,” Shristi said. “I mean, like I said before, she’s slowed down a little, and she sometimes gets the kids’ names wrong, but we haven’t seen any shift in protocol or any problems with her settings.” Most nannies came with a default of total attention to kids, with a list of override words or specific conditions that parents could set to revert their attention to authority figures. “She’s always focused on the girls, still has the routines we set up for her … ” She paused, trying to be fair, and added reluctantly, “There have been a few occasions when I had to repeat instructions, though.”
“She forgot new instructions?” Blaise asked, shocked.
“Well, it was more like … the input didn’t quite take the first time. Once we got it in, it was fine, but … ”
Blaise shook her head. “Time for a change,” she decreed. “There’s no reason you should have to go through that.”
“Nanny upgrades are so hard on the kids though,” Karen said. “Gardette is sooooo much better than our old nanny, but it still took them a long time to get over the change.”
“How is she better?” Shristi asked.
“Looking at new models?” Blaise’s voice sounded almost triumphant. “Because Satella is amazing.”
“We haven’t decided yet,” Shristi said. “Sonal is 6—we’re wondering if it’s really worth it to hire someone now and then have to go through it all again when we finally wean them off the nanny … ”
“So, what, you’re going to pick them up from school every day? Make all the snacks? Read every bedtime story?”
“Would that be so bad?”
“You’d get tired of it so fast,” Blaise said dismissively. Shristi bit back a how do you know? Blaise was not going to listen to anyone else’s opinion right now. “Besides, you need some time to yourself.”
“Well, in case you do,” Karen cut in, probably realizing how annoyed Shristi was getting. “Gardette has a really good medical diagnostic program, which is great for peace of mind; she came with an expanded cookbook and her baking is really good, like, I don’t know how she does it; and she also has this module with, like, games from around the world, so the kids are enjoying a ton of stuff I would never know to teach them.”
“How many languages does she speak?” Blaise asked.
“Only—it was, like—35 or something? I think? But we figured it’s not realistic to have the kids learning more than four or five at a time, no matter what the studies say.”
“Alia and Nathan are doing six,” Blaise said. “I wouldn’t recommend it for everyone, but Satella is really good at monitoring their progress and balancing it out.”
Karen exchanged a look with Shristi that said I knew she was setting me up for some one-upping.
“But all that is software,” Shristi said, in an attempt to reclaim the conversation.
“Software’s the most important part!” Karen answered.
Shristi shrugged, and took a swig from her bottled artichoke tea. “I guess I was spoiled, since my dad could usually customize software on whatever we needed. Do you remember the Mesta model, when we were … I don’t know, in kindergarten? With those extendable arms? I was so jealous of my friends who had her because she could help them do the monkey bars.”
“Oh, Gardette has a very robust physicality,” Karen said eagerly. “I think her arms extend to 10 feet or so, and she has plush blanket wings she can deploy for snuggling.”
“That’s great. You know, Naneen used to give the best hugs, and then I feel like there was this period when the new models weren’t really doing hugs as much.”
“It’s true,” Karen nodded. “There was a thing about how hugs should come from parents, it was too much simulated emotion for kids’ development, blah blah blah. It’s a little better now, but even so, Gardette’s arms are very much, like, blankets, and not, you know, hugging appendages.”
“Satella can fly,” Blaise said. They both looked at her. “OK, only for very short distances, but the kids love it.”
Shristi swallowed a smile. “I guess I feel like they’re getting too old for the most innovative new things.” She looked across the playground, to where Esme was on the monkey bars and Sonal was climbing to the highest slide. “It’s just … it’s a big investment—not money, I mean their emotional investment—only for them to have to transition out of having a nanny so soon.”
“You could go with a monitor instead?” Karen suggested, but even she didn’t sound enthusiastic about it. Shristi only knew one person who used a monitor instead of a nanny: Jinny, who worked part-time and lived in an apartment.
“You know what I heard?” Blaise asked, leaning in. “I heard Valeria is getting a human nanny.”
Shristi opened her mouth to say she had been thinking about that as an option.
“Show-off,” Karen muttered.
“She’s very extreme,” Blaise said, flipping her hair over her shoulder. “The school her kids go to has 50 percent human teachers. At least, that’s what their brochure says.”
“Why would you even want that?” Karen asked, disgusted. “I mean, a nanny is one thing, but when you’re talking about knowledge and prepping for standardized tests it just seems ridiculous.”
“At least with humans,” Shristi started, but stopped as Sonal ran up to her, the ribbons Naneen had braided into her hair streaming behind. “What’s up, sweetie?”
“Mom,” Sonal said, breathless, “can we have some of the ginger cookies that Gardette brought for Jason and Zoe?” Shristi glanced back at the playground, where Naneen raised a hand and signaled a smile in greeting. “Nini said we should ask you,” Sonal added, catching the exchange.
“Yes, that’s fine,” Shristi said, running a hand over her daughter’s hair. “Just one each, mind!” she called, as Sonal dashed back.
There was a silence. “Don’t you have Naneen set for autonomous nutrition evaluation?” Karen asked at last.
“Yes,” Shristi sighed. “That was another glitch. Sometimes it’s just easier to work around the glitches, though, than to get used to a whole new system.” In the periphery of her vision, Karen and Blaise exchanged glances.
Karen put a hand on Shristi’s shoulder. “It’s probably time.”
“Yeah.” To her surprise, Shristi felt her eyes filling. “It’s just … the kids will miss her so much.” But as she repeated that, she remembered Esme, a few months ago, asking if they could get a new nanny, one like her friend’s that had juice dispensers built into her carapace.
Kids were like that. They would ask for something new without realizing it meant getting rid of the old thing, without realizing it meant irrevocable change. It didn’t mean they wouldn’t sob when Naneen wasn’t there to comfort them when they woke in the night or record their stories about what had happened at school.
“They do get over it,” Karen told her. “Kids love new things.”
“To be honest,” Blaise said softly, “it’s probably harder on us.”
Was it? Shristi looked up.
“You know,” Blaise said. “Kids growing up. Our investment in our nannies, the way we trust to them to impersonate us for a while … ”
“Impersonate us?” Karen was laughing, but Shristi was stuck on Blaise’s words. Naneen felt like continuity, something tying together her childhood and her children’s across all the drastic changes time had forced in the 28 years between. She remembered when she had first dusted off Naneen, after Esme was born. She’d been expecting that moment since before she got pregnant, a smug knowledge that she had a nanny she already knew and trusted all set for her family, instead of going through the stressful shopping process so many of her friends had complained about.
But as she was waiting for Naneen to boot up, she found herself worrying not about whether the aging hardware was still functional, but about how she would talk to her. Shristi was going to have to explain that she wasn’t the child anymore, that now she was the mother, and in that moment—her nerves crackling with guilt over leaving the days-old baby alone, even though she could hear Esme’s sleeping breath from the nursery through her earpiece—she didn’t feel like the mother at all.
As it turned out, she needn’t have worried, at least about Naneen’s side of things. Naneen immediately recognized her, but apparently her system architecture included clearly defined categories for parent and child that depended on physical characteristics like age and hormones, rather than unique identity.
“Maybe we can pull out her hard drive,” she said, although it felt like capitulation. “Keep her around somehow … ”
Karen rubbed her shoulder. “Sure you can,” she said. “And if you don’t end up consulting her as often as you imagine you will, you can always clear out the disk space then.”
“So ghoulish,” Blaise said, but it sounded more like a joke than a shudder. “It’s hard, Shris, but you have to accept that her usefulness is over. It helps to remember that she’s a machine. Would you keep a driver around that wasn’t up to doing its job?”
Shristi, whose husband had been trying to talk her into upgrading their driver for months, looked down at her toes. Their polish was chipping. She’d heard about new nanny models that included mommy-care functionality for when the kids didn’t need them: pedicures, manicures, massages, making tea, even a listening function. She imagined for a second that Blaise and Karen had been replaced by autonomous intelligent nonhuman friends, sitting beside her on the bench every day, chatting inconsequentially, picking up all their cues from her instead of being distracted by their own problems.
Blaise was still talking, on and on about various machines she’d replaced over the past year. Karen was nodding along with her while sending eye-rolls toward Shristi that weren’t as subtle as she thought. Shristi no longer felt like being there. She tapped her watch without looking at it, sending a signal to Naneen that it was time to go. Without looking directly at Naneen, she registered her giving the twitch that indicated a message received. From where they were on the bench, they could all hear Naneen give the kids their five-minute warning. Shristi sighed, as if reluctant, and began her usual getting-ready-to-leave rituals. She wondered if a new model would let her perpetrate this kind of deception. One time when Karen had wanted to leave, her previous model—what was her name?—had replied to her message with the weather forecast and their planned schedule for the day, and Karen had gone bright red. “OK, maybe going home now isn’t the perfect choice for the kids,” she had complained, “but I’m ready!”
Of course, the nannies could take the kids to the park themselves; that’s what they were for. But Shristi and Karen and Blaise always came with them and hung out, at least when the weather was good. Playgrounds were supposed to be the fun part of parenting, especially when you weren’t responsible for playing chase or hauling kids up to the monkey bars. It was supposed to be an excuse to get outside into the sunshine and relax while basking in the enjoyment of your offspring.
Naneen finally had the kids rounded up and was walking them toward the bench. Shristi gathered herself and stood up. “Well,” she said, as Naneen stopped and pushed out the seat for Sonal and the riding platform for Esme. That would be another thing, if she went with a human nanny: She’d have to buy a stroller. And push it! “I guess we’ll see you tomorrow.”
Karen stood up too, also as if reluctant, although Shristi was willing to bet she couldn’t wait to get home. “Good luck with your decision.”
“Yeah,” Shristi said. “I’ll let you know.” She waved to Blaise, who waved back, and then she started the five-block walk home, followed by Naneen with the kids.
“So how was the playground?” she asked them, once they had crossed the main road.
“It was great!” Esme said, bouncing on the platform. “Why do we have to leave early?”
“Just time to get dinner ready.”
“Did you see me doing my trick on the monkey bars?” Sonal asked.
“I took a video,” Naneen said, before Shristi had to lie, and sent it to her watch immediately.
“Wow,” Shristi said, watching her daughter hang upside-down with Naneen’s safety net unfurled below her.
“Can we have a snack?” Esme asked, still bouncing.
Naneen automatically tried to pop her snack drawer, which no longer worked, and then remembered to use the extra compartment in her upper left arm instead, and pulled out some carrot sticks she had pared and sliced that morning—or, well, at some point in advance. Shristi didn’t know when she did all those things.
“Here,” Shristi said, awkwardly taking the carrot sticks herself and handing them to the kids.
“Mom! That’s too many!”
“It’s really not.”
“I only want one though! I can’t eat that many.”
“Fine!” Shristi shoved the excess carrots back in Naneen’s arm and exchanged a look of exasperation with the automated caregiver. How long would it take to build up that kind of rapport with a new nanny? Or did they come preprogrammed for that now?
At last they got to the house, the front door opening for them as they came up the walkway. “OK, kids, go change your clothes!” Shristi said, lifting Sonal out of her seat even though she knew it was easier for Naneen to do it.
“Come on Naneen,” Sonal said, tugging at one of the nanny’s hands. Six years old, and she still didn’t want to go upstairs without an adult or Naneen.
“Go with your sister, Sonal, Naneen will be right up.” Sonal grumbled and fretted, but finally Esme took her hand and up they went.
Shristi felt her shoulders go down and her face relax, and started toward the kitchen to get something to drink.
Naneen followed. “Did you want to talk to me about something, Shristi?” They had never reprogrammed her with the more formal naming most nannies used for the mothers or fathers they reported to.
“Oh.” Shristi had wanted to talk to her. She had thought maybe she could ask Naneen about how to manage the transition for the children. It seemed cruel, but then, Naneen didn’t have any feelings, no matter how much they wanted her to. Even so, she found she couldn’t quite handle that conversation, not right now.
“Actually, Naneen,” she said, and hesitated.
“Can I mix you a drink?” Naneen asked.
Shristi opened her mouth to say yes, but it didn’t come out. “Actually, Naneen,” she repeated, as she realized what she wanted. “Could I have a hug?”
“Of course you can,” Naneen said, in exactly the tone she had used when Shristi was tiny, and opened her arms.
It took her a little longer to warm up to optimal embrace temperature these days than it used to, but she still knew the exact pressure calibration that Shristi had preferred as a kid, and she didn’t show any signs of wanting to let go before Shristi was ready.
The temperature stabilized and then the hug felt perfect. Or at least it was perfectly, exactly what Shristi had grown up loving.
This story and essay, and the accompanying art, are presented by AI Policy Futures, which investigates science fiction narratives for policy insights about artificial intelligence.
AI Policy Futures is a joint project of the Center for Science and the Imagination at Arizona State University and the Open Technology Institute at New America, and is supported by the William and Flora Hewlett Foundation and Google.
Most Recent in Future Tense Fiction
“The Arisen,” by Louisa Hall
“The Song Between Worlds,” by Indrapramit Das
“No Moon and Flat Calm,” by Elizabeth Bear
“Space Leek,” by Chen Qiufan
“Zero in Babel,” by E. Lily Yu
“What the Dead Man Said,” by Chinelo Onwualu
“Double Spiral,” by Marcy Kelly
“Affordances,” by Cory Doctorow
“A Priest, a Rabbi, and a Robot Walk Into a Bar,” by Andrew Dana Hudson
And read 14 more Future Tense Fiction tales in our anthology Future Tense Fiction: Stories of Tomorrow, out now from Unnamed Press.