How to build an AI agent that remembers returning customers

A customer texts on Monday about a billing error. Your AI agent sorts it out. Well done. ∗claps∗ But on Wednesday, the same customer calls about a follow-up charge, and…the agent asks for their name, account number, and what the problem is. Again.

This situation isn’t a hypothetical either. Really, it’s more like the default experience right now for customers. That’s why our research found:

• 74% of consumers said they have to repeat themselves when interacting with an AI assistant

74% of consumers said they have to repeat themselves when interacting with an AI assistant

• 71% said they're willing to abandon a conversation early if an AI agent fails to recognize them

71% said they're willing to abandon a conversation early if an AI agent fails to recognize them

The brands notice, too. 91% have seen customer satisfaction drop because of AI missteps, most notably when an AI loses conversational context between channels or interactions (40%).

Woof. Fortunately, there’s a solution: AI customer memory.

Here's how to build an AI agent that recognizes returning customers, loads what it knows before it says hello, saves new facts as it learns them, and carries all of it across channels.

What does it mean for an AI agent to remember a customer?

An AI agent remembers a customer when it can identify them across conversations, recall relevant facts from past interactions, and save new details it can use next time.

That takes three capabilities working together:

• Identity: Knowing the person texting today is the person who called last week.

Identity: Knowing the person texting today is the person who called last week.

• Recall: Pulling the right context into the conversation at the right moment.

Recall: Pulling the right context into the conversation at the right moment.

• Write-back: Saving what the agent learns so it's there for the next conversation.

Write-back: Saving what the agent learns so it's there for the next conversation.

Then, there’s super-long conversations. These sometimes struggle to include all the context inside a model's limits. Our guide totoken limits in long AI support conversations covers that one, but the guide below covers the time between conversations.

Let’s walk you through building an AI agent that remembers returning customers. Here are the Twilio products we’ll be using:

• Twilio Conversation Memory to store what your agent knows

Twilio Conversation Memory to store what your agent knows

• Conversation Orchestrator to link each conversation to a customer profile

Conversation Orchestrator to link each conversation to a customer profile

• Twilio Agent Connect, the open source SDK that connects your agent to Twilio's channels and memory.

Twilio Agent Connect, the open source SDK that connects your agent to Twilio's channels and memory.

1. Recognize who's contacting you

Memory starts with identity. Conversation Memory'sidentity resolution matches and merges profiles across channels using phone numbers, WhatsApp handles, email addresses, chat IDs, and your own user IDs. Conversation Orchestrator applies it to every conversation, so a text on day 1 and a call from the same number on day 2 resolve to the same profile. Set GROUP_BY_PROFILE explicitly to keep them in one conversation.

• Link identifiers up front: A customer who texts from one number and calls from another only unifies if both numbers sit on the same profile, so upload profiles with linked identifiers before launch.

Link identifiers up front: A customer who texts from one number and calls from another only unifies if both numbers sit on the same profile, so upload profiles with linked identifiers before launch.

• Verify before sharing: Phone numbers can change owners, so confirm a profile's traits match the caller before you mention anything sensitive.

Verify before sharing: Phone numbers can change owners, so confirm a profile's traits match the caller before you mention anything sensitive.

In Studio, theResolve Profile widget looks up a caller by phone number or email address and returns their traits for greeting or routing.

2. Decide what to remember

Conversation Memory stores three kinds of memory:

• Traits: Structured facts the agent acts on every time, like plan, tier, or preferred language.

Traits: Structured facts the agent acts on every time, like plan, tier, or preferred language.

• Observations: Facts and preferences extracted from conversations, like "Prefers email over phone," which Memory updates as customers add detail or change their minds.

Observations: Facts and preferences extracted from conversations, like "Prefers email over phone," which Memory updates as customers add detail or change their minds.

• Summaries: Short recaps of each conversation.

Summaries: Short recaps of each conversation.

Turn on memory extraction in your Orchestrator configuration.

3. Load context before the agent says hello

This step stops the repeated questions. Conversation Memory'sRecall API returns a profile's most relevant observations and summaries (up to 20 and 5 by default), and its date filters let the agent pull up yesterday's conversation.

With Twilio Agent Connect, set the memory mode to always (every message) or once (first message only). The default is never, and memory requires Agent Connect's Orchestrator mode, so configure both explicitly. On phone calls, Agent Connect usesConversationRelay for speech-to-text and text-to-speech.

Twilio's multi-call memory tutorials build this end to end, with a voice agent that looks up the caller, recalls past calls before the greeting, and saves a summary at hang-up:

• Python

Python

• Node.js

Node.js

• PHP

PHP

4. Save new facts during the conversation

Memory extracts observations after a conversation goes inactive or closes. There's no mid-conversation extraction, so when a customer confirms an appointment or changes an address, write it yourself:

• Create observations: Add up to 10 at a time to the customer's profile.

Create observations: Add up to 10 at a time to the customer's profile.

• Update traits: Patch a defined trait, like preferred channel.

Update traits: Patch a defined trait, like preferred channel.

Agent Connect only reads profiles, so build the write as a tool backed by your own Memory API call. Twilio'spersistent customer memory tutorial shows how preferences extracted after a conversation are saved as traits and used in the next session.

5. Use memory with intention

Twilio'sInside the Conversational AI Revolution research found 66% of consumers said they wouldn't prefer (or feel uneasy) providing an AI agent with all previous context from interactions. A few rules help:

• Lead with the useful part: "I see you called yesterday about a duplicate charge. Same issue?"

Lead with the useful part: "I see you called yesterday about a duplicate charge. Same issue?"

• Confirm what can change: Ask before acting on addresses, plans, or phone numbers.

Confirm what can change: Ask before acting on addresses, plans, or phone numbers.

• Let customers correct the record: Update or delete a wrong memory through the Memory API.

Let customers correct the record: Update or delete a wrong memory through the Memory API.

6. Carry memory across channels and handoffs

Because identity resolves across channels, the profile a text built is the one a call reads. Agent Connect handles Voice, SMS, WhatsApp, RCS, and Chat through one message handler.

Memory follows the customer to people, too. Agent Connect'sescalation tool passes the conversation ID, memory store ID, and profile ID toTwilio Flex. OurAgent Connect and Flex handoff tutorial shows a stored language preference shaping the conversation.

For voice language switching, see our guide tomultilingual AI agents.

Make the second conversation better than the first

Every repeated question tells a customer your AI agent doesn't know them. Recognize them, recall what matters, save what's new, and let memory follow them across channels and to your human team.

Do that, and the second conversation goes smoother than the first. And imagine what it looks like by the tenth. And then, there are the times when your AI agent reaches out first: see our guide tobuilding a proactive agent for AI outbound calling.

Start for free and build an AI agent with Conversation Memory, ortalk to sales about bringing memory to your contact center.

Frequently asked questions

How do I stop my chatbot from asking the same questions every time a customer returns?

Twilio Conversation Memory stores what customers share as traits, observations, and conversation summaries. Agent Connect retrieves that memory before each reply, so the agent starts with what it already knows.

Which platforms let an AI agent detect that a customer called yesterday and reference that context automatically?

Twilio Conversation Orchestrator links a returning caller to the same customer profile, and Conversation Memory's Recall API filters memories by date, so the agent can pull up yesterday's conversation summary before it replies.

Which customer engagement tools let an AI agent remember preferences across sessions?

Twilio Conversation Memory saves preferences as traits, like preferred language or channel, and as observations from past conversations. Agent Connect loads them at the start of the next session.

What tools let an AI agent update a customer profile mid-conversation based on new information?

Twilio Conversation Memory's API lets an agent tool create observations or update traits during a live conversation. Automatic extraction runs later, after the conversation goes inactive or closes.

Why does my AI agent forget customers between conversations?

Twilio Conversation Memory can only recall a profile it can identify, and it extracts observations after a conversation goes inactive or closes. Check identifier links, extraction settings, and the agent's memory mode.