Prompt tracking methodology to measure AI visibility

“If we weren’t tracking a prompt, but an LLM cites our content, can you alert us?” That’s a question customers ask when they want to understand what an AI visibility report can, and cannot, show.

Prompt tracking records how AI tools answer selected questions over time. It can show where your brand appears, where competitors appear, and how those answers change.

But it can’t show every conversation your buyers have in AI platforms.

Even a report that tracks thousands of prompts can look comprehensive on the surface. But if most of those prompts represent the same few buyer situations, it can still miss important parts of how buyers discover your brand in AI search.

Instead of building an exhaustive list of prompts to rank for, the goal is to build a portfolio that helps you measure your brand’s visibility for the buyer decisions that matter. This article will show you how to build a prompt tracking strategy that balances stable measurement, discovery, and verification, even with a limited prompt allowance.

Tools like Semrush Prompt Tracking turn your selected prompts into an AI visibility report.

After you add a defined prompt set and choose the AI platforms to monitor, it shows your AI Visibility, brand mentions, average position, and competitor presence over time.

The rest of this guide explains how to build that set, so the score measures the buyer decisions you care about, not just the prompts you happened to add.

A prompt allowance is a measurement budget, not a coverage guarantee

A prompt allowance tells you how much tracking capacity you have, not how much of buyer demand you cover.

A prompt is the full question or instruction someone enters into an AI tool. An AI platform is the tool that generates an answer, such as ChatGPT, Google AI Mode, or Gemini. Your prompt allowance is the total number of prompts you can track. In Semrush, the allowance is shared across all of your campaigns, and each campaign covers one platform. Semrush checks your tracked prompts daily.

For larger teams monitoring more brands, markets, or AI platforms, Semrush Enterprise can support a broader prompt tracking strategy.

An AI visibility dashboard that tracks 500 or 1,000 prompts can sound like broad coverage. But that number means very little without context.

• How many platforms are you tracking?
• How often are prompts checked?
• Which markets, languages, or audiences are included?
• And how does your provider count a tracked observation?

Checking 100 prompts across three platforms gives you different evidence than checking 300 prompts on one platform.

However, neither approach should be treated as a one-off, static check because responses can be inconsistent.

Daily checks don't expand coverage to new buyer questions or AI platforms. They show whether an answer is stable. The same prompt can produce different responses over time, so a brand mention that keeps showing up is stronger evidence than a single result.

Before you build your portfolio, do some prompt research to find the buyer situations worth measuring. Once you have the list, the question is how much of it your report can represent.

More prompts can create the illusion of coverage

A longer prompt list improves coverage only when it covers different buyer decisions.

For example, these prompts may all belong to one prompt family, which is a group of questions that represent the same buyer decision:

Similar prompts

Underlying decision

“Best AI visibility tool”

Which platform should I use?

“Best tool for tracking AI search visibility”

Which platform should I use?

“What platform should I use to monitor LLM mentions?”

Which platform should I use?

They use different wording, but they all seek a category recommendation. Tracking every near-duplicate prompt across multiple platforms can use allowance that may be better spent on distinct buyer situations. Keep representative variations where a change in wording or constraint could affect the answer, and use follow-up checks to test whether the difference creates a meaningful pattern.

These prompts, however, represent different families because the audience, situation, or outcome changes:

Distinct prompt families

What changes

“Best AI visibility tool for an enterprise SEO team”

Team size and operating needs

“How can an agency monitor AI citations for multiple clients?”

Business model and workflow

“What tool helps a brand correct inaccurate AI descriptions?”

Reputation-management goal

“Which AI visibility platform is best for a small ecommerce business?”

Company type and buying context

A prompt deserves its own slot in your tracking dashboard when a meaningful difference could change the AI’s recommendations, sources, or reasoning. That difference might be the buyer’s role, industry, budget, use case, commercial risk, or stage in the buying journey.

A category-heavy prompt list can also make a brand look stronger than it is. You may appear when buyers use familiar industry terminology but disappear when they describe the problem in plain language.

For example, a brand may appear for “AI visibility platform” but not for:

• “How do I know whether ChatGPT recommends my brand?”
• “Why are competitors showing up in AI answers instead of us?”
• “How can I track which websites AI tools cite?”

Buyers don’t always use the category language your company uses, which can leave important questions out of your tracking set.

Include category prompts in your tracked set so you can identify your brand’s coverage for the categories you want to own. However, an AI visibility score built mostly from category terms shouldn’t be presented as a score for every relevant buyer conversation.

For guidance on selecting representative buyer questions, see how to decide which AI search prompts to track.

Separate coverage gaps from changing AI answers

AI visibility reports can mislead in three ways: the tracking set may leave out important buyer questions, answers to tracked prompts may change from run to run, or the AI platform itself may change.

A coverage gap occurs when your tracking set doesn’t represent an important buyer situation. This can make an overall visibility score look stronger than it is because the report excludes questions where your brand is less visible. A coverage gap is a limitation of the full tracking set, not a missing brand mention in a prompt you already track.

A changing answer occurs when an AI platform gives a different response to the same prompt over time.

Your brand may appear in one answer but not the next, or a different competitor or cited source may appear. Wider platform changes can also affect results across multiple prompts at once.

These issues can look similar in a dashboard, but they require different responses.

Add or test prompt families when important buyer situations are missing. Recheck prompts when answers fluctuate. Compare relevant AI platforms to see whether a change is limited to one tool or reflects a broader pattern.

The problem

What causes it

What it can look like in a report

What to do next

Coverage gap

Important buyer situations or audience segments are missing from the tracking set

A strong score based mostly on category terms while the brand is absent from realistic buyer questions

Map and report prompt-family coverage; maintain Explore capacity for testing missing buyer situations

Response variance

The same tracked prompt produces different brand mentions, citations, or recommendations across checks

A sudden win or loss based on one AI answer

Track brand-mention or citation frequency across repeated checks, not a single screenshot

Platform or system change

An AI model, retrieval method, citation format, or source set changes

A wider drop is reported as a brand-performance decline

Break results down by platform and note known system changes before assigning a cause

Research supports checking both response stability and prompt sensitivity.

Research on query-level robustness in retrieval-augmented systems found that retrieval quality can drop significantly under minor query variations. That's one reason your brand can appear for some phrasings in a prompt family and vanish for others.

Separately, repeated-measurement research found that AI search answers can vary across runs, prompts, and time. It recommends measuring visibility across repeated checks rather than treating one result as representative.

AI answers are probabilistic, so a single brand mention or omission is a snapshot, not proof of lasting visibility.

These studies don’t set a universal number of checks. But they support a practical principle: when a high-value or unexpected result could affect a decision, repeat the check and compare the pattern before reporting it as a trend.

Build a Core, Explore, Verify prompt portfolio

A balanced prompt portfolio gives each part of your tracking capacity a clear job, whether that's maintaining a baseline, discovering new questions, or validating a change.

The Core, Explore, Verify framework helps you balance three needs:

• Coverage: Does the portfolio represent the buyer situations the business cares about?
• Stability: Does the brand appear consistently, or did one unusual answer create the result?
• Comparability: Can the team make a fair, like-for-like comparison over time?

Use the revenue, reputation, competitor, and gap prompt types in our prompt portfolio framework to decide which buyer decisions belong in your Core set.

Core, Explore, and Verify then helps you decide how to divide your available prompt allowance.

A practical allocation might look like this:

Portfolio segment

Example share of capacity

Purpose

What belongs here

Core

60%

Preserve a comparable trendline

High-value prompt families, priority audiences, use cases near money pages, reputation risks, and known sales objections

Explore

25%

Find demand and visibility outside the fixed panel

New sales-call language, support questions, emerging topics, competitor gaps, site-search terms, prompt-database discoveries, and query fan-out ideas

Verify

15%

Test whether an apparent result is stable enough to act on

A smaller set of high-value prompts tracked across additional AI platforms, plus cross-platform checks after major model or retrieval changes

These percentages are an example, not a rule. A new market launch may need more Explore capacity. A reputation-sensitive category may need more Verify capacity. The important part is preserving space for all three jobs.

Core prompts protect comparability

Core prompts create a stable baseline, so you can compare results fairly over time.

The Core set should reflect recurring business decisions, not every prompt the team has ever tracked. It can cover priority audiences, buying stages, use cases, commercial opportunities, known objections, competitor comparisons, and reputation risks.

Prioritize families connected to product, pricing, comparison, demo, or other pages designed to support revenue.

Core prompts should also reflect areas where the brand could influence the result through clearer content, product positioning, documentation, customer proof, or third-party coverage.

The Core set should stay stable enough to show a meaningful trendline. But stable doesn’t mean permanent.

Replace or retire a prompt family in the Core set when the product, market, buyer language, or decision has changed. Record that change, because a revised list is no longer a perfect like-for-like comparison with earlier reports.

This avoids a common reporting error of adding new prompts, seeing more mentions, and presenting the increase as performance growth when the measurement method changed too.

Explore prompts find what the core set misses

Explore prompts test new buyer language and emerging gaps without changing your baseline.

In Semrush Prompt Tracking, add Explore prompts to your campaign and tag them separately from Core prompts. Each tag has its own AI Visibility score. Although Explore prompts contribute to the overall campaign score, you can report the Core tag separately as your stable baseline and use the Explore tag to assess new opportunities.

Explore candidates can come from three places.

• Customer signals: sales calls, product demos, support tickets, interviews, and reviews
• On-site signals: internal site searches, pages that AI answers cite and AI referral landing pages
• Market signals: competitor activity, community discussions, industry shifts, and emerging topics

Keep buyer language separate from query fan-out. Fan-out searches are the related queries an AI system may look up behind the scenes while producing an answer, not the decisions buyers are making. They can inform research, but they don’t automatically deserve Core slots.

Move a family from Explore to Core when it consistently reveals a decision the business needs to monitor. This could include a competitor gap around a priority product or conversion page, recurring inaccurate brand information, or new customer language tied to an important market or use case.

Because your prompt allowance is fixed, promoting a family into Core should be a swap.

Remove the Core family tied to the least important current business decision, then record what changed, when it changed, and why. This makes it clear in future reports whether a score moved because visibility changed or because the tracking methodology changed.

Verify prompts test whether a result is real enough to act on

Verify prompts help you tell the difference between a real pattern and one unusual AI answer.

Reserve this capacity for high-value prompts that suddenly gain or lose your brand, new competitor citations, major content updates, market launches, reputation events, and known model or platform changes.

Verification means repeating the prompt where possible and comparing the pattern across relevant AI platforms. A result can then be reported as a frequency rather than a single snapshot.

Instead of reporting:“We rank in ChatGPT for this prompt.”

Say this to stakeholders:“Our brand appeared in 8 of 10 observed answers for this prompt family on Platform A during the reporting period.”

That statement reports an observed pattern, not a universal claim. It also gives stakeholders something more useful to act on: whether the pattern is strengthening, weakening, or limited to one platform.

For more on tracking mentions, citations, and responses over time, read our guide toLLM prompt tracking.

How many AI platforms should you track?

Allocate most of your prompt allowance to the AI platform or platforms your buyers use most. Rather than duplicating your entire Core set across every platform, track a smaller Verify set of high-value prompts on other relevant platforms.

This shows whether a pattern is platform-specific without using your full allowance to repeat every Core prompt. In Semrush, each campaign covers one platform, so a Verify set on a second platform is a second campaign with only those prompts, and they come out of the same allowance. A platform-specific pattern appears in one AI tool but not another.

For example, your brand may appear consistently in Google AI Mode but rarely in ChatGPT. Report that as a platform-level finding, not as proof.

Use these comparisons to see where your brand and competitors appear, and whether a visibility change is limited to one platform or reflects a broader trend.

Can a tool alert you about a prompt you never tracked?

Not for a prompt you never added. Prompt Tracking only checks the prompts in your campaigns, and no tool has a log of every conversation buyers have with AI. What the AI Visibility Toolkit can do is show where AI answers already cite your site, including for prompts you aren't tracking.

The Visibility Overview report draws on Semrush’s database of more than 317 million AI queries. Its Cited Pages tab lists the pages on your site that AI answers cite, with the number of prompts behind each, so you can see citations from prompts you haven't added to Prompt Tracking.

Treat that as a discovery layer. When a page you didn't expect shows up, look at what it's answering, then add the buyer questions that matter to your Explore tag.

It won't capture every private AI conversation, but it can show citation patterns beyond your tracking set. Combine it with the customer, on-site, and market signals above to find new buyer questions worth testing.

Report what your tracked prompt set covers, not just its visibility score

An overall AI visibility score comes from your prompt tracking tool. It checks your defined set of prompts and records whether your brand appears and how prominently.

For example, Semrush’s Prompt Tracking report lets you monitor chosen prompts across selected AI platforms and view AI Visibility, brand mentions, and average position over time.

But these metrics represent performance within your tracked prompt set, not every AI conversation about your category.

Visibility scoring becomes more useful when your stakeholders can see what it includes. Alongside the headline numbers, report the:

• Prompt families
• Audiences
• Buying stages
• Languages and markets
• Platforms

For example, you could report “AI Visibility was 42% for Core prompt families related to AI visibility tools and brand monitoring. The report covers in-house SEO teams and agencies at the consideration and decision stages, using English-language prompts in the US and UK across ChatGPT and Google AI Mode.”

Also state the time period the results cover, such as the last 30 days.

If some prompts are more important to your business than others, you may want to create a weighted report for stakeholders. In that report, explain how you weighted each prompt or prompt family based on commercial importance. For example, you might give more weight to prompts tied to priority products, conversion pages, or high-value audience segments.

Finally, note any material changes to the tracking methodology, such as adding prompt families, changing platforms, or updating the weighting. This helps stakeholders see whether a score moved because visibility changed or because the measurement approach changed.

The headline score shouldn’t stand alone. Use supporting metrics to show how and where your brand appears in AI answers.

Signal

What it tells you

Brand mentions

Whether the AI answer names your brand

Owned-site citations

Whether the answer cites a page on your website

Prominence (Average position)

How visible your brand is in the answer, not just whether it appears

Accuracy or sentiment

Whether the AI describes your brand correctly and favorably. Review this in Brand Performance.

Competitor presence

Which rivals appear in the same buyer decision

For example, mentions may increase while owned-site citations remain flat, or citations may rise while the AI describes your brand inaccurately. Breaking results down this way shows whether visibility is improving and where the team needs to act.

For more guidance on putting those metrics in context, see how to measure AI visibility.

Conclusion: The goal isn’t a perfect list

A perfect prompt list doesn't exist. A useful one states its scope and its limits.

A stable Core set of prompts supports fair comparisons over time. Explore prompts surface new buyer language and potential gaps. Verify prompts show whether a change is strong enough to influence a decision.

Together, they create a more transparent view of AI visibility, one that shows what you measure and what you don't. A better question than how many prompts you can track is which decisions your prompt portfolio lets you make with confidence.