As AI speeds up how fast products ship, product teams are owning more metrics with less time to watch any of them. Whatever your team measures against, whether that's activation rate, weekly active users, or signup conversion, keeping tabs on it usually means the same routine: opening a dashboard every morning to see if anything moved, or pulling the number by hand every week to write up what changed for stakeholders. It works, but it's manual, and the watching only happens when you remember to do it.
KPI monitoring in Mixpanel hands that job to an AI agent. You pick a metric, set a schedule, and the agent watches it around the clock, then delivers a digest to Slack or email that tells you what changed and what's worth your attention. You stay on top of the metric and it comes to you, with context.
Definition
What is KPI monitoring?
KPI monitoring is the practice of tracking a key metric over time and getting a heads-up when it moves in a way that matters. In Mixpanel, an AI agent does the watching for you: you pick a metric, set a cadence, and it delivers a personalized digest to Slack or email that summarizes what changed and flags what it considers notable.
This is product-analytics KPI monitoring, not server or infrastructure monitoring. It's also distinct from a live dashboard you refresh and from a threshold alert that fires the instant a number breaks. KPI monitoring brings a scheduled, contextual read of your metric to you.
Here's what the KPI agent does, and how to set one up.
What the KPI Monitoring agent does
Mixpanel’s KPI Monitoring agent is a hands-off way to track a KPI over time instead of checking a dashboard yourself. What you get is steady, low-effort awareness of the one number you care about, delivered where you already work.
What the agent does & where it stops
- Runs one fixed query on the metric you pick
- Summarizes what changed against a baseline
- Flags notable shifts, with its reasoning
- Won't take actions on its own
How to set up a KPI Monitor
Setup takes a few minutes and lives entirely in the product. Here's the full sequence:
- Open the Automation workflow. In the side navigation, click + Create New, then choose Automation at the bottom of the Analysis section. (If you don't see the option, your organization may not have Mixpanel AI enabled yet, which an admin can switch on in settings.)
- Pick the metric to monitor. Choose an Insights-style event metric you've already instrumented, whether that's a sum, a count, or a unique count. The agent uses the metric's existing definition as its source of truth.
- Set a cadence. Digests can arrive daily, weekly, or monthly. Weekly is a sensible default for most teams, daily suits a metric you're actively pushing on, and monthly fits slower-moving numbers you still want on record.
- Add instructions. This is the step most teams underuse. The Instructions field shapes the recap’s commentary, not the underlying query, so you can tell the agent what matters to you.
- Pick a delivery channel. Slack or email. Slack delivery needs the Slack integration configured for your project first.
- Preview, then save. Use "run now" to see a sample digest before you commit to a cadence. When it looks right, save it. Your first scheduled recap arrives at the next cadence boundary, and you can pause, edit, or delete the monitor whenever you want.
Tip from the Mixpanel team
Don't skip the Instructions field. It shapes how the agent writes the digest, not what it queries, so brief it like a teammate: tell it what counts as notable for this metric and which comparison to lead with. Something like "focus on week-over-week change and flag anything more than 2 standard deviations from the 30-day average" gets you a sharper read than the default.
What shows up in a digest
Each digest is short by design. You get a quick summary of what the metric did in the last window against its baseline, plus any shifts the agent flagged as notable with the reasoning behind them. There's also a feedback control, so when the agent surfaces something that wasn't useful, you can tell it, and the digests get sharper over time.
In practice it reads like a note from a teammate who checked the number for you. A weekly monitor on weekly active users might open by noting the metric rose 6% week over week, then point out that most of the lift started midweek. You get the what and the where in a couple of sentences, and you decide what to do next.
KPI Monitoring vs alerts vs Root Cause Analysis
KPI Monitoring, Anomaly Detection alerts, and Root Cause Analysis all deal with metric changes, which is why they often get blurred together. They're built for different moments, though, and they work best together rather than as substitutes. Here's how to tell them apart at a glance:
Which metric should you put a monitor on?
Start with the one number you're accountable for, then add the leading indicators that tend to move before it does. If you're not sure how those inputs connect, Metric Trees map how your KPIs ladder up to an outcome and let you monitor them together, and this guide to KPI trees is a good primer on building one before you decide what to watch.
A monitor also earns its keep right after a launch. When you ship something behind a feature flag, put a monitor on the metric that flag is meant to move and watch it for the first couple of weeks. Keep the scope honest: the monitor tracks the single metric you pick and reports its trend, it doesn't compare variants against each other, which is the job of the Experiments Agent.
Set a monitor and let it come to you
Pick the metric you check most often, set a monitor on it, and stop opening a dashboard to find out whether anything moved. To learn more, see how the Mixpanel KPI Agent watches your metric and sends the update to you.
Staff Product Designer @ Mixpanel
