Rethinking Metrics for UX, Service Design + AI

Lightning Lesson | 30 minutes | FREE

Your metrics improve but your experience gets worse?

Our typical default metrics include things like: more engagement, better conversion or fewer support calls. But the numbers can look impressive while hiding problems people experience.

Examples: Time spent might reflect enjoyment, confusion or compulsive use. Retention might reflect lasting value or difficulty leaving. An AI tool might generate work faster while increasing the effort needed to check it.

In this Lightning lesson, Frank Spillers will review 25 years of ROI of UX metrics focusing on two key questions: Are our familiar success measures scalable in an AI world? Are current UX metrics and KPI’s providing strong evidence of user and customer value?

What happens when we become good at improving the wrong measures?

UX and Service Design need to revisit what counts as success. Social media addiction, pressure to monetize and AI safety raise questions that conventional dashboards can leave unanswered.

David Dylan Thomas’s No, Seriously, F**k Engagement: Building a More Human Web provides a starting point. We’ll extend that discussion to long-term customer value, AI interactions and the human effort hidden inside automation.

Why rethink metrics now?

Metrics influence what teams prioritize, fund and reward. Choosing them means making decisions about whose outcomes matter.

For UX and Service Design, that includes asking “frontstage” whether people achieve their goals, retain control and receive lasting value. And “backstage” whether we reduced cycle time, improved processes, impacted policies and so on.

With AI, it also means thinking how we measure reliability, safety and the work transferred to users or staff.

This Lightning Lesson introduces a more demanding question: what would we need to know before calling this an improvement?

Agenda:

1. Engagement and its consequences: When increased use signals value, friction or dependency.
2. Monetization and customer value: What revenue and retention reveal—and what needs further investigation.
3. How AI changes the measurement problem: Why voice, chatbots and agents require closer attention to behaviour, reliability and human effort.
4. Rethinking success: Connecting measures with meaningful outcomes, customer control and safety.

Who this is for

• UX practitioners, service designers, product designers and researchers who want stronger ways to question success claims.
• Design and product leaders reviewing metrics, business cases or AI investments.

What you’ll take away

• A clearer distinction between usage, commercial performance and customer benefit.
• Questions that expose misleading success measures.
• A starting point for reviewing one metric in your work.

About Lightning Lessons

Short, focused sessions with Frank Spillers. Explore a current UX or Service Design challenge through examples, discussion and questions.