AI insights · Email marketing

AI-Powered Email Health

Turning an opaque email health report into a clear score and useful next steps. A short internal R&D sprint that was added to the following quarter's roadmap.

Role
Product Designer
Timeline
1 sprint
Team
1 PM · 2 engineers
Scope
Reporting · AI concepts

My contribution

I used six user interviews and competitive research to design the widget, report, and assistant concepts. In a workshop with product, research, content, and customer success, we narrowed the scope to helping users understand problems and take action. It started as a short internal R&D sprint, was picked as one of the top 3 of 36 projects, and was added to the following quarter's roadmap.

Problem

Users learned about email problems too late.

Keap is an all in one CRM for small business owners, acquired by Thryv in October 2024. Many of them run their marketing through Keap email. Campaign suspensions, unclear bounce warnings, and manual list cleanup left them dependent on support. The existing experience scored 48.8 SUS in baseline testing.

01

Poor deliverability

Campaigns like a Black Friday sale landed in spam or never arrived, and users often had no idea why.

02

Little awareness of email health

Most users did not know their email health status until something broke.

03

No tools, no clear next step

Even when users saw a problem, they did not know what to do about it.

04

Support overload

Deliverability questions filled the support queue.

Original email health widget with engagement days, bounce rate, and complaint rate12
The email health widget on the dashboard.
Original email engagement report with a donut chart and a long text panel435
The email engagement report.
  1. 1

    Jargon labels

    “Average engagement (days)” and “Invalid bounce rate” meant little to a small business owner.

  2. 2

    Numbers without meaning

    21.34 days or 0.00% did not say whether things were good or bad, or what to do.

  3. 3

    A wall of text

    The side panel explained engagement in paragraphs instead of telling users what to do.

  4. 4

    No trends

    A single snapshot, so users could not see whether email health was getting better or worse.

  5. 5

    Generic help

    “Learn more” led to a general article, not steps for this account.

The contact list had the same problem: it was overwhelming, with no sense of who mattered.

Constraints

  • Speed and ambiguity. A very short timeline, no defined AI patterns, and no AI team.
  • Data. Deliverability signals are technical and noisy.
  • Org. We had to align fast while Keap and Thryv merged their platforms.

Research

Users wanted a heads-up, not another report.

I looked at product analytics, benchmarked competitors, interviewed six users, and ran a design thinking workshop with product, research, content, and customer success.

What the data showed

Fewer than 10% of users had adopted Email Health. Deliverability tickets were high, and few people took action from the reports.

What competitors did

Other tools paired a simple score with deeper analytics. None of them guided users with AI, and that was the gap.

Competitive analysis of email health safeguards
Identify the missing safeguards. Competitive analysis looked at suppression, bounce thresholds, and verification to see where the current experience left users exposed.

What we heard. Users wanted earlier warnings, reporting they could understand, and specific steps to fix bad contacts instead of trial and error cleanup. One put it simply: “I wish the system gave me a heads-up.”

The questionHow might we use AI to make Email Health clear, human, and trustworthy, while lining up with Thryv 2.0?

Design stance

Make the score explainable, and put the next step right next to the problem.

Process

Make the score explainable and the next step visible.

The workshop focused the MVP on insight, guidance, and control.

01

Replace ambiguous metrics

Use a 0 to 100 score as an entry point, then explain what contributes to it rather than leaving the number on its own.

02

Organize around three actionable categories

Deliverability explains inbox reach, Engagement explains recipient interaction, and Complaints explains negative feedback.

03

Keep resolution in the workflow

Pair warnings with cleanup or improvement actions so users can respond without leaving their day to day CRM work.

Understanding

More adoption and repeat use.

Behavior

More actions taken from AI insights.

Email performance

More engagement, fewer bounces and spam reports.

Trust

Fewer email health tickets.

Product Design

From a status report to a useful diagnosis.

The widget gives an at a glance view. The report adds category breakdowns, trends, and recommendations for a deeper look.

What 5 users said about the redesign

What worked

“I love the score... it’s the first time I’ve understood my email health at a glance.”

One big issue
“I get the numbers… but I need someone to walk me through what to do.”
What I changed

Explored AI that guides the next step, from a side panel to an agent that does the work.

What we learnedThey did not need more data. They needed AI to guide their next step.

Results

From a short sprint to the roadmap.

A second round of moderated testing with 5 small business owners checked clarity, trust, and whether people could act on what they saw.

R&D ranking
Top 3

Out of 36 internal R&D projects.

Roadmap
Next quarter

Added to the following quarter after the sprint.

Second round
5 users

Small business owners in moderated testing.

Baseline SUS
48.8

The existing experience, before the redesign.

Impact

  • Users found the new AI visuals more intuitive and meaningful.
  • Most testers found AI recommendations far more helpful than help articles.
  • Users preferred the agent style AI because it helped them get things done.
  • Set the direction for the AI foundation in Thryv 2.0.

What I learned

  • Trust matters more than UI in AI design.
  • Constraints drive better decisions.
  • AI needs a tight loop between design and engineering.