Getting started with the AI Service Agent overview dashboard

You will learn

Learn how to use the AI Agent overview dashboard to analyze your AI Service Agent’s performance and monitor its interactions.

Before you begin

To access this dashboard, you must have the AI Agent enabled in Klaviyo. Learn how to enable the AI Service Agent.

Access the AI Agent overview dashboard

Your overview dashboard offers insights into how site visitors interact with the AI Service Agent and how effectively it’s addressing or escalating inquiries.

To access the overview dashboard:

1. In Klaviyo’s left-hand navigation, select Conversations > AI Agent.
2. Select the Overview tab. 

The dashboard has 2 main sections:

• Overview: 
  Presents a summary of key performance metrics related to the AI Service Agent’s interactions with site visitors
• Conversations:
  Provides a detailed, searchable record of all interactions between your AI Service Agent and site visitors

At the top of the dashboard, use the date range selector to choose a time period for your data. Data views are based on your account’s local timezone, and are inclusive of start date and end date. For example, if today is June 1 and you choose Last 7 days, your metrics will reflect activity from June 1 through June 7, inclusive of the current date. 

Note that data will not appear in these sections if the AI Service Agent is not yet live on your website. If you disable the AI Service Agent after it has been live and collected data, the existing data will remain but new data will not be collected until it is live again.

Overview section

The Overview section displays 3 key metrics to help you understand your AI Service Agent’s activity and performance. These metrics are calculated based on entire conversations, not individual questions the AI Service Agent resolves within a chat session.

• Conversations:
  Total count of conversations initiated between the AI Service Agent and website visitors
• Resolved by AI rate:
  Percentage of conversations successfully resolved and closed by the AI Service Agent without any human intervention
• Routed to team rate:
  Percentage of conversations where the AI Service Agent initiated a handoff to your human support team via your handoff method
  • If your handoff method is a helpdesk ticket, this counts when the visitor clicks “yes” to request help from a human.

This section also features a graph showing the number of conversations initiated within your configured time period.

Conversations section

The Conversations section provides a searchable, filterable record of all conversations received by your AI Service Agent, which you can use to review individual interactions, better understand customer inquiries, and analyze how well your AI Service Agent is handling inquiries.

Conversations are organized in a table containing the following data:

• First Question: 
  The initial message sent by the visitor that started the conversation.
• Profile: 
  The name or email address of the visitor, linked to their Klaviyo profile. If the visitor has not signed into an account yet, this shows as Anonymous.
• Status: 
  The current state of the conversation, with the following possible values:
  • Resolved by AI: The AI Service Agent closed the conversation without human involvement. For a conversation to have this status, 3 criteria must be met:
    • The AI Service Agent’s message was the last in the thread,
    • No handoff was triggered (i.e., no ticket created, custom handoff message sent, or handover to a human)
    • The conversation has been closed, either by the visitor, or by auto-close after inactivity.
  • Routed to team: The AI Service Agent could not resolve the inquiry and initiated a handoff, either by showing a custom message or creating a helpdesk ticket.
  • Open: The conversation is still ongoing with the AI Service Agent, or awaiting a visitor reply. If the AI Service Agent’s last message goes unanswered and the session auto-closes, it will update to Resolved by AI.
• Messages: The total number of messages exchanged in the conversation.
• Last updated: The date and time of the most recent message in the conversation.

Clicking on any row opens a view of the full message thread so you can review the details of the interaction.

Tip: Reviewing conversations by status helps you pinpoint what your AI Service Agent can resolve independently and which inquiries are routed to your team. For best results, review conversations marked Routed to team, as these interactions can help you identify areas where your AI Service Agent needs additional training.

Additional resources

• What types of questions can the AI Service Agent answer?
• How to change the handoff preference for the AI Service Agent
• How to test and train your AI Service Agent