Assistant Monitoring
In this section
This section explains the various functions for monitoring assistant interactions.
Conversation History
The Conversation History of an assistant lets you monitor all the interactions that have taken place between end users and the assistant.
This section is accessible by opening the assistant's menu (the three dots at the top right of the card) and selecting Conversation History.
Conversation Archive
The section shows the complete list of conversations that took place in the selected time period.
You can filter conversations by:
- User ID.
- Conversation ID.
- Interaction channel.
For each conversation the following are shown:
| Field | Description |
|---|---|
| Conversation start date and time | — |
| Messages sent by the assistant to the user | — |
| Courtesy replies | See Courtesy Replies and Engagement Messages. Sent by the assistant to the user. |
| Answered questions | Number of user questions that received an answer (excluding Courtesy replies). |
| Interaction channel | — |
| Live Chat operator | See Live Chat. Operator who intervened during the conversation. |
| User contact | If provided during the interaction through a form (see Conversational Forms). |
Analyzing a single conversation
Selecting the Open Chat option displays the conversation transcript.
At the top you can see the conversation ID and the source, i.e. the channel.
In the transcript, the user's questions are on the left, in gray; the assistant's answers are on the right, in green.
User questions
Depending on the features used by the assistant, the user's question can show certain information.
Knowledge Base
If the assistant's answer comes from the Knowledge Base (see Knowledge Base), the info icon shows the name of the winning intent.
Reformulated question
If the Enable question reformulation based on conversation context feature is active (see AI Gen for the Knowledge Base), the user's question is replaced with its reformulated version and the info icon shows the original question.
In this case, even if the answer comes from the Knowledge Base, the info icon shows the user's original question.
Translated question
If the Multilingual feature is active, the user's question is replaced with its version translated into the assistant's base language (see Assistant Creation) and the translation icon shows the original question.
Assistant answers
Depending on the features used by the assistant, its answers can show certain information.
Response type
Depending on the response type, a code is shown:
| Code | Meaning |
|---|---|
| WM | Welcome message (see General Assistant Settings). |
| QA | Answer from the Knowledge Base. |
| QB | Courtesy reply. |
| QG | Answer generated with AI Gen (see AI Gen for the Knowledge Base). |
If the generated answer comes from the Knowledge Base, the QA code is used.
- SDA corresponds to an answer from Smart Document Access (see Smart Document Access).
- FORM corresponds to an answer or question from a form (see Conversational Forms).
Translated answer
If the Multilingual feature is active, the translation icon shows the actual answer given to the user.
Smart Document Access
If the answer comes from Smart Document Access, the Show documents option is available below it.
If selected, the documents used to generate the answer are shown.
If the Do not show the ‘Show documents' option after the answer feature is active, the transcript indicates that the documents were hidden during the actual interaction.
Reinforcement
The Reinforcement section displays all the questions and answers handled by the assistant.
The section is accessible by opening the assistant's menu and selecting Reinforcement.
Unanswered questions
This section contains the complete list of unanswered questions.
For each unanswered question you can view:
| Field |
|---|
| User question |
| Date and time |
| Full conversation |
For example, a user asked the assistant “Is it true that you can score goals?”, but the assistant was unable to answer.
The assistant may not have answered for two reasons:
- There is no matching intent in the KB;
- The intent is in the KB, but the algorithm, based on the input received, was unable to detect it.
Disambiguations
This section contains the complete list of disambiguations.
For each disambiguation you can view:
| Field |
|---|
| User question |
| Selected option |
| Answer corresponding to the selected option |
| Date and time |
| Full conversation |
Positively voted answers
This section contains the list of assistant answers that users voted positively.
For each positive vote you can view:
| Field |
|---|
| User question |
| Assistant answer |
| Date and time |
| Full conversation |
User votes are only available if the Enable voting option is active in Parameters and Reinforcement.
Negatively voted answers
This section contains the list of assistant answers that users voted negatively.
For each negative vote you can view:
| Field |
|---|
| User question |
| Assistant answer |
| Selected option (requires the Show suggestions feature in Parameters and Reinforcement) |
| Date and time |
| Full conversation |
User votes are only available if the Enable voting option is active in Parameters and Reinforcement.
SDA
This section contains the list of all user questions that received an answer from the Smart Document Access documentation (see Connecting an assistant to the documentation).
For each question you can view:
| Field |
|---|
| User question |
| Assistant answer |
| Link to the document |
| Date and time |
| Full conversation |
AI Gen
This section contains the list of all user questions that received an answer from AI Gen (see AI Gen for the Knowledge Base).
For each question you can view:
| Field |
|---|
| User question |
| Assistant answer |
| Date and time |
| Full conversation |
Adding an element to the knowledge base
All Reinforcement sections let you add the user's question (and any generated answer) to the assistant's knowledge base by selecting the Add option.
The user's question can be assigned to an existing intent, selected from the list.
Alternatively, you can create a new intent with the user question already assigned.
In both cases, the user's question can be edited and improved once it has been added to the knowledge base (see Intent).
Statistics
The Statistics section, accessible from the left menu, displays data useful for monitoring and analyzing the virtual assistant's performance.
In this section you can view:
- The performance of your assistant.
- The performance of the web service calls made by the assistant(s).
Web Services Calls
This section displays statistics on the web service calls made by the selected assistant, if it uses any.
By selecting the calendar icon at the top right, you can define the time range for which you want to view the statistics. The default range is one month starting from today.
You can refine the search using filters for:
- Assistant type.
- Status.
Assistants
For each assistant you can access, through the icons on the right:
| Section | Content |
|---|---|
| Conversations | Data on the assistant's performance when interacting with users. |
| Emotions | Statistics on the emotions recognized by the virtual assistant. |
| Attention level | Statistics on the attention level recognized by the virtual assistant. |
| Questions | Ranking of the most frequent requests that received an answer from the virtual assistant. |
The Emotions and Attention sections are only available for enabled accounts.
Conversations
General statistics
This section provides an overall summary of the virtual assistant's performance. The data displayed are:
| Metric | Description |
|---|---|
| Conversations | Number of conversations. |
| Messages | Number of messages exchanged. |
| Unanswered | Number of questions the assistant was unable to answer. |
| Disambiguations | Number of times the assistant went into disambiguation. |
| Conversation depth | Average number of messages exchanged in each conversation. |
| Average conversation duration | — |
| Total conversation duration | — |
| Response speed | Average response speed. |
Conversation Trend
This section offers an overview of the trends over time of the conversations handled by the virtual assistant. You can filter the data distribution by hour and by day.
Response type
In this section the assistant's answers are classified into various categories, such as correct answers, courtesy replies and disambiguations. You can filter the distribution of these answers by hour and by day, and also view the aggregate totals.
Interaction type
This section describes the different forms of interaction handled by the assistant, including forms, engagement and questions. The distribution of these interactions can be filtered by various parameters, such as hour and day, and the aggregate totals can also be viewed.
Unique users per channel
This section lists the number of distinct users who have interacted with the assistant, broken down by communication channel (e.g. web, social, etc.).
Forms
This section contains data on the forms that users filled in during their interactions with the assistant. Forms are classified as completed, failed, incomplete and interrupted.
Input type
This section catalogs the different types of input received by the assistant, including keyboard, click, engagement and voice input.
Unknown input refers to the Facebook, Whatsapp and Telegram channels.
The distribution of these inputs can be filtered by hour and day, and there is also the option to view the aggregate totals.
Openings by domain
This section lists the openings of new conversations, categorized by source domain (e.g. company website, app, etc.).
If the assistant is integrated in a single domain, the chart shows a single slice.
Openings by type
Here you find data on the openings of new conversations, broken down by opening type.
The opening types are:
| Type |
|---|
| Text conversation |
| Voice and text conversation |
| Digital human conversation |
Live Chat Messages
This section contains statistics on the messages exchanged in Live Chat mode, including the number of messages sent and received.
Live Chat Actions
This part reports the actions taken during Live Chat sessions.
Consumable Dashboard
The Dashboard lets you monitor the status of the account's consumables.
Textual
The textual conversations counter shows the number of conversations in which text interaction (i.e. via keyboard) was used at least once.
Voice
The voice conversations counter shows the number of conversations in which voice interaction (i.e. via microphone) was used at least once.
AI Gen
The AI Gen messages counter shows the number of messages for which at least one AI Gen feature was used.
If more than one AI Gen feature was used for a message, the number of consumables used corresponds to the number of features used.
For example, if an assistant has both Enable question reformulation based on conversation context and Enable answer reformulation based on the context of the question active (see AI Gen for the Knowledge Base), one assistant answer will use two consumables.
For example, if an assistant has answers generated through Smart Document Access active (see Connecting an assistant to the documentation) and has Launch form set by AI Gen active, one assistant answer will use two consumables.