Edit

Share via

Manage Quality Evaluation Agent

Conversations: Applies to: Dynamics 365 Contact center only

Cases: Applies to: Dynamics 365 Customer Service only

Emails: Applies to: Dynamics 365 Customer Service only

Quality Evaluation Agent is an AI agent that assesses customer engagement using an evaluation framework defined by supervisors. The quality evaluation framework comprises evaluation criteria, evaluation plan, and evaluations that are essential for the AI Agent to work.

Quality Evaluation Agent autonomously scores cases and conversations, delivering actionable insights to help supervisors improve interaction quality. It evaluates cases and closed conversations to make sure they comply with required standards. When standards aren't met, the AI Agent recommends actions to enhance future interactions.

Important

  • Evaluation for emails is a preview feature.
  • Preview features aren’t meant for production use and might have restricted functionality. These features are subject to supplemental terms of use, and are available before an official release so that customers can get early access and provide feedback.

Evaluation criteria:

Create a form with questions, answer choices, scoring metrics, and detailed instructions for the Quality Evaluation Agent. The AI agent uses this form to assess interactions.

Evaluation plan:

Set up plans to schedule when to evaluate interactions. You can select interactions based on specific conditions and use the right evaluation criteria to review interactions systematically. You can create, activate, and manage evaluation plans, request on-demand evaluations, and enable bulk evaluations to streamline your review process.

Evaluations:

The Quality Evaluation Agent evaluates case and conversations and provides summaries of the interactions, with insights and recommendations that help improve customer interactions. You can use on-demand evaluation to check cases, conversations, and emails when needed.

Important

  • This feature is intended to help customer service managers or supervisors enhance their team's performance and improve customer satisfaction. It isn't intended to be used, and shouldn't be used, to make decisions that affect the employment of an employee or group of employees, including compensation, rewards, seniority, or other rights or entitlements.
    Customers are solely responsible for using Dynamics 365, this feature, and any associated feature or service in compliance with all applicable laws, including laws that are related to accessing individual employee analytics, and monitoring, recording, and storing communications with users. As part of this compliance, customers must adequately notify users that their communications with customer service representatives (service representatives or representatives) might be monitored, recorded, or stored. As required by applicable laws, customers must also obtain consent from users before they use this feature with them. In addition, customers are encouraged to have a mechanism in place to inform their service representatives that their communications with users might be monitored, recorded, or stored.

Prerequisites

Data movement across regions

For optimal performance, service calls might be routed outside the customer's regional boundary if local capacity is temporarily unavailable. To participate, customers are requested to provide consent for potential data movement across regions. This consent enables us to deliver a seamless experience by using the full capabilities of Copilot and generative AI features within Power Platform.

Learn more in Move data across regions for Copilots and generative AI features – Power Platform.

Role and privileges

Persona Role Privileges
Administrator Quality Administrator - Configure Quality Evaluation Agent.
- Create evaluation criteria and evaluation plan.
- Complete or assign an evaluation.
Supervisor Quality Manager - Create evaluation criteria and evaluation plan.
- Complete or assign an evaluation.
Supervisor Quality Evaluator Complete or assign an evaluation.

Configure connection references

When you navigate to the Quality Evaluation Agent page in Copilot Service admin center, a Prerequisites section appears at the top of the page that indicates whether connection references are set up. You need to configure connection references for Quality Evaluation Agent flow to integrate with Microsoft services. These connections link flow to essential data sources such as Microsoft Dataverse and Copilot Studio, ensuring smooth operation and enhanced functionality.

  1. In Copilot Service admin center, go to Customer Support > Quality management. The Quality management page appears.
  2. Select Manage for Quality Evaluation Agent. The Quality Evaluation Agent page appears.
  3. In the Prerequisites section, verify if Step 1: Connection References, Step 2: Power Automate Flows, and Step 3: Copilot Studio Agent show as Ready.
    1. If Step 1: Connection References shows as In progress or Incomplete, then select Manage connections.
    2. In the Configure Connections dialog, select Update connection references to use your connector to complete connection references.
    3. In Step 2: Power Automate Flows, if a flow is turned off, use the navigation link to open the flow in Power Automate and enable it.
    4. Once all tiles show as Ready, select Publish in Step 3: Copilot Studio Agent to complete the setup.

After completing the configuration, you might need to perform a hard refresh to see the updated status.

If you have issues configuring connection references from the Quality Evaluation Agent page, you can do a manual setup. Follow the steps provided in Connection references for Quality Evaluation Agent flow.

Enable Quality Evaluation Agent

You need to enable Quality Evaluation Agent for your supervisors in Copilot Service admin center. You can select cases, conversations, or email record type. You can also set a scoring and threshold value. Each criteria and question in the evaluation is scored out of 100 points. You can set thresholds to define good and poor-quality metrics.

  1. In Copilot Service admin center, go to Customer Support > Quality management. The Quality management page appears.
  2. Select Manage for Quality Evaluation Agent. The Quality Evaluation Agent page appears.
  3. In the Enablement by record type section:
  4. Select Case, Conversation, or Email and save.
    1. If you select Case and need to perform bulk evaluation for cases, select the Enable bulk evaluations option.
  5. For each record type, in Specify data, you can see the default entities that are added. For Email record type, you need to add the Description in the Data field and save it.
    1. Select Manage data to go to the Specify data dialog, where you can:
      • Delete data types or clear a row to temporarily exclude the data type from being summarized.
      • Add more data type fields for Quality Evaluation Agent input configuration by selecting Add data.

        Note

        You can add up to 10 one-to-one data types and six one-to-many data types in addition to the ones available by default.

  6. In the Evaluation criteria score section, select Enable scoring for criteria. The Evaluation criteria scoring turned on dialog appears.
  7. Select Turn on. You can't turn off this feature after you enable it.
  8. From the Set threshold value out of 100 dropdown list, select a threshold value.
  9. Select Save.

Use evaluation plan
Use on-demand evaluation
Use evaluation criteria
Use evaluations

  • Last updated on