Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
[This article is prerelease documentation and is subject to change.]
Use natural language to define up to three custom metrics you want Copilot Studio to create. Tailor your agent's custom metrics to measure agent outcomes and business impact on conversational messages. Display your custom metrics graphically as labeled donut graphs under Custom metrics on the Analytics page.
Important
This article contains Microsoft Copilot Studio preview documentation and is subject to change.
Preview features aren't meant for production use and may have restricted functionality. These features are available before an official release so that you can get early access and provide feedback.
If you're building a production-ready agent, see Microsoft Copilot Studio Overview.
Key value proposition
This feature uses AI to create a prompt that calculates results of a custom metric you define by using natural language. The metric applies to a sampling of conversational message (session) data. You can fine-tune and iterate on the metric before saving it in your agent. Create up to three custom metrics per agent. You can test metrics by using sample session data and review results by using built-in reasoning. You can then refine the metrics to better reflect outcomes and business impact.
Create a custom metric
On your agent's Analytics page, select Custom metrics above the Summary card.
In Create a custom metric, describe what you want to measure in plain language, in terms of what outcomes you want to track in all conversations.
For each possible result you want for the metric you defined, add a result category and describe how the AI should recognize when this metric result occurs.
Note
- For each metric, you can define up to 10 result categories.
- Any unmatched session results are mapped to the default category Other.
Select Next.
The AI uses your metric and result descriptions, and your agent's existing metadata, to create a prompt. The prompt calculates the new metric's results based on session data. The new custom metric's tile appears in the Preview tab.
Preview and iterate on your custom metric's prompt. The prompt is used to calculate the custom metric results.
Preview and iterate on your custom metric's prompt
Use the custom metric preview functionality to fine-tune your metric's prompt. When testing your custom metric on the Create a custom metric panel, the metric draws on data from a sampling of up to 20 sessions.
On the first page of Create a custom metric, select Next.
Select Preview to view how your custom metric looks when displayed graphically on a tile.
Select Results to view a list of results for the sampled data and a description of the AI's reasoning for each result.
Evaluate whether the reasoning and results are correct.
Select Prompt to see a description of the AI's understanding of what you want the custom metric to measure and calculate and with what result categories. The Prompt tab provides transparency and insight into what the underlying logic is being used to generate the results.
If you want to tune your prompt to give more precise or more relevant results, make adjustments to the inputs you initially provided on the first page of the Create a custom metric panel. Then retest the adjusted prompt with the sample data.
Select Save metric.
Delete a custom metric
On your agent's Analytics page, on the custom metric tile you want to delete, select the three dots (…), and then select Delete.
On the confirmation dialog, select Delete.