Agent effectiveness

The Agent effectiveness report brings together agent usage, Agent Experience Score, and Model fit for local coding-agent sessions captured by AI Code Insights. Use it to understand what work agents are doing, find interaction friction, and explore the models, Skills, and MCP servers used in those sessions.

Note: This report requires AI Code Insights, Transcripts & Agent Experience enabled, and permission to view individual metrics in reports. Transcript access is controlled separately.

Agent effectiveness report showing Agent Experience Score, agent work, and model usage

Requirements

This report requires the AI Code Insights daemon and Transcripts & Agent Experience enabled under Admin → AI Code Insights. It shows captured local agent sessions; coverage depends on your daemon rollout and supported agent capabilities.

Transcript access is controlled separately. See Agent sessions access and configuration for permissions on session titles, summaries, evaluation comments, and conversations.

When to use Agent effectiveness

This report helps you:

  • Find interaction friction — Compare requirements, steering, and scope ratings, then inspect sessions behind low scores.
  • Understand agent work — Track the types of work agents perform and how that mix changes over time.
  • Explore model choices — See which models are used and whether they match the complexity of the work.
  • Track Skill and MCP usage — See which Skills and MCP servers appear in sessions and how often they are invoked.
  • Compare teams and groups — Review experience scores alongside session volume for the same period and cohort.

Filters and time periods

Use the date range, team or group, attribute, and Tools filters to choose which sessions to include. Use the display controls to change the time interval for trend charts.

Sessions are grouped by their start time. The report includes top-level sessions; captured subagent tasks can be inspected within their parent session’s activity.

Agent Experience Score

The Agent Experience Score section shows the mean score for the selected period, its trend over time, the distribution of session scores, and average ratings for Requirements, Steering, and Scope.

After an eligible session ends, DX evaluates its transcript with a separate model. Under the current rubric, each dimension receives a 1–5 rating, and the session score is:

(Requirements + Steering + Scope) ÷ 3

See the Agent Experience Score scoring rubric for evaluation details and limitations.

Score distribution shows the number and percentage of scored sessions in four bands: 1.0–1.9, 2.0–2.9, 3.0–3.9, and 4.0–5.0. Select a band or a point on the score trend to inspect its sessions, or select View all sessions to explore the selected cohort.

Sessions without an evaluation do not contribute to the score. Missing scores should not be treated as low or high scores.

Agent work

Agent work by session shows work volume over time. Switch between Work types and Categories to explore detailed tasks or broader groups, such as exploration, features, fixes, and maintenance.

A session can have up to three work types, so counts can add up to more than the number of sessions. Select a chart segment or a work type or category in the list to inspect the sessions behind it.

See Agent work for how sessions are classified and the complete list of work categories.

Model usage by session

Model usage by session shows session counts over time, grouped by the primary model used. Sessions without a recorded model appear under Unknown.

Select a chart segment or model in the list to inspect its sessions. These counts describe model use across captured sessions, not token consumption or spend.

Model fit (beta)

Model fit compares session task complexity with the model used, grouped by work type and model. It highlights potential savings from using a lower-cost model and potential quality improvements from using a more capable one.

Select a result to inspect the sessions behind that work type and model pairing. See Model fit for assessment rules, supported models, and how individual session results relate to aggregate findings.

Skill and MCP usage (beta)

Skill usage and MCP usage show usage over time, grouped by Skill name or MCP server. Each chart has two views:

  • Sessions counts distinct sessions that used each Skill or MCP server.
  • Usage counts every Skill invocation or MCP tool call, including repeated use within the same session.

For example, a session that invokes the same Skill three times contributes 1 to that Skill’s Sessions count and 3 to its Usage count. A session can use several Skills or MCP servers, so counts across names are not a unique session total.

Select a chart segment or name in the list to inspect the associated sessions. Availability depends on the agent integration’s tool-usage support.

Breakdown by team or group

The breakdown table shows Agent Experience Score, Requirements, Steering, Scope, and Sessions by team, group, or selected attribute. Scores are means across evaluated sessions; the session count also includes sessions without scores.

Select a row to inspect its sessions. Account anonymity settings can hide the report or individual rows when too few contributors are represented.

Inspect session evidence

Drilldowns connect report patterns to individual sessions. Select a session to review its summary, evaluation, Model fit, activity, and linked delivery work. See Agent session details.

Use the score, work type, and model together to choose a specific workflow improvement, such as clearer task context, a smaller task, or a different model. Compare the same cohort and work mix after making the change. Agent Experience Score evaluates the interaction; it does not measure code quality, employee performance, or business impact.