How to Measure AI Adoption Across a Company
Measure AI adoption by what agents actually do in company tools rather than by license counts or logins. Track how many people have connected at least one app, which tools are called and how often, which teams use them, and which calls fail. That data comes from the layer agents use to reach company apps. In Metorial, every tool call is logged with the person, the tool, and the result, so these numbers come from real usage.
Most AI adoption reports count seats. Seats tell you what was bought. What you want to know is whether agents are doing work in the systems where work happens, and that is visible only where agents connect to those systems.
Which metrics matter?
Connected people. The share of a team that has connected at least one app. This is the first sign that AI is set up for work rather than tried once.
Tool calls by tool. Which tools are called and how often. This shows what AI is used for, and which published tools nobody needs.
Tool calls by team. Where adoption is happening. A team with many calls is a source of workflows for other teams.
Failed calls. Calls that errored, by tool and by reason. These are the fastest thing to fix, and they often explain a team that stopped using AI.
Shared workflows in use. How often shared skills run. A skill that runs every day is a candidate to publish to more teams.
Where does the data come from?
From the layer between agents and company apps. If agents connect to apps directly from laptops, there is no central record and you are back to surveys.
In Metorial, every call is recorded in Tracing and the audit logs with the tool, arguments, result, session, and the person. Each Workforce account shows that person's Magic MCP servers, operations, and connections, and each agent has its own view of tool calls and connections.
How often should you review it?
Weekly during a team's first month, then monthly. Early reviews are for fixing: broken connections, missing permissions, tools nobody calls. Later reviews are for deciding which workflows to spread.
What should you report to leadership?
Keep it per team and tie it to work: how many people on each team have AI connected to their tools, which workflows run most, and what changed since the last report. Avoid per-person rankings, which make people stop using the approved setup.
Frequently asked questions
Why are license counts a poor adoption measure?
A license shows who could use AI, not who does. Even logins only show that someone opened the chat. Tool calls into company apps show AI doing work.
What is a good first adoption metric?
The share of people on a team who have connected at least one app. It is simple, and it separates people who tried a chat window from people who set AI up for their work.
How do we find out why a team is not using AI?
Look at failed calls and at tools nobody calls. Failed calls usually mean a missing permission or a broken connection. Unused tools usually mean the wrong tools were published.
Can we measure time saved?
Not directly from logs. Logs tell you which workflows run and how often. Pair that with a short estimate from the team of how long each workflow took by hand.
Should we measure adoption per person?
Use per-person data to fix problems, such as a broken connection, not to rank people. Report adoption to leadership per team.