OneRange

AI Skills Are Now in Performance Reviews. Here's How to Evaluate Them Fairly

· 9 min read · OneRange Research

Meta, Shopify, Duolingo, and Accenture now weigh AI in reviews or promotions. Why usage metrics backfire, and how to evaluate AI skills fairly instead.

Data card: 91% of employees say managers aren't ready to guide AI skills, as companies add AI to performance reviews

In November 2025, Meta told employees that "AI-driven impact" would become a core expectation in 2026 performance reviews. It wasn't the first. Shopify and Duolingo added AI use to reviews in 2025, and in February 2026 Accenture told senior staff that use of its key AI tools would be "a visible input to talent discussions" for promotions.

AI skill is becoming part of how performance is judged. Most companies haven't worked out how to judge it well, and the early attempts show what goes wrong.

Who is tying AI to performance

CompanyWhat they didWhen
ShopifyCEO Tobi Lütke: "Using AI effectively is now a fundamental expectation." AI usage questions added to performance and peer reviewsApril 2025
DuolingoCEO Luis von Ahn: AI use will be part of hiring and of what is evaluated in performance reviewsApril 2025
MicrosoftDeveloper division president Julia Liuson: "Using AI is no longer optional" and should be part of managers' reflections on performanceJune 2025
CoinbaseCEO Brian Armstrong required engineers to onboard to AI coding tools within a week and let go a few who didn't have a good reasonAugust 2025
MetaAI-driven impact becomes a core expectation in 2026 reviewsAnnounced November 2025
ZapierPublished a four-level AI fluency rubric; "Capable" is now the minimum hiring barFebruary 2026
AccentureUse of key AI tools becomes a visible input to promotions for senior staff, after training 550,000 peopleFebruary 2026

Broader surveys point the same way. One survey of 1,295 companies, reported by IT Brew, found 58% require at least some employees to use AI.

What goes wrong when you measure usage

The easiest thing to measure is how much people use AI. It's also the easiest thing to game.

  • Amazon shut down an internal token leaderboard in May 2026 after employees made up tasks to inflate their numbers. The company said it "was never intended to promote the use of AI for usage's sake"
  • At Meta, an employee-built dashboard ranking colleagues by token use, with titles like "Token Legend", was shut down in April 2026 two days after it was reported
  • Uber spent its entire 2026 AI coding-tools budget in about four months, and its COO said token costs are hard to justify without a direct line to useful features

SHRM summed up the problem: "'Are they using it?' is an adoption question. 'Tokens per quarter' is a performance metric." Usage tells you who has adopted AI. It doesn't tell you who is good at it, and rewarding volume rewards waste.

Most employees and managers aren't ready

  • 35% — Employees with no AI training at all (Study.com, 2026)
  • 18% — Of trained employees who feel ready to work with AI on their own (Study.com, 2026)
  • 83% — Say there is a gap between reported and demonstrated AI skill (Acorn, 2026)
  • 91% — Employees who say managers aren't ready to guide AI skills (Acorn, 2026)

Acorn also found 77% of organizations treat finishing a course as proof of capability. That's the second trap: completion isn't skill. If you rate people on AI, you need a way to see what they can actually do.

There's a fairness risk too. A 2025 study of 1,026 engineers found women who used AI received 13% lower competence ratings from peers, against 6% for men. If managers judge the tool rather than the work, AI in reviews can widen existing gaps.

A fairer way to evaluate AI skills

Measure thisNot this
Skill level from a consistent assessment (for example Beginner to Expert)Logins, prompts, or tokens per week
Outcomes on real work: time saved, quality, error ratesSelf-reported hours saved with no baseline
Repeatable AI workflows the person built and others adoptedOne-off impressive demos
Judgment: checking output, protecting data, knowing when not to use AIUsing AI on everything
Growth since the last reviewCourse completions
  1. Train before you rate. Make sure everyone has had training and access to approved tools before AI affects a rating. Accenture trained 550,000 people before tying AI use to promotions
  2. Set a baseline. Assess each person's AI fluency at the start of the cycle with the same instrument for everyone. Our AI fluency assessment shows what that looks like
  3. Define good by role. What strong AI use looks like for a recruiter differs from an engineer. Write two or three role-specific examples per level. Our post on proficiency levels that mean something covers how
  4. Judge outcomes and judgment. Ask for evidence of results and of good calls: output checked, data kept safe, AI skipped when it wasn't the right tool
  5. Calibrate across managers. Review ratings side by side to catch bias, including penalizing people for using AI at all
  6. Keep usage data for adoption. Track it at team level to see where support is needed, not as an individual score

Zapier's rubric is a useful public example. It separates people who use AI occasionally from those who build "repeatable systems, not one-off prompts" with "clear impact on quality, efficiency." That's the right instinct: grade what AI helps someone achieve, not how often they open it.

Managers need training first. Our guide on how to measure AI fluency is a good primer, and the new managers page covers AI training for people leaders.

How OneRange Vero helps

Vero is interactive AI training with skill assessments built in. It gives managers and HR something better than usage counts to bring to a review.

  • Skill assessments that place each person at Beginner, Intermediate, Advanced, or Expert for each AI skill, with strengths and areas for improvement
  • Results that write to each employee's skill profile, so progress between review cycles is visible
  • Workforce Results for managers, showing where each team stands and who needs support before ratings are due
  • Role-based AI training to close the gaps the assessment finds

Sources

Tags: AI, AI Fluency, Performance Management, Learning & Development

FAQ

Frequently asked questions

Are companies really including AI in performance reviews?

Yes. Meta made AI-driven impact a core expectation in 2026 reviews, Shopify and Duolingo added AI use to performance reviews in 2025, Microsoft told managers AI should be part of their reflections on performance, and Accenture made use of its AI tools a visible input to promotion discussions.

Should we measure AI usage or token counts in reviews?

Not as a performance score. When Amazon and Meta ran internal token leaderboards, employees gamed them and both were shut down. Usage data is useful for tracking adoption, but it measures activity, not skill or results.

What should we measure instead of AI usage?

Measure proficiency and outcomes: a skill level from a consistent assessment, plus role-specific results such as time saved on a workflow, quality or error rates, and new AI workflows the person built that others adopted.

Is it fair to evaluate AI skills if employees weren't trained?

It's risky. Study.com found 35% of employees have had no AI training. Give people training, tool access, and a baseline assessment before AI skills affect their ratings.

Can evaluating AI use introduce bias?

Yes. A 2025 study of 1,026 engineers found women who used AI received 13% lower competence ratings, versus 6% for men. Judge the work, not the tool, and calibrate ratings across managers.

Are managers ready to assess AI skills?

Mostly not. Acorn found 77% of executives believe managers are ready to guide AI skills development, while 91% of employees disagree. Train managers on the rubric before the review cycle.

Related