# GitHub — GitHub Copilot Evals is now in public preview

- Company: GitHub (github.com)
- Announced: 2026-11-17T18:00:33+00:00
- Category: developer-tool-release
- Coverage: not counted
- Announcement: yes
- Group: announcements
- Source: https://github.blog/changelog/2026-11-17-github-copilot-evals-is-now-in-public-preview
- Record: https://forck.live/items/37455-github-copilot-evals-is-now-in-public-preview
- Subject: Copilot

GitHub has opened a public preview of GitHub Copilot Evals, a tool that lets people test whether their own Copilot customizations still produce good results. An eval is a repeatable test for AI behavior: you give an agent a realistic task, describe what a good result looks like, and check both the result and how the agent got there. Because AI output can change from one run to the next, GitHub says a single good answer does not show whether a customization works, so users can rerun the same eval after changing a skill, agent or MCP tool to catch regressions. Users can ask Copilot in their own words or use the /eval command to create an eval for a skill, custom agent or MCP tool in their project, then run it and see which checks passed and failed, with each run approved before it starts. Results can show why a check failed and let users compare runs from before and after a change. Evals uses Vally, an extensible local eval tool, as its evaluation engine, and installs as a Visual Studio Code extension, a Copilot CLI plugin, or a plugin from the github/evals marketplace in the Copilot app. The public preview covers Copilot Individual, Copilot Business and Copilot Enterprise plans.

## Evidence

Verbatim from https://github.blog/changelog/2026-11-17-github-copilot-evals-is-now-in-public-preview:

> You can now use GitHub Copilot Evals to optimize GitHub Copilot for yourself. As your prompts, models, skills, and tools change, are you still getting the most out of Copilot? Evals helps you answer that question for the skills, agents, and MCP tools in your setup. Create an eval, inspect how they behave, and use the results to decide what to improve.
> 
> An eval is a repeatable test for AI behavior. You give an agent a realistic task, describe what a good result looks like, and check both the result and how the agent got there.
> 
> AI output can change from one run to the next, so one good answer doesn’t tell you whether a customization works well for you. After changing a skill, agent, or MCP in your Copilot setup, rerun the same eval to check the behavior you care about and catch regressions.
> 
> Ask Copilot in your own words or use the /eval command to:
> 
> Create an eval for a skill, custom agent, or MCP tool in your project.
> 
> Run it and see which checks passed and failed. You approve each run before it starts.
> 
> Find out why a check failed and decide what to change in your Copilot setup.
> 
> Compare runs from before and after a customization change to see whether it helped.

---

Record: https://forck.live/items/37455-github-copilot-evals-is-now-in-public-preview
Catalogue: https://forck.live/llms.txt
Current issue: https://forck.live/feed.md
