jsonscraper
Markets
Back to journal

GitHub Adds a Metric for Copilot Code Review Comments Developers Address

Organization reports now show actions taken on AI review comments and help identify gaps in editor telemetry.

Editorial Policy Report an error

As of November 5, teams using GitHub Copilot Code Review can see in API reports which comments developers have acted on. That same day, GitHub added fields to help administrators find telemetry gaps and outdated editor versions. These updates give engineering leaders and organization owners a more concrete way to assess how AI review is working and how complete their Copilot usage data is.

Article: GitHub Adds a Metric for Copilot Code Review Comments Developers Address
Summary: On November 5, GitHub expanded its Copilot usage metrics API. Organizations can now assess which Copilot Code Review comments developers act on and identify users with incomplete data due to settings or outdated tools.
Image role: cover — the central idea
Section: 
Visual subject: Editorial photograph of printed software pull request pages and a marked-up code revi
Updated GitHub reports distinguish actions on Copilot comments from the completeness of editor usage data.© jsonscraper · AI-generated

How to measure responses to AI comments

The daily repository report row, repos-1-day, now includes a copilot_code_review object. Its fields include total_comments_evaluated, the number of comments reviewed, and total_comments_addressed, the number of comments developers responded to. To calculate the share of addressed comments for a repository, divide the second value by the first. The new fields are described in the Copilot usage metrics API update.

GitHub counts a comment as addressed if a developer applies the suggested fix, asks Copilot to make it, edits the code themselves, or replies that the fix will be included in the next pull request. The report also breaks comments down by severity, review level, comment type, and whether additional instructions or tools were involved. This lets teams investigate, for example, whether responses to security comments differ from responses to other comments, or which categories require more manual work.

The metric shows a recorded action, not a verified fix. It is therefore useful to read it alongside code review and pull request status: a high share of addressed comments indicates that comments received a response, not that all changes are correct or necessary. Another important detail: comments are counted on the day the pull request is merged, not the day Copilot posted them.

Why the addressed rate differs from suggestion acceptance

Previous fields tracked Copilot Code Review runs and instances where a user clicked the button to apply a ready-made suggestion. The new metric covers more ways to respond: developers can make a fix themselves, ask Copilot to do it, or defer the change to the next pull request. So total_comments_addressed answers a question about actions taken on comments, while total_copilot_applied_suggestions counts only suggestions that can be applied with a single action.

The report also includes total_merged_no_human_change_request_after_review: the number of merged pull requests reviewed by Copilot after whose first review no human requested changes. These metrics describe different stages of the process and are best read together: the first reflects responses to comments, while the second tracks the review process through to merge.

New fields show why reports may be incomplete

A separate November 5 update adds information about client telemetry and tool versions. The client_telemetry_present field indicates whether events from a user's editor were received on a given day; usage_metrics_requirements indicates whether the IDE and extensions meet the minimum versions required for metrics. Administrators also get daily counts of users without telemetry and users with at least one outdated component. Details are listed in the note on Copilot data gaps.

This helps distinguish user activity from reporting completeness. GitHub explains that a user may still count as active even if editor events were not received—in that case, the report loses details such as usage breakdowns by feature and model. Possible causes include an outdated IDE or extension, disabled editor telemetry, or a proxy or network firewall blocking events.

The two gap counters should not be added together to estimate the number of unique users: one person may appear in both groups. In addition, an up-to-date editor does not guarantee that telemetry is received; network restrictions, for example, may still apply. If a value is missing or set to null, GitHub treats it as unavailable data, not as zero or evidence that no issue exists.

What this means for teams

The updates let teams replace a broad Copilot activity metric with two more practical checks: what happens to Code Review comments and how reliable the data behind internal reports is. Engineering leaders can compare comment types and repositories, while administrators can identify users who may need an IDE or extension update or a network settings check.

Reports are available at the organization and enterprise levels to users with the appropriate Copilot metrics viewing permissions; the Copilot usage metrics policy must be enabled. Code Review data appears in repos-1-day rows, while telemetry and version fields appear in user reports covering one and 28 days. This makes the update especially useful for organizations that already administer Copilot centrally: they can now investigate not only whether the assistant is being used, but also how developers respond to its reviews and why measurements may be incomplete.

People

No people listed for this article yet.

Keep readingGitHub Separates Copilot BYOK Usage in Organization Reports
Read the next article

Turn what you read into a working integration

Explore jsonscraper's social-data APIs, test requests and build your next workflow.

Explore APIs