Graphite computes stats for each user as an author and as a reviewer. Stats are computed on merged PRs only.

​
Methodology

Active users: For a user to be active in the selected time frame and repos they must have:

  1. created a PR on GitHub to be considered active on GitHub or

  2. created a stack of 2+ PRs on Graphite to be considered active on Graphite

Rollups: The graphed metrics are weekly rollups, where weeks start on Monday and end on Sunday. Rollups are medians-of-medians, where we first take each individual active user’s median value for the week, then take the median of those values.

Calculating pre-Graphite median: For each active Graphite user in the selected time frame, we look at the median of each user’s 28 days of usage on GitHub before they ever created a stack with Graphite (in the selected repos). Then take the median again across all users. The goal is to create a fair benchmark of performance before these users starting using Graphite

Minimum 5+ active Graphite users: We won’t attempt to draw comparisons between developers using Graphite or GitHub if there are fewer than 5 active Graphite users in the selected timeframe.

​
Author stats

  • PRs merged: Total number of PRs where the user was an author

  • Time waiting on reviews: Median time of PRs authored by the user (that have open requests for reviews) were sitting without any reviews

  • Wait time to first review: Median time of a PR being marked as ready for review for the first time, and someone reviewing it

  • Publish to merge time: Median time between a PR being marked as ready for review for the first time, and it being merged

  • Review cycles until merge:

    • Computes the max number of review cycles any reviewer had until the PR was merged

    • Then across all PRs, computes the median of this number

​
Reviewer side stats

  • PRs reviewed: Total number of PRs that the user left a review on

Response time to review requests: Median of the difference in time between the user being requested (or re-requested) for a review, (referred to as review request/review pairs) for the given user

​
Insights example

Here’s an example of a sequence of events and how that activity will impact Insights stats:

  1. User X authors a PR in draft state that adds 10 lines of code, and deletes 5 lines of code

  2. User X publishes the PR, making it ready for review

  3. User X requests reviews from User A, User B, and User C

  4. User A requests changes on the PR

  5. User B leaves a review comment on the PR

  6. User X makes updates, and re-requests reviews from User A and User B

  7. User A leaves a review comment on the PR

  8. User B leaves a review comment on the PR

  9. User X makes updates, and re-requests review from User A

  10. User A approves the PR

  11. User X merges the PR

These events will affect each stats in the following ways:

  • PRs merged will go up by 1 for User X

  • PRs reviewed will go up by 1 for User A and User B (User C did not review, so no change there)

  • Response time to review requests will be computed for each reviewer on each review request/review pair as (*review time *- review request time). This will then feed into the reviewer’s aggregated median statistic.

    • User A response time for first review request = time at event #4 - time at event #3

    • User B response time for first review request = time at event #5 - time at event #3

    • User A response time for second review request = time at event #7 - time at event #6

    • User B response time for second review request = time at event #8 - time at event #6

    • User A response time for third review request = time at event #10 - time at event #9

  • Time waiting on reviews will be computed based on how long the author was waiting on reviews for that PR. This then feeds into the author’s aggregated median statistic.

    • Time waiting on reviews on the PR above for User X as author = (time at event #4 - time at event #3) + (time at event #7 - time at event #6)

    • Note that we didn’t count the time between event #10 and event #9 here because there was a review from User B on the PR—so the author wasn’t waiting on reviews during this time period. In case User X re-requested review from both User A and User B in event #9, we would also add (time at event #10 - time at event #9) into the computation above.

  • Wait time to first review will be computed based on how long the author waited until anyone reviewed their PR after it was published/ready for review. This will then feed into the author’s aggregated median statistic.

    • Wait time until first review on above PR for User X as author = time at event #4 - time at event #2

  • Publish to merge time will be computed based on how long it took for the PR to be merged after it was published/ready for review. This will then feed into the author’s aggregated median statistic.

    • Publish to merge time on above PR for User X as author = time at event #11 - time at event #2

  • Review cycles until merge for a PR will be computed as the max number of review request/review pairs that a reviewer had until the PR was merged. This will then feed into the author’s aggregated median statistic.

    • Review cycles for User A as reviewer = [(event #10, event #9), (event #7, event #6), (event #4, event #3)] (count = 3)

    • Review cycles for User B as reviewer = [(event #8, event #6), (event #5, event #3)] (count = 2)

    • Max review cycles on above PR for User X as author = max(3, 2) ⇒ 3

  • Lines of code added will go up by 10 for User X

  • Lines of code deleted will go up by 5 for User X