Re: [jgit-dev] JGit Code Review - Report

[Enrico Fregnan <fregnan@xxxxxxxxxx>]

Hi Jonathan,

Thank you very much for your reply.

1) Yes, this was the reason that brought us to create this tool. The idea is having some insights on what is really going between the different patches of review might help to achieve a better understanding of the process in place. 

From this, it would be possible to asses if everything is happening as it’s supposed to or if something could be changed or improved. For instance, too many documentation changes might be a symptom that the style policies of the project are not too clear or that a better style checker should be in place.

As you said, code review has other outcomes too, e.g., knowledge transfer. 

There is a paper published in 2013 in which a group of researchers investigate this aspect of code review at Microsoft (I took the liberty to attach a link to the paper at the end of the email, in case somebody is interested).

2) The idea is that a evolvability change concerns any change that does not immediately impact the functionality of the code itself, but its evolution in the long term; for instance, how the code is organised. A readability change can be generally considered as a documentation change, which is a subtype of the evolvability change.

3) We are currently in the process of preparing a publication on this work, so the code is not available yet. However, I plan to make it open source as soon as possible. I would be happy if you would like to have a look at it when it will be available.

This tool represents just a starting point to create a sort of code review analytics tool. It would be great if you could give us some feedback by compiling the survey in the report to give me a better idea of which features are good, which can be improved and so on.

Thanks a lot.

Best,

  Enrico

Link to the paper I mentioned above: https://www.microsoft.com/en-us/research/publication/expectations-outcomes-and-challenges-of-modern-code-review/

On 19 Aug 2019, at 19:31, Jonathan Nieder <jrn@xxxxxxxxxx> wrote:

Hi Enrico,

Enrico Fregnan wrote:

Dear JGit developers,

I am Enrico, a researcher at the University of Zurich and I am studying how to evaluate the code review process to understand how it could be improved.

I created a tool that analyzes the changes that happen during a review on Gerrit, to measure the effects of code review on code and understand which kinds changes reviewers pay attention to.

For example, are you aware that the majority of the changes happened during a code review in the last year in your project involve documentation issues?

You will find this and more data about your project at the following link:

https://www.surveygizmo.com/s3/5168478/Code-review-changes-report-JGit

The link contains the report and some questions we would like to ask you about it.

Here's a few more details from private discussion:

| The link attached contains a report of our findings on the code
review process of JGit. We analysed which kind of changes happen
during a review, after a developer submits the first patch.
|
| We hope this information might offer some interesting insights to
improve your code review process. For example, noticing that much
effort is spent on fixing styling issue might lead to some rethinking
of the code review policies of the project.
|
| Of course, this is just a first step towards our goal of creating a
code review analytics tool that is able to give a complete summary of
what happens on Gerrit.
|
| You can find some graphs and information in the report. The report
is on survey Gizmo so you can give us some feedback (which would be
much appreciated) if you want, but don’t feel obliged to.
| The main goal of the survey attached is to understand if the
information already present in our tool is useful and can spark some
further reflections or what and how we need to improve.

Some initial thoughts:

1. I like your focus on changes to code that happened as a result of
review. This provides a concrete way to measure one of the effects of
code review. (See "Why Code Review?" at
https://gerrit-review.googlesource.com/Documentation/#_about_gerrit
for a description of a few of the other effects, such as knowledge
transfer.)
2. I am not sure I understand the distinction being made between
evolvability and understandability or readability.
3. Is the code you are using for these analytics available somewhere
we can look at?

Thanks,
Jonathan