Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] GerritForge's JGit E2E tests with Gerrit in 2022

On Jan 20, 2022, at 4:46 AM, Luca Milanesio <luca.milanesio@xxxxxxxxx> wrote:

I don't understand why another branch is needed for implementing that.
Instead these tests should be run
on the open changes.

You do need a "stream of changes together" (to highlight that the focus is NOT the branch) because when you run the integration tests, you need to see and check things working together, not in isolation.

Let me give you a simple example:
  • Change Ifoo removes a foo() method in JGit that is not used by anyone
  • Change Ibar introduces a new bar() method based on the foo() method
  • You verify Change Ifoo and Ibar in isolation (NOT streamlined) and they both get the Verified +1 and merged. 
Verifying all individual changes in isolations won't tell you if they work together.

With the "stream of changes", you will do instead:
  • Change Ifoo removes a foo() method in JGit that no one uses.
  •  Change Ibar introduces a new bar() method based on the foo() method.
  • Change Ifoo and Ibar are put (cherry-picked or rebased) to the "validation stream" and tested together. The build breaks, and none of the two changes are validated.
The above example is very simplistic. The problem is a lot more complex than that; however, it shows that creating a "stream of changes" for validation is key when you want to integrate and validate multiple components.

There is also an issue with execution times and tradeoffs with the speed of validating incoming changes.
Running E2E tests takes a long time (hours) and is very expensive (hundreds of $ of infrastructure time): you want to limit that to ONLY those changes that:
- Have passed the normal verification process (e.g. Eclipse's CI)
- Have been reviewed and have no vetos from being approved

The two aspects depicted above represent the reason why the trigger for running the E2E tests *cannot* be applied for all open changes targeting master.

(Resending from the email address subbed to jgit-dev)

Matthias replied to most other aspects here, but I wanted to point out that this “stream of changes” has also been commonly called a “batch of changes” and it just so happens a Gerrit plugin exists to make validating a batch of changes easier:

That said, the current upstream state of that plugin lags significantly behind our internal version, but fixing that is a top priority for my team this quarter. I would strongly encourage use of a plugin like this over an actual branch, since the plugin addresses the needs of the use case without the downsides of an actual branch.


Back to the top