Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] Flaky tests in the master branch

Hi Matthias,

Thanks for your reply!
On 10/04/2024 23:51, Matthias Sohn wrote:
Some of the tests (e.g. RacyGitTests) depend on timing which may
depend on hardware and OS.
Appreciate any improvement with flaky tests. AFAIK all JGit committers
use either MacOS or Linux.

Would it make sense to disable or even remove those tests? I could
submit a patch for that. Of course it would be better to fix the tests
so that they keep testing the intended functionality but pass reliably.
While embarking on a journey to debug a race condition in tests which
cover functionality I am unfamiliar with sounds like an interesting
challenge, it's unlikely I can afford to take time for that unfortunately.

Given that this seems to be a hurdle for other new contributors (not
just for me), I would say that the detrimental effect of those flaky
tests on contributor experience may outweigh the potential benefit of
them catching a regression in the future. But it's not an easy call to
make for sure.

By the way, I took the time to write up a blog post about my overall
experience contributing to JGit, in case you are interested (it's
something I do for other projects too, I don't mean to single JGit out):

To summarize, the main hurdles I encountered were:

- difficulty to identify where to submit issues and patches

- Eclipse making changes to tracked editor config files

- unreliable test suite

I hope it provides interesting insights :) Thanks a lot for your work on
this library!



Back to the top