Hi,
I see that the benchmarks are causing issues again.
To be fair it's not just the benchmarks, I'm seeing issues everywhere the maven shade plugin is used at the moment. Working on it.
Since those benchmarks were created I’ve started creating benchmarks in the test directory, so that they aren’t packaged with the release. The only downside is that the benchmarks now have to be run through intellij, since there is no benchmark.jar being built.
I'm not sure I follow - why would packaging them with the release be a concern? We can't redistribute the jmh libraries themselves, but our own code is fine to be distributed (in fact, it _is_ being distributed, rdf4j-benchmark is a maven module available from Central).
What I would really wish for would be a jenkins plugin that runs the benchmarks and lets you know if you are within 5-10% of the baseline for your pull request.
Not sure we can do something like this for every pull request, resource-wise, but a setup that runs a nightly benchmark trend report would be very useful. There's plugins for this. Main problem I'm facing is lack of time to properly dive into how to configure this.
Jeen
