[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [tracecompass-dev] Use specific commit for test CTF traces?
|
Hi Alex,
I agree with everything you said. Once there is a property, we can
easily have a "ctf-testsuite job" to remind us when things start to fail
but they are technically not regressions.
I see the master/gerrit job as a way to detect regressions *in Trace
Compass*. Adding a new failing test is not a regression (well, unless
the test could pass before!). I would see the new ctf-testsuite job a
bit like our tracecompass-stagging job that is meant to catch problems
early with the rest of the release train, Java 8, etc. In fact, to
improve build repeatability, tracecompass-master should probably build
with a specific version of Eclipse 4.5 instead of picking up the latest
from the stagging update site. But in practice, we haven't had a problem
(yet?).
Marc-Andre
On 2015-05-08 02:59 PM, Alexandre Montplaisir wrote:
Hi,
After discussing / thinking about it a bit more, I do realize that
breaking our main builds and effectively blocking our Gerrit queue due
to something not under our control (ctf-testsuite in this case) is a
bit too invasive.
I was given a good example:
We should treat user-reported bugs at a higher priority than
ctf-testsuite edge cases, in general.
If a user reports a bug on the ML or on Bugzilla, we become aware of
it, but it won't break our builds or stop us from committing. However
if a new ctf-testsuite test is introduced, it *will* break our build
right away. This is a bit illogical!
To get out of this situation, it's probably inevitable to have our
master branch point to a specific commit in ctf-testsuite. I think
it's important to keep the supported subset of ctf-testsuite into the
main build, because it can spot regressions. But as far as bringing
new tests, we can do so only manually.
Adding a separate Hudson job on the side to track ctf-testsuite's
master would not even be required, but it would indeed be nice to have.
I can look into adding a property in the Ant script to point to a
specific commit.
Does this cover everyone's concerns?
Cheers,
Alex
On 2015-05-08 08:38 AM, Bernd Hufmann wrote:
Hello
could we have a special Jenkins Job that runs in the night and just
verifies the ctf-testsuite's master branch? If there are new traces
added and it fails then we would know right away. Nobody would need
to check it manually.
For Trace Compass master and stable branches, however, we would build
from a specific commit so that we don't get these surprise test
failures when submitting patches to Gerrit or nightly build.
It should not be hard to setup such a Jenkins job. First master
branch is checked out then we could apply a patch before running the
build.
/Bernd
On 05/08/2015 12:09 AM, Alexandre Montplaisir wrote:
Hi Marc-André,
I was thinking about the same thing, after today's breakage ;)
The advantage of always pulling the ctf-testsuite's master branch is
that we don't have to think about it, new tests come in and we get
notified right away if we are non-conforming. Making this process
manual would add more burden to the maintainers.
But having the possibility to point to a specific commit could be
useful. I would still leave master pointing to ctf-testsuite's HEAD,
but in stable branches we could point to a specific commit instead.
That way those won't break unexpectedly, and as you mention, it
guarantees that the branch keeps working in the future.
Thoughts?
Cheers,
Alex
On 2015-05-07 10:47 PM, Marc-André Laperle wrote:
Hi,
I'm wondering if we should use a specific commit when pulling the
ctf-testsuite when running the tests. I think it would be good
because it would let older Trace Compass commits still execute
tests properly and would also reduce the risk of "bad surprises",
especially around release ;) We can periodically update that
commit, whenever we have time to tackle more CTF improvements.
Let me know what you think,
Marc-Andre
_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev
_______________________________________________
tracecompass-dev mailing list
tracecompass-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or
unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/tracecompass-dev