I'm not convinced we want the TCK results on the specification
page. Most people are not going to care about the TCK results.
All Compatible Implementations need to publish their TCK summary,
whether they're part of a specification release review or not. We
should find a way for EE4J projects to publish that even if they're
not a CI included with the release. The release review should just
reference the TCK results for the included CI(s).
EE4J projects could publish the results in the GitHub wiki for their
project, or on the web site for their project, or in the GitHub
release record for each release. (Results will need to be published
and maintained for each release as long as that release is
available, which for Maven Central means forever.)
Scott Stark wrote on 7/16/19 12:45 PM:
I would say we add a section to the https://github.com/jakartaee/specification-committee/blob/master/spec_page_template.md that
the project uses as its _index.md that contains the runs of the
TCK against whatever implementations the spec project want to
include as verification.
That can also have a link to the full test
run/instructions if desired, but I don’t see that as critical
for the release review. That information would be included in
the certification request for the implementation.
And where exactly should it be
published?
Scott Stark wrote on 7/13/19 10:34
AM:
I would say just the most concise console
summary is sufficient.
On Jul 13, 2019, at
3:49 AM, arjan tijms <arjan.tijms@xxxxxxxxx
<mailto:arjan.tijms@xxxxxxxxx>>
wrote:
Hi,
I was wondering the exact same thing. Some guidance
would be useful here.
Kind regards,
Arjan
On Sat, Jul 13, 2019 at 12:34 AM Bill Shannon <bill.shannon@xxxxxxxxxx
<mailto:bill.shannon@xxxxxxxxxx>>
wrote:
We're supposed to publish TCK (summary) results in
a "stable" location.
Where should we publish them for an EE4J project?
And what exactly should they include?
For example, I have this job:
https://jenkins.eclipse.org/javamail/job/eftl-mailtck-build-run/
Which includes this junit-format summary:
https://jenkins.eclipse.org/javamail/job/eftl-mailtck-build-run/lastSuccessfulBuild/artifact/junitreports/mailtck-junit-report.xml
(sorry, you might not be able to access that)
Or I could publish this summary output from the
console output:
[javatest.batch] Number of tests completed: 321
(321 pass, 0 fail, 0 errors)
[javatest.batch]
***************************************************************
[javatest.batch] Completed running 321 tests.
[javatest.batch] Number of Tests Passed = 321
[javatest.batch] Number of Tests Failed = 0
[javatest.batch] Number of Tests with Errors = 0
[javatest.batch] Number of Tests Not Run = 0
…
_______________________________________________
jakarta.ee-spec.committee mailing list
jakarta.ee-spec.committee@xxxxxxxxxxx
To change your delivery options, retrieve your password,
or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee
|
