Yes, but we still need to decide what to do for EE4J projects.
Scott Stark wrote on 7/16/19 1:44 PM:
That is fine and gets away from the problem David is describing on
the other thread. Then the PR can just have a checkbox for
including one ore more TCK summary results.
I'm not
convinced we want the TCK results on the specification
page. Most people are not going to care about the TCK
results.
All Compatible Implementations need to publish their TCK
summary, whether they're part of a specification release
review or not. We should find a way for EE4J projects to
publish that even if they're not a CI included with the
release. The release review should just reference the TCK
results for the included CI(s).
EE4J projects could publish the results in the GitHub wiki
for their project, or on the web site for their project,
or in the GitHub release record for each release.
(Results will need to be published and maintained for each
release as long as that release is available, which for
Maven Central means forever.)
Scott Stark wrote on 7/16/19
12:45 PM:
I would say we add a section to the https://github.com/jakartaee/specification-committee/blob/master/spec_page_template.md that
the project uses as its _index.md that contains the runs
of the TCK against whatever implementations the spec
project want to include as verification.
That can also have a link to the full test
run/instructions if desired, but I don’t see that as
critical for the release review. That information
would be included in the certification request for the
implementation.
And
where exactly should it be published?
Scott
Stark wrote on 7/13/19 10:34 AM:
I would say
just the most concise console summary is
sufficient.
On Jul 13,
2019, at 3:49 AM, arjan tijms <arjan.tijms@xxxxxxxxx
<mailto:arjan.tijms@xxxxxxxxx>>
wrote:
Hi,
I was wondering the exact same thing. Some
guidance would be useful here.
Kind regards,
Arjan
On Sat, Jul 13, 2019 at 12:34 AM Bill
Shannon <bill.shannon@xxxxxxxxxx
<mailto:bill.shannon@xxxxxxxxxx>>
wrote:
We're supposed to publish TCK (summary)
results in a "stable" location.
Where should we publish them for an EE4J
project?
And what exactly should they include?
For example, I have this job:
https://jenkins.eclipse.org/javamail/job/eftl-mailtck-build-run/
Which includes this junit-format summary:
https://jenkins.eclipse.org/javamail/job/eftl-mailtck-build-run/lastSuccessfulBuild/artifact/junitreports/mailtck-junit-report.xml
(sorry, you might not be able to access
that)
Or I could publish this summary output
from the console output:
[javatest.batch] Number of tests
completed: 321 (321 pass, 0 fail, 0 errors)
[javatest.batch]
***************************************************************
[javatest.batch] Completed running 321
tests.
[javatest.batch] Number of Tests Passed
= 321
[javatest.batch] Number of Tests Failed
= 0
[javatest.batch] Number of Tests with
Errors = 0
[javatest.batch] Number of Tests Not Run
= 0
…
_______________________________________________
jakarta.ee-spec.committee mailing list
jakarta.ee-spec.committee@xxxxxxxxxxx
To change your delivery options, retrieve your
password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee
|
