Re: [jakartaee-tck-dev] Input on user requirements for specifying Specif

[Scott Marlow <smarlow@xxxxxxxxxx>]

https://www.eclipse.org/lists/jakartaee-tck-dev/msg00927.html has more information about the TCK tooling for assertions. 

If someone wants to volunteer to deal with documentation assertions with TCK tooling, please speak up.  Otherwise i think the alternative is for new tests to manually update test assertions in the TCK sources (https://github.com/eclipse-ee4j/jakartaee-tck/pull/759 did this but may need some more changes).

Scott

On 11/3/21 6:18 PM, Scott Marlow wrote:

The current TCK test output report files contain the assertion_ids value as determined while the test is running.  An example value from com.sun.ts.tests.jpa.core.criteriaapi.CriteriaBuilder.Client:
"
assertion_ids=PERSISTENCE\:JAVADOC\:735 See assertion html documents.\nPERSISTENCE\:SPEC\:1766 See assertion html documents.\nPERSISTENCE\:SPEC\:1766.1 See assertion html documents.
"

The source https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/src/com/sun/ts/tests/jpa/core/criteriaapi/CriteriaBuilder/Client.java#L2603 contains:

"

/*
   * @testName: all
   *
   * @assertion_ids: PERSISTENCE:JAVADOC:735; PERSISTENCE:SPEC:1766;
   * PERSISTENCE:SPEC:1766.1;
   *
   * @test_Strategy: convert the following JPQL into CriteriaQuery
   *
   * Select hardProd.modelNumber From HardwareProduct hardProd where
   * hardProd.modelNumber > ALL ( Select subHardProd.modelNumber From
   * HardwareProduct subHardProd where subHardProd.modelNumber < 10050 )
   *
   *
   */

"

The ^ assertion is defined in the JPA API assertions html doc file https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/install/jpa/docs/assertions/api/api-assertions-3_0.html#L3126 (

PERSISTENCE:JAVADOC:735

_expression_

jakarta.persistence.criteria.CriteriaBuilder.all ( Subquery )

Create an all _expression_ over the subquery results.

true

true

)

If we manually create/update the api-assertions html file, we may need to also ensure that running TCK tests have access to the same assertion information.  Perhaps we need to also create a new https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/internal/docs/jpa/Persistence31JavadocAssertions.xml.  Seems like we need more investigation of how the internals work with assertion id handling for new TCK tests so we can identify exactly how to correctly add new assertions.

Scott

On 11/3/21 2:33 PM, Scott Marlow wrote:

I was just reading the https://www.eclipse.org/lists/jakartaee-tck-dev/msg00253.html thread again from 2019 which has some related details that I will try to summarize here:

- assertion docs are generated for spec, java doc API + the ids are in the docs with a brief description.
- compatibility rules dictate that once a SPEC is final, no more TCK tests can be added for it.
- assertion tooling is in https://github.com/eclipse-ee4j/jakartaee-tck-tools/tree/master/tools.
- I think that point https://www.eclipse.org/lists/jakartaee-tck-dev/msg00257.html + https://www.eclipse.org/lists/jakartaee-tck-dev/msg00265.html, WebSockets SPEC included the assertion ids.
- https://www.eclipse.org/lists/jakartaee-tck-dev/msg00261.html talks about how the tooling works and also that the tooling cannot cover all cases, so the assertion list is updated manually for those cases (also see https://www.eclipse.org/lists/jakartaee-tck-dev/msg00265.html).

I think from the last point, the Persistence AutoCloseable enhancement will not be handled by the assertion tooling, so its likely that manually updating the assertion doc for that should be allowed.  Lets leave this thread open for a few days before we make a dermination as to how to handle creating new TCK assertions for Jakarta EE 10.

Scott

On 11/2/21 3:11 PM, Scott Marlow wrote:

As a TCK user, I am interested in reading the identified Specification/Javadoc text that the test was written to validate.  I'm not especially interested in how the Jakarta EE development process deals with finalizing the specifics of mapping internal ID codes to the Specification/Javadoc sections, although I am interested in having the unique ID code perhaps for reporting

For example, Jakarta Persistence 3.1 is adding AutoCloseable enhancement to EntityManager/EntityManagerFactory interfaces which is tested by [1].  The relevant JavaDoc will be generated from [2][3] and published to [4].  But what process should the TCK test contributor follow to generate a new https://github.com/eclipse-ee4j/jakartaee-tck/blob/master/install/jpa/docs/assertions/api/api-assertions-3_1.html file for EE 10?  Should they (or someone) start creating a new api-assertions-3_1.html file via the instructions mentioned in [1]?  Or could they manually create the api-assertions-3_1.html (or spec-assertions-3.1.html) file?

I started a blank wiki page [5] for documenting the answer, based on your feedback.

Scott

[1] https://github.com/eclipse-ee4j/jakartaee-tck/pull/759
[2] https://github.com/eclipse-ee4j/jpa-api/blob/master/api/src/main/java/jakarta/persistence/EntityManager.java#L54
[3] https://github.com/eclipse-ee4j/jpa-api/blob/master/api/src/main/java/jakarta/persistence/EntityManagerFactory.java#L35
[4] https://jakarta.ee/specifications/persistence/3.1/
[5] https://github.com/eclipse-ee4j/jakartaee-tck/wiki/Guide-to-adding-Specification-Assertions