Re: [jakarta.ee-community] Help wanted: improved signature test tool

[Andy Guibert <andy.guibert@xxxxxxxxx>]

Hi Bill,

When I was porting JSON-B TCK tests out of the JT harness I came across this topic. IIRC SigTest is part of the TCK because in the old days of JavaEE we didn't actually ship an API jar that implementations could include -- all of the implementations had to write up the API classes/interfaces themselves.

Nowadays all of the implementations just pull in the API jars from Maven Central and expose them in the runtime, so SigTest doesn't really provide any testing value for implementations.

However, SigTest can be useful for detecting API changes as specs evolve, particularly to make sure we follow semantic versioning rules in the API. For this, I suggest we use a maven plugin called "bnd-baseline-maven-plugin" [1]. This plugin essentially scans the produced API jar at build time and compares it to the previous version to ensure that semantic versioning rules are followed.

[1] https://github.com/bndtools/bnd/tree/master/maven/bnd-baseline-maven-plugin

--

Andy

On Thu, Feb 13, 2020 at 7:38 PM Bill Shannon <bill.shannon@xxxxxxxxxx> wrote:

-- Background

Part of our TCK test suites is a signature test that verifies that the API
signatures of an implementation match what's expected from the specification.
The signature test is based on a file that records the API signatures.  The
sigtest tool [1] (part of the OpenJDK project) is used to generate the signature
file base on a "reference implementation" - an implementation that is believed
to exhibit the correct signatures.

The recorded signatures include every field and method in every class and
interface in the API, as well as the information for every superclass or
interface.  A consequence of this is that it records the signatures of
every JDK class used by the API.

That means that a different signature file is needed to test an API on
JDK 12 vs JDK 11 (for example), even though the API works identically on
both.  There's no way to say "this API requires JDK 11 or newer".

There are some features in the sigtest tool that seem like they should
work to limit the recorded signatures to only the API being tested, or
to exclude the signatures for JDK classes, but these features have never
worked the way I wanted them to work.  I've tried for years to convince
the sigtest team of what I wanted, but their perspective is that of testing
the JDK itself, which obviously never has this problem.


-- Help Wanted

I'd love it if someone in the Jakarta EE community could work with the
sigtest team, explain to them what's needed, and help them make the existing
features work as needed, or create new features to do what we need.

Worst case, someone could fork the sigtest tool and add the feature that
we need.  The obvious disadvantage of this approach is that the tool
would need to be kept up to date with every new JDK release.  We need
a long term solution, not a one-off solution.

If you're interested, you should start by familiarizing yourself with the
existing sigtest tool, try it out with a few APIs, then contact me for more
background.

Thanks for any help!


P.S.  I've filed a bug [2] against the platform TCK that can be used to track
any work in this area.


[1] https://wiki.openjdk.java.net/display/CodeTools/sigtest
[2] https://github.com/eclipse-ee4j/jakartaee-tck/issues/156
_______________________________________________
jakarta.ee-community mailing list
jakarta.ee-community@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jakarta.ee-community