[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [jakarta.ee-spec.committee] Votes and Staging Repo Expirations
|
Thanks, David, that explanation was helpful.
It seems that we have a number of competing requirements. The most basic
requirement is that we ship exactly what we review and approve. There
doesn't seem to be an easy way to enforce this, so the question is how
much effort we need to go to to ensure that mistakes are not made.
Ideally, during the review period, there would be exactly one staging
repository for each API, which would not change, and after approval that
staging repository would be released to Maven Central. If that were the
case, it wouldn't matter whether we reviewed the "view" URL or the
"numbered" URL.
One known issue that makes the above difficult is the expiration of
staging repositories. 30 days is enough for any one API, but it's not
really long enough for the collection of all APIs going into Jakarta EE.
Again, ideally, this would be increased to 60 days. Lacking that, we might
need the "backup and restore" tools for the staging repositories.
I think we agreed that having things like GlassFish or the Jakarta EE API
project explicitly list every "numbered" staging repository is unworkable.
So it seems like we need to make the "view" URL work for us. This is the
primary reason I thought we should include the "view" URL in the PR - it
has to work, and this is a way for us to check that it's referring to the
right content.
The primary reason for including the "numbered" URL seems to be that it's
a fixed reference to the artifacts that can not change, and that if we
always operate on that numbered repository we can ensure that we get what
we want. But I don't see that there's any way to ensure that the "view"
URL will only find artifacts in that numbered repository, unless we assume
or ensure that there is only one such repository for each API.
Maybe we should create an audit tool that can check that there is only
one staging repository for each API, and that the artifacts visible
through the "view" URL are the same artifacts that are in that one "numbered"
staging repository?
So, again, how much effort is this worth?
I don't see any simple operational steps that will ensure we get exactly
what we want. We either need to make some assumptions or create some tools.
David Blevins wrote on 8/24/19 2:15 PM:
> The most important thing for us to know -- and I've tried and failed to explain this before -- is that this repository is basically like a database view:
>
> - https://oss.sonatype.org/content/repositories/staging/
>
> Nexus has the ability to sew together multiple physical repositories as one logical view. When those physical repositories are not located in that exact Nexus install, Nexus will act like a caching proxy. It's possible to create a view that basically consists of all actual staging repositories, plus a cache of external repositories like Maven Central.
>
> So for example, this is the actual physical staging repository for our deployment 1.7.2 binaries up for vote:
>
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.2/
>
> An actual staging repository (not a view) cannot be modified once closed. So there's nothing we can do to keep it fresh or trick it into staying around. Once the above `jakartaenterprisedeploy-1002` staging repository is deleted, we'll see it's contents are no longer available in the "view" here:
>
> - https://oss.sonatype.org/content/repositories/staging/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.2/
>
> Now as I mentioned the view also will allow you to "see" things in Maven Central. When we do releases, we enable `https://oss.sonatype.org/content/repositories/staging/` as a repository in our builds and our Maven Central dependencies end up flowing through the view and being cached. For example here's a chunk of the log output from the JAXR release James did 3 days ago:
>
> ----
> Downloaded from sonatype-nexus-staging: https://oss.sonatype.org/content/repositories/staging/com/headius/options/1.4/options-1.4.jar (14 kB at 6.7 kB/s)
> Downloading from sonatype-nexus-staging: https://oss.sonatype.org/content/repositories/staging/com/martiansoftware/nailgun-server/0.9.1/nailgun-server-0.9.1.jar
> Downloading from sonatype-nexus-staging: https://oss.sonatype.org/content/repositories/staging/com/jcraft/jzlib/1.1.3/jzlib-1.1.3.jar
> ----
>
> Each one of these is now sitting cached in Sonatype's staging view:
>
> - https://oss.sonatype.org/content/repositories/staging/com/headius/options/1.4/
> - https://oss.sonatype.org/content/repositories/staging/com/martiansoftware/nailgun-server/0.9.1/
> - https://oss.sonatype.org/content/repositories/staging/com/jcraft/jzlib/1.1.3/
>
> Where I tried to explain this was when I recommended we use the actual physical staging repo names in our PRs because these are the repos we must take action on when a release completes. For deployment there are actually several physical staged repos. I found them via URL hacking:
>
> ----
> 1000 404 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1000/
> 1001 200 Jul 25, 2019 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1001/
> 1002 200 Jul 25, 2019 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/
> 1003 200 Jul 27, 2019 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1003/
> 1004 200 Jul 27, 2019 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1004/
> 1005 200 Aug 02, 2019 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1005/
> 1006 404 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1006/
> 1007 404 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1007/
> 1008 404 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1008/
> 1009 404 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1009/
> 1010 404 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1010/
> ----
>
> The 1.7.2 binaries are in 1002 and 1.7.3 is in 1005.
>
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.2/
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1005/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.3/
>
> Both of these will be gone by the time their vote is schedule to complete. Any builds that pulled them in via the local staging view will begin to fail. There are some garbage repos we should just drop:
>
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1001/jakarta/enterprise/deploy/deployment-spec/1.7/
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1003/jakarta/enterprise/deploy/deployment-spec/1.0/
> - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1004/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.0.9/
>
> Here's Bill's mail release (again found via url hacking). Once the vote concludes on the 28th, he'll have about 3 days to publish the jakartamail-1011 repo which should be enough.
>
> ----
> 1010 404 https://oss.sonatype.org/content/repositories/jakartamail-1010/
> 1011 200 Aug 01, 2019 https://oss.sonatype.org/content/repositories/jakartamail-1011/
> 1012 404 https://oss.sonatype.org/content/repositories/jakartamail-1012/
> 1013 404 https://oss.sonatype.org/content/repositories/jakartamail-1013/
> ----
>
> Here's the transaction release. This vote completes Sept 2, so we're going to potentially lose this binary right at the finish line.
>
> ----
> 1021 404 https://oss.sonatype.org/content/repositories/jakartatransaction-1021/
> 1022 200 Aug 02, 2019 https://oss.sonatype.org/content/repositories/jakartatransaction-1022/
> 1023 404 https://oss.sonatype.org/content/repositories/jakartatransaction-1023/
> ----
>
> Here's the xmlrpc release. This vote also completes on Sept 2 and we risk losing this one at the finish line as well.
>
> ----
> 1003 404 https://oss.sonatype.org/content/repositories/jakartaxmlrpc-1003/
> 1004 200 Aug 02, 2019 https://oss.sonatype.org/content/repositories/jakartaxmlrpc-1004/
> 1005 404 https://oss.sonatype.org/content/repositories/jakartaxmlrpc-1005/
> ----
>
> Here are the JSP and JSTL repos, which are 1017 and 1019 respectively. The JSP vote concludes Sept 2, so we might possibly make this one if we're extremely fast. JSTL concludes Sept 9, so we have plenty of time to act before that one would be gone.
>
> ----
> 1016 404 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1016/
> 1017 200 Aug 03, 2019 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1017/
> 1018 404 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1018/
> 1019 200 Aug 19, 2019 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1019/
> 1020 404 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1020/
> ----
>
> Ultimately, this is why I was very against people using `https://oss.sonatype.org/content/repositories/staging/` urls in their PRs. Because it's too many levels removed from reality, perpetuates confusion and creates too many opportunities for the wrong thing to happen.
>
> We haven't seen it this release, but there could actually be several physical staging repos containing the same version of an API. Basically every attempt sticks around till it is either dropped or expires. Were this to happen we would still see only one copy in the logical view. The gotcha would be that as each expires and is deleted the one that wins will change. So the logical URL of `https://oss.sonatype.org/content/repositories/staging/jakarta/wombat/2.0/`, which is effectively a symbolic link, might map to `jakartawombat-1003/jakarta/wombat/2.0/` at the start of a vote and `jakartawombat-1006/jakarta/wombat/2.0/` at the end of a vote. The people who voted at the beginning and the people who voted at the end would have voted for completely different binaries.
>
> I appreciate this is a lot of information to digest.
>
>
