Re: [jakarta.ee-spec.committee] Votes and Staging Repo Expirations

["Kevin Sutter" <sutter@xxxxxxxxxx>]

Thanks, David.  That
explanation does help.  I appreciate the effort to even create that
email...  :-)

I tried to re-build
enterprise-deployment (as Scott did as well) with no luck.  Very strange
error that it can't find the "api" project to build...  Maybe
Dmitry can spot something to help with re-staging deployment...

+ cd /home/jenkins/workspace/enterprise-deployment-spec-api-release
+ mvn -U -C -pl api -DnewVersion=1.7.2 -DgenerateBackupPoms=false clean
org.codehaus.mojo:versions-maven-plugin:2.7:set
[INFO] Scanning for projects...
[ERROR] [ERROR] Could not find the selected project in the reactor: api
@ 
[ERROR] Could not find the selected project in the reactor: api -> [Help
1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the
-e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MavenExecutionException


---------------------------------------------------
Kevin Sutter 
STSM, MicroProfile and Jakarta EE architect
e-mail:  sutter@xxxxxxxxxx     Twitter:  @kwsutter
phone: tl-553-3620 (office), 507-253-3620 (office)    
LinkedIn: https://www.linkedin.com/in/kevinwsutter



From:
       David
Blevins <dblevins@xxxxxxxxxxxxx>
To:
       Jakarta
specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>
Date:
       08/24/2019
04:15 PM
Subject:
       [EXTERNAL]
Re: [jakarta.ee-spec.committee] Votes and Staging Repo Expirations
Sent
by:        jakarta.ee-spec.committee-bounces@xxxxxxxxxxx

---

The most important thing for us to
know -- and I've tried and failed to explain this before -- is that this
repository is basically like a database view:

 - https://oss.sonatype.org/content/repositories/staging/

Nexus has the ability to sew together multiple physical repositories as
one logical view.  When those physical repositories are not located
in that exact Nexus install, Nexus will act like a caching proxy.  It's
possible to create a view that basically consists of all actual staging
repositories, plus a cache of external repositories like Maven Central.

So for example, this is the actual physical staging repository for our
deployment 1.7.2 binaries up for vote:

 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/
 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.2/

An actual staging repository (not a view) cannot be modified once closed.
 So there's nothing we can do to keep it fresh or trick it into staying
around.  Once the above `jakartaenterprisedeploy-1002` staging repository
is deleted, we'll see it's contents are no longer available in the "view"
here:

 - https://oss.sonatype.org/content/repositories/staging/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.2/

Now as I mentioned the view also will allow you to "see" things
in Maven Central.  When we do releases, we enable `https://oss.sonatype.org/content/repositories/staging/` as a repository in our builds and our Maven Central dependencies end
up flowing through the view and being cached.  For example here's
a chunk of the log output from the JAXR release James did 3 days ago:

----
Downloaded from sonatype-nexus-staging: https://oss.sonatype.org/content/repositories/staging/com/headius/options/1.4/options-1.4.jar (14 kB at 6.7 kB/s)
Downloading from sonatype-nexus-staging: https://oss.sonatype.org/content/repositories/staging/com/martiansoftware/nailgun-server/0.9.1/nailgun-server-0.9.1.jar
Downloading from sonatype-nexus-staging: https://oss.sonatype.org/content/repositories/staging/com/jcraft/jzlib/1.1.3/jzlib-1.1.3.jar
----

Each one of these is now sitting cached in Sonatype's staging view:

 - https://oss.sonatype.org/content/repositories/staging/com/headius/options/1.4/
 - https://oss.sonatype.org/content/repositories/staging/com/martiansoftware/nailgun-server/0.9.1/
 - https://oss.sonatype.org/content/repositories/staging/com/jcraft/jzlib/1.1.3/

Where I tried to explain this was when I recommended we use the actual
physical staging repo names in our PRs because these are the repos we must
take action on when a release completes.  For deployment there are
actually several physical staged repos.  I found them via URL hacking:

----
1000                
404                
                 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1000/
1001                
200                
Jul 25, 2019                
https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1001/
1002                
200                
Jul 25, 2019                
https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/
1003                
200                
Jul 27, 2019                
https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1003/
1004                
200                
Jul 27, 2019                
https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1004/
1005                
200                
Aug 02, 2019                
https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1005/
1006                
404                
                 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1006/
1007                
404                
                 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1007/
1008                
404                
                 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1008/
1009                
404                
                 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1009/
1010                
404                
                 https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1010/
----

The 1.7.2 binaries are in 1002 and 1.7.3 is in 1005.

 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1002/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.2/
 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1005/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.3/

Both of these will be gone by the time their vote is schedule to complete.
 Any builds that pulled them in via the local staging view will begin
to fail.  There are some garbage repos we should just drop:

 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1001/jakarta/enterprise/deploy/deployment-spec/1.7/
 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1003/jakarta/enterprise/deploy/deployment-spec/1.0/
 - https://oss.sonatype.org/content/repositories/jakartaenterprisedeploy-1004/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.0.9/

Here's Bill's mail release (again found via url hacking).  Once the
vote concludes on the 28th, he'll have about 3 days to publish the jakartamail-1011
repo which should be enough.

----
1010                
404                
                 https://oss.sonatype.org/content/repositories/jakartamail-1010/
1011                
200                
Aug 01, 2019                
https://oss.sonatype.org/content/repositories/jakartamail-1011/
1012                
404                
                 https://oss.sonatype.org/content/repositories/jakartamail-1012/
1013                
404                
                 https://oss.sonatype.org/content/repositories/jakartamail-1013/
----

Here's the transaction release.  This vote completes Sept 2, so we're
going to potentially lose this binary right at the finish line.

----
1021                
404                
                 https://oss.sonatype.org/content/repositories/jakartatransaction-1021/
1022                
200                
Aug 02, 2019                
https://oss.sonatype.org/content/repositories/jakartatransaction-1022/
1023                
404                
                 https://oss.sonatype.org/content/repositories/jakartatransaction-1023/
----

Here's the xmlrpc release.  This vote also completes on Sept 2 and
we risk losing this one at the finish line as well.

----
1003                
404                
                 https://oss.sonatype.org/content/repositories/jakartaxmlrpc-1003/
1004                
200                
Aug 02, 2019                
https://oss.sonatype.org/content/repositories/jakartaxmlrpc-1004/
1005                
404                
                 https://oss.sonatype.org/content/repositories/jakartaxmlrpc-1005/
----

Here are the JSP and JSTL repos, which are 1017 and 1019 respectively.
 The JSP vote concludes Sept 2, so we might possibly make this one
if we're extremely fast.  JSTL concludes Sept 9, so we have plenty
of time to act before that one would be gone.

----
1016                
404                
                 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1016/
1017                
200                
Aug 03, 2019                
https://oss.sonatype.org/content/repositories/jakartaservletjsp-1017/
1018                
404                
                 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1018/
1019                
200                
Aug 19, 2019                
https://oss.sonatype.org/content/repositories/jakartaservletjsp-1019/
1020                
404                
                 https://oss.sonatype.org/content/repositories/jakartaservletjsp-1020/
----

Ultimately, this is why I was very against people using `https://oss.sonatype.org/content/repositories/staging/` urls in their PRs.  Because it's too many levels removed from reality,
perpetuates confusion and creates too many opportunities for the wrong
thing to happen.

We haven't seen it this release, but there could actually be several physical
staging repos containing the same version of an API.  Basically every
attempt sticks around till it is either dropped or expires.  Were
this to happen we would still see only one copy in the logical view.  The
gotcha would be that as each expires and is deleted the one that wins will
change.  So the logical URL of `https://oss.sonatype.org/content/repositories/staging/jakarta/wombat/2.0/`, which is effectively a symbolic link, might map to `jakartawombat-1003/jakarta/wombat/2.0/`
at the start of a vote and `jakartawombat-1006/jakarta/wombat/2.0/` at
the end of a vote.  The people who voted at the beginning and the
people who voted at the end would have voted for completely different binaries.

I appreciate this is a lot of information to digest.


-- 
David Blevins
http://twitter.com/dblevins
http://www.tomitribe.com
310-633-3852

> On Aug 24, 2019, at 11:49 AM, Kevin Sutter <sutter@xxxxxxxxxx>
wrote:
> 
> Hi,
> I'm not convinced that these Stagings will be expiring...  Specifically,
I looked at Deployment.  Yes, the directory (https://oss.sonatype.org/content/repositories/staging/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/) itself has a timestamp of July 23, but the contents have newer timestamps
(https://oss.sonatype.org/content/repositories/staging/jakarta/enterprise/deploy/jakarta.enterprise.deploy-api/1.7.3/).
> 
> And, we have other contents of the Staging repository that are much
older than 30 days...  For example,
> https://oss.sonatype.org/content/repositories/staging/jakarta/activation/jakarta.activation-api/1.2.1/
> https://oss.sonatype.org/content/repositories/staging/jakarta/jws/jakarta.jws-api/1.1.1/
> 
> So, maybe the 30 day expiration gets reset everytime the Staging Repo
gets touched?
> 
> ---------------------------------------------------
> Kevin Sutter 
> STSM, MicroProfile and Jakarta EE architect
> e-mail:  sutter@xxxxxxxxxx     Twitter:  @kwsutter
> phone: tl-553-3620 (office), 507-253-3620 (office)    
> LinkedIn: https://www.linkedin.com/in/kevinwsutter
> 
> 
> 
> From:        David Blevins <dblevins@xxxxxxxxxxxxx>
> To:        Bill Shannon <bill.shannon@xxxxxxxxxx>
> Cc:        Jakarta specification committee <jakarta.ee-spec.committee@xxxxxxxxxxx>
> Date:        08/23/2019 05:19 PM
> Subject:        [EXTERNAL] Re: [jakarta.ee-spec.committee]
Votes and Staging Repo Expirations
> Sent by:        jakarta.ee-spec.committee-bounces@xxxxxxxxxxx
> 
> 
> 
> Thanks, Bill.  I just asked that in the other thread.  No
need to follow up.  Looks like if I get EJB up we have a small window
to push a Jakarta EE API jar before deployment expires.
> 
> I'll be back shortly....
> 
> 
> -- 
> David Blevins
> http://twitter.com/dblevins
> http://www.tomitribe.com
> 310-633-3852
> 
> > On Aug 23, 2019, at 3:07 PM, Bill Shannon <bill.shannon@xxxxxxxxxx>
wrote:
> > 
> > jws is not part of Jakarta EE 8.
> > 
> > The 2.1.0 version of jws-api was published Aug 22.
> > 
> > David Blevins wrote on 8/23/19 2:54 PM:
> >> I've just an exchange with Brian Fox whose CTO of Sonatype.
 He confirms the ejb staged binaries were deleted due to expiration
and they've had to start cracking down on the 30-day period due to starved
resources.  I did not ask about the messy state of the staging repo
overall.
> >> 
> >> What this means for us is that we need new builds of:
> >> 
> >> - jakarta.ejb-api
> >> - jakarta.jws-api
> >> - jakarta.enterprise.deploy-api
> >> 
> >> The above are either gone already or will be gone by Monday.
 If we do not get these up, there is no hope of a Jakarta EE 8 vote
on Monday.
> >> 
> >> I'll do EJB.  Kevin if you can redo jakarta.enterprise.deploy-api,
that'd be awesome.
> >> 
> >> That leaves us with jakarta.jws-api.  How will we get
that one up?
> >> 
> >> 
> >> 
> 
> _______________________________________________
> jakarta.ee-spec.committee mailing list
> jakarta.ee-spec.committee@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
> https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee
> 
> 
> 
> 
> _______________________________________________
> jakarta.ee-spec.committee mailing list
> jakarta.ee-spec.committee@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
> https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee

_______________________________________________
jakarta.ee-spec.committee mailing list
jakarta.ee-spec.committee@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://www.eclipse.org/mailman/listinfo/jakarta.ee-spec.committee