Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] TLS ALPN ACME Lets Encrypt challange

Hi Simone,

I talked to the acmebot engineer at Lets Encrypt about the Pebble ACME test server and on what is happening. The issue seems to be that Jetty closes the connection after not finding a matching connection factory on ALPN-TLS testing, you can see that the SNI is being matched (Chose explicit alias=acme_challenge_key keyType=RSA on sun.security.ssl.SSLEngineImpl@3a6e27bf) :

2022-12-23T13:21:49,395 | INFO  | qtp33227220-120  | NegotiatingServerConnection      | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | ALPNServerConnection@3cda681c::DecryptedEndPoint@61b9107f{l=/0:0:0:0:0:0:0:1:8444,r=/0:0:0:0:0:0:0:1:59190,OPEN,fill=-,flush=-,to=597/30000} application selected protocol 'acme-tls/1', but no correspondent org.eclipse.jetty.server.ConnectionFactory has been configured

Resulting on the Pebble side:

2022/12/23 13:21:49 va: localhost/[::1]:8444 TLS error: EOF

The complete flow:

Jetty:

2022-12-23T13:21:30,560 | DEBUG | pipe-acme:renew  | ActionRenew                      | 195 - eu.abeel.platform.security.acme.command - 1.0.0 | Triggering certificate renewal on scheduler AcmeQuartzScheduler 2022-12-23T13:21:30,566 | DEBUG | AcmeQuartzScheduler_Worker-1 | ACMERenewJob                     | 194 - eu.abeel.platform.security.acme.api - 1.0.0 | Triggering renew certificate job for account https://localhost:14000/my-account/16516a068e4d195f on ACME server acme://pebble 2022-12-23T13:21:30,617 | INFO  | AcmeQuartzScheduler_Worker-1 | ACMERenewJob                     | 194 - eu.abeel.platform.security.acme.api - 1.0.0 | Authorization for pending domain localhost 2022-12-23T13:21:30,864 | DEBUG | AcmeQuartzScheduler_Worker-1 | ACMERenewJob                     | 194 - eu.abeel.platform.security.acme.api - 1.0.0 | Added challenge certificate to key store 2022-12-23T13:21:40,112 | DEBUG | Scanner-0        | KeyStoreScanner                  | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | changed C:\Users\betzm\Development\karaf\runtime-3.2.1\etc\keystore\digital.gaston-schul.com.jks 2022-12-23T13:21:40,113 | DEBUG | Scanner-0        | KeyStoreScanner                  | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | reloading keystore file C:\Users\betzm\Development\karaf\runtime-3.2.1\etc\keystore\digital.gaston-schul.com.jks 2022-12-23T13:21:40,119 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate alias=digital.gaston-schul.com SAN dns=digital.gaston-schul.com in X509@2cea9e94(digital.gaston-schul.com,h=[],a=[],w=[]) 2022-12-23T13:21:40,120 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate CN alias=digital.gaston-schul.com CN=digital.gaston-schul.com in X509@2cea9e94(digital.gaston-schul.com,h=[digital.gaston-schul.com],a=[],w=[]) 2022-12-23T13:21:40,121 | INFO  | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | x509=X509@2cea9e94(digital.gaston-schul.com,h=[digital.gaston-schul.com],a=[],w=[]) for Server@7ac968b[provider=null,keyStore=file:///C:/Users/betzm/Development/karaf/runtime-3.2.1/etc/keystore/digital.gaston-schul.com.jks,trustStore=null] 2022-12-23T13:21:40,122 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate alias=acme_challenge_key SAN dns=localhost in X509@670fc279(acme_challenge_key,h=[],a=[],w=[]) 2022-12-23T13:21:40,123 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate CN alias=acme_challenge_key CN=acme.invalid in X509@670fc279(acme_challenge_key,h=[localhost],a=[],w=[]) 2022-12-23T13:21:40,123 | INFO  | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | x509=X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]) for Server@7ac968b[provider=null,keyStore=file:///C:/Users/betzm/Development/karaf/runtime-3.2.1/etc/keystore/digital.gaston-schul.com.jks,trustStore=null] 2022-12-23T13:21:40,123 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate alias=acme_challenge_cert SAN dns=localhost in X509@a37adb8(acme_challenge_cert,h=[],a=[],w=[]) 2022-12-23T13:21:40,124 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate CN alias=acme_challenge_cert CN=acme.invalid in X509@a37adb8(acme_challenge_cert,h=[localhost],a=[],w=[]) 2022-12-23T13:21:40,124 | INFO  | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | x509=X509@a37adb8(acme_challenge_cert,h=[localhost, acme.invalid],a=[],w=[]) for Server@7ac968b[provider=null,keyStore=file:///C:/Users/betzm/Development/karaf/runtime-3.2.1/etc/keystore/digital.gaston-schul.com.jks,trustStore=null] 2022-12-23T13:21:40,125 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate alias=digital.gaston-schul.com (r3) SAN dns=digital.gaston-schul.com in X509@118c477a(digital.gaston-schul.com (r3),h=[],a=[],w=[]) 2022-12-23T13:21:40,125 | DEBUG | Scanner-0        | X509                             | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Certificate CN alias=digital.gaston-schul.com (r3) CN=digital.gaston-schul.com in X509@118c477a(digital.gaston-schul.com (r3),h=[digital.gaston-schul.com],a=[],w=[]) 2022-12-23T13:21:40,125 | INFO  | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | x509=X509@118c477a(digital.gaston-schul.com (r3),h=[digital.gaston-schul.com],a=[],w=[]) for Server@7ac968b[provider=null,keyStore=file:///C:/Users/betzm/Development/karaf/runtime-3.2.1/etc/keystore/digital.gaston-schul.com.jks,trustStore=null] 2022-12-23T13:21:40,126 | DEBUG | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | managers=[org.eclipse.jetty.util.ssl.SniX509ExtendedKeyManager@706f6ebf] for Server@7ac968b[provider=null,keyStore=file:///C:/Users/betzm/Development/karaf/runtime-3.2.1/etc/keystore/digital.gaston-schul.com.jks,trustStore=null] 2022-12-23T13:21:40,127 | DEBUG | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selected Protocols [TLSv1.2] of [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1, SSLv3, SSLv2Hello] 2022-12-23T13:21:40,128 | DEBUG | Scanner-0        | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selected Ciphers [TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_ 2022-12-23T13:21:41,879 | DEBUG | AcmeQuartzScheduler_Worker-1 | ACMERenewJob                     | 194 - eu.abeel.platform.security.acme.api - 1.0.0 | Triggering ACME server challenge 2022-12-23T13:21:45,508 | DEBUG | qtp33227220-156  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Customize sun.security.ssl.SSLEngineImpl@7716d8e7 2022-12-23T13:21:46,560 | DEBUG | qtp33227220-143  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | SNI matching for type=host_name (0), value=localhost 2022-12-23T13:21:46,560 | DEBUG | qtp33227220-143  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | SNI host name localhost 2022-12-23T13:21:46,563 | DEBUG | qtp33227220-143  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=null keyType=EC on sun.security.ssl.SSLEngineImpl@7716d8e7 2022-12-23T13:21:46,564 | DEBUG | qtp33227220-143  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=null keyType=EC on sun.security.ssl.SSLEngineImpl@7716d8e7 2022-12-23T13:21:46,564 | DEBUG | qtp33227220-143  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selecting alias: keyType=RSA, sni=localhost, sniRequired=false, certs=[X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]), X509@118c477a(digital.gaston-schul.com (r3),h=[digital.gaston-schul.com],a=[],w=[])] 2022-12-23T13:21:46,565 | DEBUG | qtp33227220-143  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selected alias=acme_challenge_key 2022-12-23T13:21:46,565 | DEBUG | qtp33227220-143  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Matched SNI localhost with alias acme_challenge_key, certificate X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]) from aliases [acme_challenge_key, digital.gaston-schul.com (r3)] 2022-12-23T13:21:46,565 | DEBUG | qtp33227220-143  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=acme_challenge_key keyType=RSA on sun.security.ssl.SSLEngineImpl@7716d8e7 2022-12-23T13:21:46,572 | INFO  | qtp33227220-143  | NegotiatingServerConnection      | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | ALPNServerConnection@5ee17e35::DecryptedEndPoint@2669d4ea{l=/0:0:0:0:0:0:0:1:8444,r=/0:0:0:0:0:0:0:1:59188,OPEN,fill=-,flush=-,to=1063/30000} application selected protocol 'acme-tls/1', but no correspondent org.eclipse.jetty.server.ConnectionFactory has been configured 2022-12-23T13:21:47,312 | DEBUG | qtp33227220-156  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Customize sun.security.ssl.SSLEngineImpl@3a6e27bf 2022-12-23T13:21:48,797 | DEBUG | qtp33227220-143  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Customize sun.security.ssl.SSLEngineImpl@711952fd 2022-12-23T13:21:48,799 | DEBUG | qtp33227220-119  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | SNI matching for type=host_name (0), value=localhost 2022-12-23T13:21:48,799 | DEBUG | qtp33227220-119  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | SNI host name localhost 2022-12-23T13:21:48,802 | DEBUG | qtp33227220-119  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=null keyType=EC on sun.security.ssl.SSLEngineImpl@3a6e27bf 2022-12-23T13:21:48,803 | DEBUG | qtp33227220-119  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=null keyType=EC on sun.security.ssl.SSLEngineImpl@3a6e27bf 2022-12-23T13:21:48,804 | DEBUG | qtp33227220-119  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selecting alias: keyType=RSA, sni=localhost, sniRequired=false, certs=[X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]), X509@118c477a(digital.gaston-schul.com (r3),h=[digital.gaston-schul.com],a=[],w=[])] 2022-12-23T13:21:48,804 | DEBUG | qtp33227220-119  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selected alias=acme_challenge_key 2022-12-23T13:21:48,805 | DEBUG | qtp33227220-119  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Matched SNI localhost with alias acme_challenge_key, certificate X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]) from aliases [acme_challenge_key, digital.gaston-schul.com (r3)] 2022-12-23T13:21:48,805 | DEBUG | qtp33227220-119  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=acme_challenge_key keyType=RSA on sun.security.ssl.SSLEngineImpl@3a6e27bf 2022-12-23T13:21:48,813 | INFO  | qtp33227220-119  | NegotiatingServerConnection      | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | ALPNServerConnection@7f901ab9::DecryptedEndPoint@4578cf47{l=/0:0:0:0:0:0:0:1:8444,r=/0:0:0:0:0:0:0:1:59189,OPEN,fill=-,flush=-,to=1500/30000} application selected protocol 'acme-tls/1', but no correspondent org.eclipse.jetty.server.ConnectionFactory has been configured 2022-12-23T13:21:49,384 | DEBUG | qtp33227220-120  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | SNI matching for type=host_name (0), value=localhost 2022-12-23T13:21:49,384 | DEBUG | qtp33227220-120  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | SNI host name localhost 2022-12-23T13:21:49,386 | DEBUG | qtp33227220-120  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=null keyType=EC on sun.security.ssl.SSLEngineImpl@711952fd 2022-12-23T13:21:49,386 | DEBUG | qtp33227220-120  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=null keyType=EC on sun.security.ssl.SSLEngineImpl@711952fd 2022-12-23T13:21:49,387 | DEBUG | qtp33227220-120  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selecting alias: keyType=RSA, sni=localhost, sniRequired=false, certs=[X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]), X509@118c477a(digital.gaston-schul.com (r3),h=[digital.gaston-schul.com],a=[],w=[])] 2022-12-23T13:21:49,388 | DEBUG | qtp33227220-120  | SslContextFactory                | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Selected alias=acme_challenge_key 2022-12-23T13:21:49,388 | DEBUG | qtp33227220-120  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Matched SNI localhost with alias acme_challenge_key, certificate X509@670fc279(acme_challenge_key,h=[localhost, acme.invalid],a=[],w=[]) from aliases [acme_challenge_key, digital.gaston-schul.com (r3)] 2022-12-23T13:21:49,388 | DEBUG | qtp33227220-120  | SniX509ExtendedKeyManager        | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | Chose explicit alias=acme_challenge_key keyType=RSA on sun.security.ssl.SSLEngineImpl@711952fd 2022-12-23T13:21:49,395 | INFO  | qtp33227220-120  | NegotiatingServerConnection      | 100 - org.eclipse.jetty.util - 9.4.43.v20210629 | ALPNServerConnection@3cda681c::DecryptedEndPoint@61b9107f{l=/0:0:0:0:0:0:0:1:8444,r=/0:0:0:0:0:0:0:1:59190,OPEN,fill=-,flush=-,to=597/30000} application selected protocol 'acme-tls/1', but no correspondent org.eclipse.jetty.server.ConnectionFactory has been configured 2022-12-23T13:21:51,817 | INFO  | AcmeQuartzScheduler_Worker-1 | JobRunShell                      | 202 - org.quartz-scheduler.quartz - 2.3.2 | Job acme.action.job.group.ACMERenewJob threw a JobExecutionException: org.quartz.JobExecutionException: Challenge has failed due to Failed to connect to [::1]:8444 for the tls-alpn-01 challenge     at eu.abeel.platform.security.acme.api.job.ACMERenewJob.executeJob(ACMERenewJob.java:128) ~[?:?]     at eu.abeel.platform.security.acme.api.spi.ACMEJob.execute(ACMEJob.java:42) ~[?:?]
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [!/:?]
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) [!/:?] 2022-12-23T13:21:51,818 | WARN  | AcmeQuartzScheduler_Worker-1 | ACMEActionJobListener            | 194 - eu.abeel.platform.security.acme.api - 1.0.0 | Job ACMERenewJob in group acme.action.job.group executed in 21253 ms 2022-12-23T13:21:51,819 | WARN  | AcmeQuartzScheduler_Worker-1 | ACMEActionJobListener            | 194 - eu.abeel.platform.security.acme.api - 1.0.0 | Challenge has failed due to Failed to connect to [::1]:8444 for the tls-alpn-01 challenge


Pebble:


Pebble 2022/12/23 13:19:37 There are now 1 accounts in memory
Pebble 2022/12/23 13:19:43 GET /dir -> calling handler()
Pebble 2022/12/23 13:19:43 GET /dir -> calling handler()
Pebble 2022/12/23 13:19:43 HEAD /nonce-plz -> calling handler()
Pebble 2022/12/23 13:19:43 POST /order-plz -> calling handler()
Pebble 2022/12/23 13:19:43 There are now 1 authorizations in the db
Pebble 2022/12/23 13:19:43 Added order "27J9nNbWCr5AVDhqQ7V25xb1DnOnv1nLLGz4aZZwmwc" to the db
Pebble 2022/12/23 13:19:43 There are now 1 orders in the db
Pebble 2022/12/23 13:19:43 POST /authZ/ -> calling handler()
Pebble 2022/12/23 13:19:55 POST /chalZ/ -> calling handler()
Pebble 2022/12/23 13:19:55 Pulled a task from the Tasks queue: &va.vaTask{Identifier:acme.Identifier{Type:"dns", Value:"localhost"}, Challenge:(*core.Challenge)(0xc0002701e0), Account:(*core.Account)(0xc0002480c0)}
Pebble 2022/12/23 13:19:55 Starting 3 validations.
Pebble 2022/12/23 13:19:55 Sleeping for 1s seconds before validating
Pebble 2022/12/23 13:19:55 Sleeping for 1s seconds before validating
Pebble 2022/12/23 13:19:55 Sleeping for 4s seconds before validating
Pebble 2022/12/23 13:19:59 POST /chalZ/ -> calling handler()
2022/12/23 13:19:59 va: localhost/[::1]:8444 TLS error: EOF
2022/12/23 13:19:59 va: localhost/[::1]:8444 TLS error: EOF
Pebble 2022/12/23 13:19:59 authz esruTFYFSBzwPSHWBlVw2id8z0SyJ5e93VfYsURbIGI set INVALID by completed challenge vkL5h9Jcaag1SXRsNe770GywQpaAE5bRON832AGt8d0 Pebble 2022/12/23 13:19:59 order 27J9nNbWCr5AVDhqQ7V25xb1DnOnv1nLLGz4aZZwmwc set INVALID by invalid authz esruTFYFSBzwPSHWBlVw2id8z0SyJ5e93VfYsURbIGI 2022/12/23 13:19:59 va: localhost/[::1]:8444 TLS error: read tcp [::1]:59172->[::1]:8444: wsarecv: An established connection was aborted by the software in your host machine.

On 23/12/2022 12:11, Simone Bordet wrote:
Hi,

On Thu, Dec 22, 2022 at 4:15 PM Info <info@xxxxxxxxxx> wrote:
2022-12-22T15:11:04,639 | INFO  | AcmeQuartzScheduler_Worker-1 |
JobRunShell                      | 190 - org.quartz-scheduler.quartz -
2.3.2 | Job acme.action.job.group.ACMERenewJob threw a
JobExecutionException:
org.quartz.JobExecutionException: Challenge has failed due to Failed to
connect to [::1]:8444 for the tls-alpn-01 challenge

The pebble Certificate and Key-Pair are stored in the key-store and
Jetty picks it up by hot reloading, then I trigger the challenge the SNI
gets matched and I close the connection because the handshake should be
all it needs transferring the challenge certificate.

But I get a failed challenge due to connection failure. Should I not
close the connection?
The log says you failed to connect.
Where do you connect to?
I ask because I don't think the server needs to connect anywhere.
Once the acme client has successfully downloaded the acme certificate,
it will contact the CA, download the renewed certificate in the place
you configured.
You repackage it into a keystore, overwrite the old keystore, and the
new keystore should be hot-reloaded by Jetty.



Back to the top