Hello, good people of Jetty.
I am looking for some guidance with this exception
java.net.SocketTimeoutException: Connect Timeout
at org.eclipse.jetty.io.ManagedSelector$Connect.run(ManagedSelector.java:955)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
We have a subclass of AsyncMiddleManServlet which uses Jetty's HttpClient to proxy requests. HttpClient is configured with a connect-timeout of 1000 ms. What is interesting is that we see socket timeout exception is thrown without utilizing the full 1000 ms allocated for connect timeout, for example, we see calls getting timed out under 25 ms.
Exception is thrwon from here (ManagedSelector:955)
public void run()
{
if (_selectorManager.isConnectionPending(channel))
{
if (LOG.isDebugEnabled())
LOG.debug("Channel {} timed out while connecting, closing it", channel);
failed(new SocketTimeoutException("Connect Timeout"));
}
}
One theory was that the proxied service was not responding fast enough, due to it being under-provisioned. In order to test that theory, we scaled the backend service 3x, and still, we are seeing these types of exceptions.
Looking at the Jetty code, "SocketTimeoutException: Connect Timeout" is thrown if the connection on the channel is pending. I don't really understand how the connect timeout is related here. If it is how would one configure it. Basically what I am asking for there is there a configuration parameter for addressing the pending connection issue which leads to SocketTimeoutException.
Jetty version: 9.4.42.v20210604
Envirnoment: Kubernetes
Thanks, any advice is much appreciated,
-Lu
