Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Failing to connect with v9.2.2 HttpClient

On 08/21/2014 01:52 PM, Thomas Hurd wrote:
On 08/15/2014 05:09 PM, Thomas Hurd wrote:
On 08/15/2014 04:00 PM, Simone Bordet wrote:

On Fri, Aug 15, 2014 at 9:46 PM, Thomas Hurd<thurd@xxxxxxxxxx>  wrote:
I am having issues using Jetty HttpClient after upgrading from v9.0.4 to

I using RHEL 5.10 on a Dell Server. uname -r outputs: 2.6.18-371.8.1.el5
The machine does not have support for IPv6.
I am using Oracle Java 7u60 JRE.

When using v9.2.2 Jetty JARs, I did not see any connection attempt made to
the destination.  Running tcpdump confirmed this.
After running strace on my test program[1], I found that the socket syscall is failing with 9.2.2 with PF_INET6 and not retrying with PF_INET. With
v9.0.4, it retries with PF_INET and is successful.

strace -f output using v9.0.4:
not supported by protocol)
#then later...
#Afterwards the connect syscall was successful, and request/response was

strace -f output using v9.2.2:
not supported by protocol)
# no attempt at socket(PF_INET, ...)

Additionally, I tried running the program with, but with the v9.2.2 JARs it still attempted
to create the socket with PF_INET6.
I am able to run a Jetty Servlet with v9.2.2 on the same machine without any
I also tried v9.1.5, and had the same results as with v9.2.2

Any help would be appreciated.
Code looks good (minor nit you could use the fluent style to configure
the request).
The Jetty code to connect has not changed between 9.0.4 and 9.2.2, it
has just been refactored into a different class, but it's exactly the
same code.

Frankly, it's really weird.
Are you sure you're not using a non OpenJDK/Oracle JDK in one of the two cases ?
Often old machines come with preinstalled java from who knows where,
and it's enough to type "java" rather than "./java" (or something like
that - different user different bash settings, etc.) to screw up

Thanks for the quick reply.

The Java 7u60 JDK is the only instance of Java I have installed on the system.

Both tests were created exactly the same except I pointed to v9.0.4 or v9.2.2 client/http/io/util JARs as referenced libraries before exporting to a runnable JAR in eclipse.

I ran the tests the same way:
strace -f -o jetty-test-9.0.4 java -jar jetty-test-9.0.4.jar
strace -f -o jetty-test-9.2.2 java -jar jetty-test-9.2.2.jar
I was able to successfully connect to the destination using 9.2.2 Jetty after changing the size of the thread pool, but I don't understand why that was causing issues. I am still seeing the failed IPv6 socket in strace output, and I am not sure if that was ever relevant :-)

When I setup my HttpClient as follows, I'm getting a silent failure:
HttpClient jettyHttpClient = new HttpClient();
final ExecutorService executorService = Executors.newFixedThreadPool(10);

I was able to debug down to:
HttpClient.newConnection calling resolver.resolve(address.getHost(), address.getPort(), new Promise<SocketAddress() on line 524 of resolver.resolve() on line 122 of is calling is calling executor.execute(new Runnable() java.util.concurrent.execute in Oracle 7u60 JDK source on line 1335 of

When I set the number of threads in the pool to 10, c is -536870902 ( & with CAPACITY = 10). It fails the check < corePoolSize (10). It gets offered on the workQueue. When it calls !isRunning(recheck) it returns nonzero and doesn't call remove(), reject(), or addWorker(). The executor.execute() then returns without creating the InetSocketAddress object. When I set the number of threads to 13, c is -536870900 ( & with CAPACITY = 12 ) and addWorker(command, true) is called. executor.execute() calls the run method and the InetSocketAddress object is created. Eventually HttpClient.newConnection() returned successfully and the rest of the exchange was successful.

I am curious as to why it worked in 9.0.4 with a fixedThreadPool of 10, but it fails for me in 9.2.2 (but works with 13). I looked at the diff between the two versions of jetty-util, and nothing stuck out to me.

In any case, I switched to a QueuedThreadPool(max=50, min=10), and didn't have any issues with 9.2.2.

To follow up again, I found the reason that 13 threads works and 10 doesn't. There ended up being a change after 9.0.4 on how many threads are created by default for Client Selector Managers.

jetty-client was updated to create Client Selector Threads based on the number of processors:
    public HttpClientTransportOverHTTP()
        this(Math.max(1, Runtime.getRuntime().availableProcessors() / 2));

On my machine, I have 24 total cores, so the # of selectors it wants to create is 24/2 = 12, which exceeds my thread pool (and 10 client selector threads are created). Curious as to why the SocketAddressResolver wasn't able to use one of these threads to create the InetSocketAddress, I found HttpClient.setDispatchIO().

The comment for setDispatchIO says:
dispatchIO true to dispatch I/O operations in a different thread, false to execute them in the selector thread.

With it set to true (by default), it makes sense that the task for creating the InetSocketAddress is never executed on the thread pool because they are all selector threads.

jetty-users mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit

jetty-users mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit

Back to the top