Eclipse Community Forums: Eclipse SmartHome » Multiple HttpClients used in Eclipse SmartHome and bindings

Help

Home

Home » Archived » Eclipse SmartHome » Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off?

Show: Today's Messages :: Show Polls :: Message Navigator

Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1704184]

Sat, 08 August 2015 09:30

Jochen Hiller

Messages: 11
Registered: July 2009

Junior Member

Hi all,

I am just working on getting some bindings running on a non-Equinox OSGi runtime environment.
I struggled over that the Eclipse SmartHome framework and some bindings (from ESH, from openHAB2, ignoring openHAB1 for the moment) are using different HttpClients. To minimize the footprint over restricted devices (like Raspberry Pi 1) it is a bad idea to use multiple clients at same time, will increase memory footprint and startup time.

I made a rough analysis of the code base:

# bundles using Apache HttpClient 3.x
$ find . -name "MANIFEST.MF" -exec grep -l "org.apache.commons.httpclient" {} \;
./openhab2-master/git/openhab2/addons/binding/org.openhab.binding.autelis/META-INF/MANIFEST.MF
./openhab2-master/git/openhab2/addons/binding/org.openhab.binding.squeezebox/META-INF/MANIFEST.MF
./openhab2-master/git/openhab2/bundles/core/org.openhab.core/META-INF/MANIFEST.MF
./openhab2-master/git/openhab2/bundles/core/org.openhab.core.compat1x/META-INF/MANIFEST.MF
./smarthome-master/git/smarthome/bundles/io/org.eclipse.smarthome.io.net/META-INF/MANIFEST.MF
./smarthome-master/git/smarthome/bundles/ui/org.eclipse.smarthome.ui/META-INF/MANIFEST.MF

# bundles using Apache HttpClient 4.x
$ find . -name "MANIFEST.MF" -exec grep -l "org.apache.http." {} \;
./openhab2-master/git/openhab2/addons/binding/org.openhab.binding.ipp/META-INF/MANIFEST.MF

# bundles using Jetty Client
$ find . -name "MANIFEST.MF" -exec grep -l "org.eclipse.jetty.client" {} \;
./openhab2-master/git/openhab2/addons/binding/org.openhab.binding.avmfritz/META-INF/MANIFEST.MF

I also checked the different footprint of the used http clients:
* Apache HttpClient 3.x: org.apache.commons.httpclient, org.apache.commons.codec (about 600 kB)
* Apache HttpClient 4.x: org.apache.httpcomponents.httpclient, org.apache.httpcomponents.httpcore, org.apache.commons.codec (about 1,5 MB)
* Jetty Client 9.2.x: Jetty Client, Http, IO, Util (about 900 kB)

As HttpClient 3.x is considered as legacy this should be removed long term. So decision would be between Apache HttpClient 4 and Jetty Client. Considering the footprint Jetty would be first choice, but the client libraries might be not so popular like Apache HttpClient.
In general such a change would break the API in one case (HttpUtil.createHttpMethod), but this will not be used until now in ESH, openHAB2 (only 1x used in openHAB1 NEST binding), so no real issue.

What do you think? Is this the effort worth to try to get rid of not needed HttpClients?
Same time new bindings should be asked/enforced to stick on same http client.

Bye, Jochen

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1704185 is a reply to message #1704184]

Sat, 08 August 2015 09:51

John Cocula

Messages: 21
Registered: June 2015

Junior Member

Makes perfect sense to trim the bloat, as long as whatever HTTP client library is chosen allows for manually catching and redirecting 307 codes as the Nest API requires. I would have used just org.openhab.io.net.http.HttpUtil.executeUrl had it not been for this requirement.

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1704209 is a reply to message #1704185]

Sat, 08 August 2015 21:03

Kai Kreuzer

Messages: 673
Registered: December 2011

Senior Member

Hi Jochen,

I think we all agree that we should get rid off Apache HttpClient 3 - thanks for highlighting this.
Regarding the question of the best replacement: So far we had documented that Apache HttpClient 4 should be used: https://www.eclipse.org/smarthome/documentation/development/bindings/dependencies.html
But there were reasons why some bindings used the Jetty library instead. I think I just recalled them: What you have missed is that the Apache HttpClient 4 does not support asynchronous processing of requests - you will need an additional component called asyncclient (https://hc.apache.org/httpcomponents-asyncclient-4.1.x/index.html), which ends up at 2MB with all its dependencies. Furthermore, the version 4.1 somehow suggests that it is not very actively maintained (not in sync with the HttpClient 4 at least - the issue tracker shows last activity in February this year: https://issues.apache.org/jira/browse/HTTPASYNC/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).

The Jetty client seems to be much more active and modern - it e.g. already comes with HTTP/2 support as well and many other features, see https://webtide.com/http2-support-for-httpclient/. I personally have some positive experiences with using this library (asynchronously) and thus would think that it probably would be a good choice. But I would love to hear some other opinions and experiences as well!

Cheers,
Kai

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1705511 is a reply to message #1704209]

Mon, 17 August 2015 09:53

Karel Goderis

Messages: 198
Registered: March 2014

Senior Member

There are a few bindings-to-be-approved (OH2) that also use an HTTP client in the context of JSON-RPC and REST clients. But they can easily (I hope) be switched over.

K

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1705743 is a reply to message #1704185]

Tue, 18 August 2015 17:51

Jochen Hiller

Messages: 11
Registered: July 2009

Junior Member

John Cocula wrote on Sat, 08 August 2015 05:51

Makes perfect sense to trim the bloat, as long as whatever HTTP client library is chosen allows for manually catching and redirecting 307 codes as the Nest API requires. I would have used just org.openhab.io.net.http.HttpUtil.executeUrl had it not been for this requirement.

Thanks John for the feedback.

I checked the code you mentioned at https://github.com/openhab/openhab/blob/master/bundles/binding/org.openhab.binding.nest/src/main/java/org/openhab/binding/nest/internal/messages/AbstractRequest.java#L139

Could this be a default implementation in org.openhab.io.net.http.HttpUtil.executeUrl for a HTTP 307 return?

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1705744 is a reply to message #1704209]

Tue, 18 August 2015 17:56

Jochen Hiller

Messages: 11
Registered: July 2009

Junior Member

Hi Kai,

Kai Kreuzer wrote on Sat, 08 August 2015 17:03

What you have missed is that the Apache HttpClient 4 does not support asynchronous processing of requests - you will need an additional component called asyncclient (https://hc.apache.org/httpcomponents-asyncclient-4.1.x/index.html), which ends up at 2MB with all its dependencies.
Furthermore, the version 4.1 somehow suggests that it is not very actively maintained (not in sync with the HttpClient 4 at least - the issue tracker shows last activity in February this year: https://issues.apache.org/jira/browse/HTTPASYNC/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).

These are valid arguments against Apache HttpClient 4.x, I agree. Thanks for pointing to the async stuff missing in my overview.

Kai Kreuzer wrote on Sat, 08 August 2015 17:03

The Jetty client seems to be much more active and modern - it e.g. already comes with HTTP/2 support as well and many other features, see https://webtide.com/http2-support-for-httpclient/. I personally have some positive experiences with using this library (asynchronously) and thus would think that it probably would be a good choice. But I would love to hear some other opinions and experiences as well!

I agree with you, we should give it a try. I will work on a PR to change org.eclipse.smarthome.io.net to use Jetty.

We have then to revise the recommendations at
https://www.eclipse.org/smarthome/documentation/development/bindings/dependencies.html to move over to Jetty.

Bye, Jochen

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1705746 is a reply to message #1705511]

Tue, 18 August 2015 18:01

Jochen Hiller

Messages: 11
Registered: July 2009

Junior Member

Hi Karel,
karel goderis wrote on Mon, 17 August 2015 05:53

There are a few bindings-to-be-approved (OH2) that also use an HTTP client in the context of JSON-RPC and REST clients. But they can easily (I hope) be switched over.
K

good point. I only checked the bindings actually in ESH and openHAB2 master, not any forks/pending PRs.

Maybe we can help with a good sample binding (like avmfritz) how to use Jetty client.

Bye, Jochen

Report message to a moderator

Re: Multiple HttpClients used in Eclipse SmartHome and bindings - get rid off? [message #1705755 is a reply to message #1705746]

Tue, 18 August 2015 20:21

Karel Goderis

Messages: 198
Registered: March 2014

Senior Member

The Tesla binding uses javax.ws.rs.client (via WebTarget), which is I guess the Jersey client implementation in the end
The Helios binding uses WS-Notification implementation from the Apache CXF library (that is pure SOAP, not clear what http client that is, and no clue how to switch that over)
The HDAnywhere binding relies on org.eclipse.smarthome.io.net.http.HttpUtil
The Miele binding is based on JSON-RPC, implemented via java.net.HttpURLConnection
The ATSAdvanced binding (will not be published due to SDK license constraints) is based on the Apache CXF as well

Report message to a moderator