Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » Eclipse Communications Framework (ECF) » Selecting best implementation for Remote Service
Selecting best implementation for Remote Service [message #1779022] Thu, 28 December 2017 20:56 Go to next message
Alain Picard is currently offline Alain PicardFriend
Messages: 215
Registered: July 2009
Senior Member
We have been using r-OSGI for many years in one of our product, but for another product we are looking at the best implementation given some of our constraints.

We have a server with local RCP clients, which also serves web content and provides REST/JSON services. Due to security and firewalls we have to run all communications over HTTPS/SSL (standard port 43).

This has lead me to run some tests with the r-OSGI/ Web socket implementation and also looked at support for multiple providers (i.e. REST) which is great.

Questions are, on the discovery front we have a single server (at the moment) and don't think we can use discovery with Zeroconf or SLP given our network limitations. What is best solution, would still like to discover what the server is offering. Almost considering using EDEF right now, which I tested, but that doesn't seem to be optimal. From reading it seems that generic server has built in discovery through shared objects (from old slides "• Connected clients see all service proxies"). Also from discussions http://karaf.922171.n3.nabble.com/Proposal-Lightweight-standalone-remote-OSGi-implementation-for-karaf-cellar-td4045343.html
it does appear the the generic server might be faster. Would that then be the way to go as we expect quite a bit of "traffic". But it seems that the websocket implementation is for r-OSGI even if the original (https://github.com/videonext/ECF-WebSocket) seemed to have been an implementation for ecf-generic-server.

What is the suggested path forward.

Thanks for your help and insight.

Alain

Re: Selecting best implementation for Remote Service [message #1779041 is a reply to message #1779022] Sat, 30 December 2017 00:52 Go to previous messageGo to next message
Scott Lewis is currently offline Scott LewisFriend
Messages: 1023
Registered: July 2009
Senior Member
Hi Alain,

Alain Picard wrote on Thu, 28 December 2017 15:56
We have been using r-OSGI for many years in one of our product, but for another product we are looking at the best implementation given some of our constraints.
We have a server with local RCP clients, which also serves web content and provides REST/JSON services. Due to security and firewalls we have to run all communications over HTTPS/SSL (standard port 43).

This has lead me to run some tests with the r-OSGI/ Web socket implementation and also looked at support for multiple providers (i.e. REST) which is great.


Yes. I imagine you have been trying the websockets-based rosgi transport implementation...it's available here: https://github.com/ECF/Websockets. It's been about a year since I worked with it, but I know it's being used successfully in a production setting.

Quote:


Questions are, on the discovery front we have a single server (at the moment) and don't think we can use discovery with Zeroconf or SLP given our network limitations. What is best solution, would still like to discover what the server is offering.



I'm not an expert with zookeeper, but I think it can run in both 'lan/multicast' mode (which I imagine could be difficult for you) or in a server-client mode (which might be easier).

Another option for you might be to use etcd, which is Google's discovery system. Built on http/https and server (cluster)-based it might serve your needs. We have a discovery provider for it here: https://github.com/ECF/etcd-provider

Quote:


Almost considering using EDEF right now, which I tested, but that doesn't seem to be optimal.



Just to be clear...EDEF is the RSA-standardized xml file format...for representing remote service endpoint descriptions in a serialized form. It is possible to initiate service discovery by setting up EDEF files in bundles (with the appropriate meta-data info), and that can work fairly well if your services are pretty static...i.e. their endpoint description data doesn't change over time. Is that what you mean by 'using EDEF right now'...i.e. using file-based discovery?

Because another more/dynamic way to use EDEF is to setup a template EDEF file, read it in and modify the contents of the endpoint description programmatically (in java), and then using the RSA API directly to import the service (again in java). This allows quite a bit more flexibility in the use of EDEF.

Quote:


From reading it seems that generic server has built in discovery through shared objects (from old slides "• Connected clients see all service proxies").



This was true prior to the standardization of Remote Service Admin (RSA), but since then this aspect of the generic server has been removed...to comply with the specified behavior of RSA....which has a role called the topology manager that decides whether an endpoint description is created/a proxy constructed.

Quote:


Also from discussions http://karaf.922171.n3.nabble.com/Proposal-Lightweight-standalone-remote-OSGi-implementation-for-karaf-cellar-td4045343.html
it does appear the the generic server might be faster. Would that then be the way to go as we expect quite a bit of "traffic".



To be direct about it...I expect that parameters specific to your communication patterns...i.e. how frequently messages are sent and how much serialization is involved in marshalling rpc arguments and return values... will be more important than which of the tcp-based distribution providers you use out of a) generic; b) rosgi; c) something else tcp-based (e.g. JMS, Fabric). I would, however, expect that all of the http/https-based distribution providers (e.g. jaxrs/jersey) to be slower if lots of remote calls are made, as they may not maintain the connection between calls (if they do keep the connection open then they may perform just fine relative to generic, jms, rosgi, fabric). I'm not sure how jersey does connection handling between calls.

BTW, Johannes and I did eventually create a distribution provider using the Fabric APIs...it works fine and is pretty performant for his purposes, but he/we haven't made it generally available via a github repo. I believe we could do so, but we haven't done so. If that's desired I can/would contact Johannes directly again.

Quote:


But it seems that the websocket implementation is for r-OSGI even if the original (https://github.com/videonext/ECF-WebSocket) seemed to have been an implementation for ecf-generic-server.



Yes, the websocket repo at http://github.com/ECF is for r-OSGi. I wasn't aware of the ECF-WebSocket previously...and although it looks pretty close to complete I haven't tried/tested it yet. I would be willing to put some resources into creating websocket-based transport for ecf generic. It's not a huge job (as suggested by the ECF-WebSocket repo contents), but I've been occupied with other concerns recently.

Quote:


What is the suggested path forward.



To start, I would suggest first trying the rosgi remote services provider, along with the rosgi websockets transport : https://github.com/ECF/Websockets

rosgi has a few small idiocyncracies, however, that may or may not meet your needs (e.g. it assumes that every service consumer is also a service provider). I can help with these, as I did for the previous consumer of the rosgi websockets transport.

Another option would be to use/fork the videonext work to create a websockets transport for the ecf generic provider. This would take some work, but much of what videonext has done could probably be reused. I've wanted/still want to have a websockets transport for the generic provider, and I would be willing to help with construting, testing, debugging one...but realistically it would probably require some consulting arrangement to get such a thing to production grade quickly. Just to be clear, I would be open to such a consulting arrangement if you/your org is.

WRT discovery, I would suggest taking a look at the etcd-provider as described above. etcd is very heavily used (i.e. in Kubernetes), and so well-maintained, and I recently have started using etcd-provider for another client, so I know it works properly for discovering osgi remote services.

FWIW, it's also possible to create your own discovery provider that uses your own discovery mechanism (via the org.eclipse.ecf.discovery api). This would take some work, but if you already have a discovery mechanism that you would like to use to discover osgi remote services then it would allow you to do so with ECF remote services.

If you would like to contact me directly I can be reached at slewis at composent.com or scottslewis at gmail.com.

Quote:


Thanks for your help and insight.

Alain



No problem. Glad to hear that ECF remote services might be able to help.

Scott
Re: Selecting best implementation for Remote Service [message #1779042 is a reply to message #1779022] Sat, 30 December 2017 00:52 Go to previous messageGo to next message
Scott Lewis is currently offline Scott LewisFriend
Messages: 1023
Registered: July 2009
Senior Member
Hi Alain,

Alain Picard wrote on Thu, 28 December 2017 15:56
We have been using r-OSGI for many years in one of our product, but for another product we are looking at the best implementation given some of our constraints.
We have a server with local RCP clients, which also serves web content and provides REST/JSON services. Due to security and firewalls we have to run all communications over HTTPS/SSL (standard port 43).

This has lead me to run some tests with the r-OSGI/ Web socket implementation and also looked at support for multiple providers (i.e. REST) which is great.


Yes. I imagine you have been trying the websockets-based rosgi transport implementation...it's available here: https://github.com/ECF/Websockets. It's been about a year since I worked with it, but I know it's being used successfully in a production setting.

Quote:


Questions are, on the discovery front we have a single server (at the moment) and don't think we can use discovery with Zeroconf or SLP given our network limitations. What is best solution, would still like to discover what the server is offering.



I'm not an expert with zookeeper, but I think it can run in both 'lan/multicast' mode (which I imagine could be difficult for you) or in a server-client mode (which might be easier).

Another option for you might be to use etcd, which is Google's discovery system. Built on http/https and server (cluster)-based it might serve your needs. We have a discovery provider for it here: https://github.com/ECF/etcd-provider

Quote:


Almost considering using EDEF right now, which I tested, but that doesn't seem to be optimal.



Just to be clear...EDEF is the RSA-standardized xml file format...for representing remote service endpoint descriptions in a serialized form. It is possible to initiate service discovery by setting up EDEF files in bundles (with the appropriate meta-data info), and that can work fairly well if your services are pretty static...i.e. their endpoint description data doesn't change over time. Is that what you mean by 'using EDEF right now'...i.e. using file-based discovery?

Because another more/dynamic way to use EDEF is to setup a template EDEF file, read it in and modify the contents of the endpoint description programmatically (in java), and then using the RSA API directly to import the service (again in java). This allows quite a bit more flexibility in the use of EDEF.

Quote:


From reading it seems that generic server has built in discovery through shared objects (from old slides "• Connected clients see all service proxies").



This was true prior to the standardization of Remote Service Admin (RSA), but since then this aspect of the generic server has been removed...to comply with the specified behavior of RSA....which has a role called the topology manager that decides whether an endpoint description is created/a proxy constructed.

Quote:


Also from discussions http://karaf.922171.n3.nabble.com/Proposal-Lightweight-standalone-remote-OSGi-implementation-for-karaf-cellar-td4045343.html
it does appear the the generic server might be faster. Would that then be the way to go as we expect quite a bit of "traffic".



To be direct about it...I expect that parameters specific to your communication patterns...i.e. how frequently messages are sent and how much serialization is involved in marshalling rpc arguments and return values... will be more important than which of the tcp-based distribution providers you use out of a) generic; b) rosgi; c) something else tcp-based (e.g. JMS, Fabric). I would, however, expect that all of the http/https-based distribution providers (e.g. jaxrs/jersey) to be slower if lots of remote calls are made, as they may not maintain the connection between calls (if they do keep the connection open then they may perform just fine relative to generic, jms, rosgi, fabric). I'm not sure how jersey does connection handling between calls.

BTW, Johannes and I did eventually create a distribution provider using the Fabric APIs...it works fine and is pretty performant for his purposes, but he/we haven't made it generally available via a github repo. I believe we could do so, but we haven't done so. If that's desired I can/would contact Johannes directly again.

Quote:


But it seems that the websocket implementation is for r-OSGI even if the original (https://github.com/videonext/ECF-WebSocket) seemed to have been an implementation for ecf-generic-server.



Yes, the websocket repo at http://github.com/ECF is for r-OSGi. I wasn't aware of the ECF-WebSocket previously...and although it looks pretty close to complete I haven't tried/tested it yet. I would be willing to put some resources into creating websocket-based transport for ecf generic. It's not a huge job (as suggested by the ECF-WebSocket repo contents), but I've been occupied with other concerns recently.

Quote:


What is the suggested path forward.



To start, I would suggest first trying the rosgi remote services provider, along with the rosgi websockets transport : https://github.com/ECF/Websockets

rosgi has a few small idiocyncracies, however, that may or may not meet your needs (e.g. it assumes that every service consumer is also a service provider). I can help with these, as I did for the previous consumer of the rosgi websockets transport.

Another option would be to use/fork the videonext work to create a websockets transport for the ecf generic provider. This would take some work, but much of what videonext has done could probably be reused. I've wanted/still want to have a websockets transport for the generic provider, and I would be willing to help with construting, testing, debugging one...but realistically it would probably require some consulting arrangement to get such a thing to production grade quickly. Just to be clear, I would be open to such a consulting arrangement if you/your org is.

WRT discovery, I would suggest taking a look at the etcd-provider as described above. etcd is very heavily used (i.e. in Kubernetes), and so well-maintained, and I recently have started using etcd-provider for another client, so I know it works properly for discovering osgi remote services.

FWIW, it's also possible to create your own discovery provider that uses your own discovery mechanism (via the org.eclipse.ecf.discovery api). This would take some work, but if you already have a discovery mechanism that you would like to use to discover osgi remote services then it would allow you to do so with ECF remote services.

If you would like to contact me directly I can be reached at slewis at composent.com or scottslewis at gmail.com.

Quote:


Thanks for your help and insight.

Alain



No problem. Glad to hear that ECF remote services might be able to help.

Scott
Re: Selecting best implementation for Remote Service [message #1779050 is a reply to message #1779042] Sun, 31 December 2017 06:59 Go to previous messageGo to next message
Scott Lewis is currently offline Scott LewisFriend
Messages: 1023
Registered: July 2009
Senior Member
I spent some time upgrading the Websockets repo contents. I renamed the repo to ROSGI-Websockets-Provider: https://github.com/ECF/ROSGI-Websockets-Provider setup a mavent/tycho-based build and turned on the Travis CI build, and I upgrated the java_websockets implementation to 1.3.7 from 1.3. It needs a little testing, as the API for java_websockets had some breaking changes, but I will test it over the next several weeks.
Re: Selecting best implementation for Remote Service [message #1779056 is a reply to message #1779042] Sun, 31 December 2017 11:09 Go to previous messageGo to next message
Alain Picard is currently offline Alain PicardFriend
Messages: 215
Registered: July 2009
Senior Member
Scott,

Thanks for the very valuable info. I looked at etcd but I'm not sure that will work. But given our scenario of a single known server, I really like your idea of the template EDEF that is programmatically updated.

In regards to Websockets, I think I will need to adapt to go with Jetty as I need to mix in with other traffic on the same server. Maybe something I can release at large.

Cheers,
Alain
Re: Selecting best implementation for Remote Service [message #1779064 is a reply to message #1779056] Mon, 01 January 2018 06:22 Go to previous messageGo to next message
Scott Lewis is currently offline Scott LewisFriend
Messages: 1023
Registered: July 2009
Senior Member
Alain Picard wrote on Sun, 31 December 2017 06:09
Scott,

Thanks for the very valuable info. I looked at etcd but I'm not sure that will work. But given our scenario of a single known server, I really like your idea of the template EDEF that is programmatically updated.

In regards to Websockets, I think I will need to adapt to go with Jetty as I need to mix in with other traffic on the same server. Maybe something I can release at large.



The videonext impl was apparently based upon Jetty's websockets impl [1]. It looks to me like it could be updated pretty easily.

And I expect that a jetty-based websockets transport for rosgi (like [2] but using jetty rather than java-websockets) could be created pretty easily. If you start the project I will contribute to it, and if you contribute it to ECF github I will make it available.

Scott
Re: Selecting best implementation for Remote Service [message #1779104 is a reply to message #1779064] Tue, 02 January 2018 20:41 Go to previous message
Alain Picard is currently offline Alain PicardFriend
Messages: 215
Registered: July 2009
Senior Member
Scott,

Quick question for you. I'm working on the code to update the EDEF by using the EndpointDescriptionReader and EndpointDescriptionWriter and I'm noticing that there is code in EndpointDescription#setPropertiesOverrides that is used for example by ImportEndpoint but I don't understand why this override method doesn't invoke verifyECFProperties, which would guarantee that the various properties are both validated and updated by the override.

Thanks
Alain
Previous Topic:Selecting best implementation for Remote Service
Next Topic:ECF Bundles for service discovery in OSGI with Bnd tools
Goto Forum:
  


Current Time: Mon Jun 18 15:49:40 GMT 2018

Powered by FUDForum. Page generated in 0.02137 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top