|Re: [ecf-dev] Problem with Service Endpoint matching when using different network names|
On 7/18/2018 4:50 AM, Peter Hermsdorf wrote:
Hi, sorry for being late with my answer. See my comments inline.I see. So external clients are using VPN addressing and internal clients LAN addresses. Correct? And you would like a single server(s) (with 1...or multiple nics?), to host remote service(s) to both types of clients....internal and external. Something like this:C1-->addr:foo1.bar.com:1234-->VPN-->IP1 addr of server1-->Server1-->Serviceinstance| C2-> IP1 (or IP2?) addr of server1---| Seem right?CorrectFor the moment, forget about the dns names. What do the IP addresses/port look like for your C1 and C2? e.g. what does C1, C2 server address look like, what are IP1 and IP2, does the Server1 have multiple nics (IP1/IP2) or just one?In most installations the server has one IP address. Sometimes we can use this IP address when accessing the server trough VPN, sometimes there is a mapping somewhere in between...
When the mapping is done, who/what does the mapping?..and how is it done? It seems to me that's the problematic case. Does it map both IP address and port, or just IP address? Do you know if it's using NAT or some other tech?
Are there other services on this net...e.g. a web server...that are working properly with the addressing properties that you are looking for? If so, how is that done? A reverse proxy or load balancing hw or ?
from my point of view a unique ID for a remote service could well be ecftcp://server .I think what we need is a completely hostname (and port) agnostic way to match the service endpoints between the client and the server.In other words: one thing is to define the network target (by hostname or IP and port) and the other thing would be to match the remote service once the client and server are connected by network (via the service name and version etc.).Modifying a single generic provider to do this could be difficult. The main reason for this is that the generic provider has the notion of a group of connected processes... .the server's unique group name must consist of *some* address that is understood as unique among all group members connected within a particularly container instance ...e.g. ecftcp://foo1.bar.com:3282/server is the 'name/ID' of the server in the group connected that container instance (and it's unique ID).
Unfortunately it can't be that, because if more than one server used that same identifier and a single client was trying to get to two different service instances there would be no way to tell them apart.
But the challenge here is probably to split the "do the network connection" code from "match the correct service" code, since as far as I can tell the endpoint ID (ecftcp://foo1.bar.com:3282/server) is used for both parts.
To be clear, even with the generic provider there is a separation between the id identifying the java process that contains remote services (ecftcp://foo1.bar.com:3282/server) and the remote service instance: (i.e. 1, 2, 3...(long). In the edef the container-relative service instance is the property with name: ecf.rsvc.id.
But the real problem is that since the generic provider has the notion of a connected group of processes (containers)...i.e. peer-to-peer, rather than a simple client -> server structure, it's necessary to give the group manager (the server with a hub and spoke topology) a unique id within the group that all clients can agree on. For tcp-based comm (generic provider), that id is usually the host:port combination (InetAddress) since that's guaranteed globally unique and doesn't require any additional name mapping. However, on the network you are targeting it seems that some clients need to use IP address 1 and other clients IP address 2...to access the same server and service.
I suspect that your network topology may require moving to a strict client -> server notion of rpc, rather than using a group topology. It would mean that all services would be exported by a server and imported by clients, and that no services would be exported by clients or imported by server.
If having a strict client->server works for your services, then I would suggest you try the either the JaxRSRemoteService providers  which are based upon HttpService (jetty server usually). It still seems to me that you would need a reverse proxy like nginx to expose the same server to access via multiple IP addresses/networks, and I'm not sure if that's possible on your target network, but nginx is frequently used for that.
With strict client-server it's possible that ROSGi (based upon tcp) would also work, but since I don't yet understand what your address mapping is about (e.g. NAT, etc), and I haven't tried ROSGI under anything other than 'normal' addressing/networking scenarios, I don't feel comfortable suggesting that. It also seems to me that a tcp reverse proxy might be needed also (a relatively recent nginx feature).
However, it's quite possible to export a single service with multiple generic provider container instances...e.g. couldn't you export service with a container for VPN clients, and another that exports the same service using 'internal' addressing (with different IP and port)?I would like to avoid to having different ways/options/configuration to connect to remote services since that would mean a more or less complicated configuration for developers and consultants connecting to remote services running at the customerAs for other distribution providersIt's possible that R0SGi could work for this, but the VPN might still present a problem.You could use one of the http-based providers (e.g. Jersey or CXF) in combination with the generic provider...e.g. you could export with both generic and http-based distribution providers simultaneously to expose the remote service via two (or more) different transports.
If I'm understanding you right, you will at least need to provide different target IP addresses/port combinations to different clients C1 and C2, right? I don't see a way to avoid that, unless you provide some other public service that does that...i.e. analogous to DNS but just for your app. You could do that, btw without too much difficulty...i.e. create a public/internet etcd server, for example (etcd is the http-based discovery protocol that Google uses for Kubernetes). It would require running yet another server (etcd), maintaining, etc...but at least you don't have to develop it.
Also: Is it possible to consider alternative network topologies? e.g. having both an external and internal servers? or running the service outside vpn/in cloud?No, all installations are on premise at the customersI'm happy to point you in the right direction, but I have a feeling that there could be multiple ways to deal with this, so there might not be just one 'right direction'. Let's keep discussing.I'm open to use/adapt a different distribution provider when that solves the problem.
I would try strict client->server over http first. Actually I would suggest just running small http server on your server process...with a single servlet or even just page, and if you can configure things to access in the way you want via C1 and C2...CN, then you almost certainly will be able to use osgi, jetty + one of the JaxRSRemoteService providers.
My main "wish" would be to decouple the "raw" network connection (use any IP that works or any hostname that works - in term of getting a network connection) and not mix that with the actual service that is then consumed/provided at that network location.
Yes. I think if you can adopt a client -> server/service model for all your services, and can get a combination of tech (e.g. reverse proxy + http server) that allows http access with the desired addressing over your target networks, then I expect you can use one of the JaxRS distribution providers without too much trouble. It will likely require changing the serialization of method args and return values.
For ECF Photon I've done quite a lot of work on the JaxRS distribution providers (e.g. to move them up to OSGi R7 spec and to support both Jersey and CXF underneath), but I've tried to make them extensible and configurable (e.g. using Jaxrs extensions) and will provide direct support for these/all providers as I can.
Scott  https://github.com/ECF/JaxRSProviders  https://wiki.eclipse.org/Tutorial:_JaxRS_Remote_Services_on_Karaf
Back to the top