Nice.
So, Jim! You can run the new client (which is based on JAX-RS 2.0) without needing a jax-rs implementation (like Jersey or RESTeasy)?
I thought JAX-RS is the API, and one needs an implementation, but does not seem to be the case on the client side.
regards
______________________________
Jad El-khoury,
PhD
KTH Royal Institute of Technology
School of Industrial Engineering and Management, Mechatronics Division
Brinellvägen 83, SE-100 44 Stockholm, Sweden
Phone: +46(0)8 790 6877 Mobile: +46(0)70 773 93 45
jad@xxxxxx,
www.kth.se
From: lyo-dev-bounces@xxxxxxxxxxx [mailto:lyo-dev-bounces@xxxxxxxxxxx]
On Behalf Of Jim Amsden
Sent: 28 January 2019 22:15
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Today's progress:
Created two new maven projects, oslc4j-client and oslc4j-client-samples, based on the existing oslc-java-client and oslc-java-samples projects, but implemented using JAX-RS 2.0 and not using Wink.
Reverted the existing oslc-java-client and oslc-java-samples changes back to their 2.4.0 master, and leave these implemented on oslc4j-wink, but still compiled and run against oslc4j-core based on
JAX-RS 2.0.
Both of these compile and run against oslc4j-cm-sample. So other than duplicate code, these two different implementations of an Lyo Java client for OSLC both work and accomplish our goals to:
-
keep the old oslc-java-client built on Wink untouched for existing Java client applications that don't want to migrate to JAX-RS 2.0
-
have a Java client that is simpler, based on JAX-RS 2.0, with no Wink dependencies, and provides more flexible client application configuration for secure access (SSL) and authentication (Basic,
OpenIDConnect, JEE Form)
-
build and run both clients in OSLC4J 4.0.0 and beyond.
Tomorrow I'll start figuring out how to configure the JAX-RS 2.0 ClientBuilder to support SSL (I'll do TrustAll for the example), and JEE Form based authentication. This will identify the constructors
we need.
Some differences in oslc4j-client (based on JAX-RS 2.0)
-
This will be built on OpenIDConnect, not OAuth 1.0a. So OslcOAuthClient, and all its quirks can be removed. This will eliminate the need for OslcClient constructor to take an HttpClient argument
-
I renamed all the sample applications to be different than the Wink-based client samples, and be more logically named.
-
I removed the DoorsOAuthSample because we'll use OpenIDConnect instead, and not worry about updating the sample client for DOORS Web Access - that can stay on the Wink client.
Jim Amsden, Senior Technical Staff Member
OSLC and Linked Lifecycle Data
919-525-6575
From: Andrii Berezovskyi <andriib@xxxxxx>
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Date: 01/28/2019 12:09 PM
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Sent by: lyo-dev-bounces@xxxxxxxxxxx
The HTTP client in Groovy is just for inspiration. I am not planning to bring it in here (for now).
Just an idea of implementing classes with the same name in the same package in different modules, so that the clients don't see much of a difference when working with different impl.
/A
On 2019-01-28 , at 15:14, Jad El-Khoury <jad@xxxxxx> wrote:
Sorry! Email sent itself without my premission
>There is a quick-n-dirty HTTP client in Groovy,…
I guess we need to first decide if we want to freeze Wink at 2.4.0 or maintain it, right?
regards
______________________________
Jad El-khoury, PhD
KTH Royal Institute of Technology
School of Industrial Engineering and Management, Mechatronics Division
Brinellvägen 83, SE-100 44 Stockholm, Sweden
Phone: +46(0)8 790 6877 Mobile: +46(0)70 773 93 45
jad@xxxxxx,
www.kth.se
From:
lyo-dev-bounces@xxxxxxxxxxx[mailto:lyo-dev-bounces@xxxxxxxxxxx]
On Behalf Of Jad El-Khoury
Sent: 28 January 2019 15:13
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
>There is a quick-n-dirty HTTP client in Groovy,
Doesn't Wink just use HttpClient for the client access? Couldn't that be preserved in the current implementation?
Wink can indeed be configured to use HttpClient. This is what is done in the current 2.4.0 implementation.
I see that this can also be done with Jersey.
But, I don’t believe it is possible (or I did not manage to work it out) how that can be done at the JAX-RS 2.0 level. This means, that there will be very little in common (at the intended client-core
level) between the oslc-client-wink and oslc-client-jersey implementations.
> Another option is to freeze Wink support at 2.4.0…
That’s what we were originally aiming for. I guess this latest discussion is to hear from the community (@Nick?) how they feel about that.
>I think we should explore a bit more how to separate olsc4j-wink from JAX-RS 2.0 and support either one in 4.0.0, even if it results in some code duplication. We can deal with that.
There is a lot of good code in 2.4.0 that we want to carry over. (oauth, jazz,…) If we can accept code duplication, it would be much easier, since I am afraid we may not be able to share a common
code base (nor want to refactor as you mentioned below)
>What I don't want to do is spend a lot of time refactoring and possibly introducing defects in oslc4j-wink. I'd rather invest in the future.
My thinking too!
regards
______________________________
Jad El-khoury, PhD
KTH Royal Institute of Technology
School of Industrial Engineering and Management, Mechatronics Division
Brinellvägen 83, SE-100 44 Stockholm, Sweden
Phone: +46(0)8 790 6877 Mobile: +46(0)70 773 93 45
jad@xxxxxx,
www.kth.se
From:
lyo-dev-bounces@xxxxxxxxxxx[mailto:lyo-dev-bounces@xxxxxxxxxxx]
On Behalf Of Andrii Berezovskyi
Sent: 28 January 2019 14:15
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Just a quick note (mostly for myself):
There is a quick-n-dirty HTTP client in Groovy, and there you can just choose an implementation when you add a Maven dependency:
https://http-builder-ng.github.io/http-builder-ng/asciidoc/html5/#_quick_start(note the last sentence under Add Library to Project).
I think we can so something similar: define the same classes in both libs and allow the implementation to be chosen at the maven level. Need to investigate how do they do it but seems like a normal reuse (or a slight
abuse) of the same Java package name in multiple artefacts.
/A
On 2019-01-28 , at 14:02, Jim Amsden <jamsden@xxxxxxxxxx> wrote:
Jad,
Doesn't Wink just use HttpClient for the client access? Couldn't that be preserved in the current implementation?
Another option is to freeze Wink support at 2.4.0 and focus new server development work on 4.0.0 and beyond. That leaves current server implementations that use Wink stuck on 2.4.0, but that might be fine since these servers might not be getting a lot of new
development and won't need the new features in 4.0.0 and beyond?
I think we should explore a bit more how to separate olsc4j-wink from JAX-RS 2.0 and support either one in 4.0.0, even if it results in some code duplication. We can deal with that.
What I don't want to do is spend a lot of time refactoring and possibly introducing defects in oslc4j-wink. I'd rather invest in the future.
Jim Amsden, Senior Technical Staff Member
OSLC and Linked Lifecycle Data
919-525-6575
From: Jad El-Khoury <jad@xxxxxx>
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Date: 01/27/2019 12:12 PM
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Sent by: lyo-dev-bounces@xxxxxxxxxxx
Hi
Nick! Any assistance from you or colleagues would be very appreciated.
At the very least, it seems to me that you could be a representative of the expectations & needs on Lyo Client, given your contacts with IBM customers.
On a technical note, ...
1. client-core would depends only on JAX-RS 2.0 and oslc-core
2. client-wink would be what we have now, unchanged
My original thought was that client-wink could be refactored to be built on top of client-core. This would be in the same way oslc4j-wink builds on oslc-core (which is to be based on jax-rs2.0).
This way the different client implemention can share common code base.
I did some experiments over the weekend, and this unfortunately does not seem to be possible. The simple reason being that - unlike for server-side - there is no JAX-RS defined client API. So, the Wink classes have nothing in common with the jax-Rs client API,
Which is new in 2.0.
Maybe this was obvious for many?
Jim! I guess that’s what you mean by stating that “client-wink would be unchanged” literally?
If we do that, there will be much duplicated code between the wink and jax-rs2.0 implementations.
Regards,
Jad
On 25 Jan 2019, at 17:39, Nicholas Crossley <nick_crossley@xxxxxxxxxx> wrote:
Client code is pretty much my only interest in Lyo - I have no requirements to create servers - so ye, I am very interested in a new Java client. I know that same interest is shared by several IBM customers who want to write integrations with existing OSLC
servers (IBM CLM applications, to be specific), and some of those have asked for more modern examples than the current Lyo ones. I would like to contribute, but my availability is extremely limited. I will also look into getting assistance from some of my
colleagues, but cannot promise anything in that regard.
Nick.
From: "Jim Amsden" <jamsden@xxxxxxxxxx>
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Date: 01/25/2019 08:13 AM
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Sent by: lyo-dev-bounces@xxxxxxxxxxx
Nick,
Thanks for the feedback, you're opinion means a lot.
Re: authentication: there's perhaps a few approaches we could take.
1. its up to the client app - OslcClient(ClientBuilder) wouldn't provide any authentication other than throwing an Unauthorized exception to the client app. Client apps would be responsible for setting up a ClientBuilder or handling the authentication challenge
exceptions that met their needs.
This is easy for eclipse/Lyo, but pushes too much onto clients.
2. OslcClient(ClientBuilder, userID, password) would configure the REST client, and be prepared to handle authentication challenges by examining returned headers, and using the appropriate authentication technique to respond to the challenge. It would be nice
to support:
-
JEE Form
-
Basic
-
OAuth 1.0a
-
OpenIDConnect
This makes authentication transparent to the users, but would be a lot of work for eclipse/Lyo, and might not handle all the cases.
3. A combination of the two where authentication challenges not handled by OslcClient would be raised to the the client app, where additional ClientConfiguration of other approaches can deal with the challenge.
This could be a significant amount of work and testing. I wonder what the demand is for this. Does anyone on this list want/need a new OSLC4J Java Client? Would anyone be interested in contributing?
Jim Amsden, Senior Technical Staff Member
OSLC and Linked Lifecycle Data
919-525-6575
From: "Nicholas Crossley" <nick_crossley@xxxxxxxxxx>
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Date: 01/25/2019 10:03 AM
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Sent by: lyo-dev-bounces@xxxxxxxxxxx
Jim,
I think your description of how to handle the JAX RS stuff is fine.
What about the authentication aspects? This is one of the hardest parts to get right when using OSLC to work with IBM CLM integration, so I would prefer the newer client libraries did handle the recommended authentication. In other words I completely agree
with your earlier statement that we should "create a completely new client API and sample apps [using Jersey] that does support SSL, Basic and OpenIDConnect properly, while being sufficiently open with the client configuration to support whatever the client
application needs". I would interpret that as a requirement either that client-core supports SSL, Basic and OpenIDConnect authentication, or that authentication is split into another library with one implementation providing such support. Separating the authentication
code might make that easier to update in the future.
Nick.
From: "Jim Amsden" <jamsden@xxxxxxxxxx>
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Date: 01/25/2019 05:34 AM
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Sent by: lyo-dev-bounces@xxxxxxxxxxx
Jad,
I think you have this exactly right.
1. client-core would depends only on JAX-RS 2.0 and oslc-core
2. client-wink would be what we have now, unchanged
3. OSLC client applications that use client-core would provide whatever implementation of JAX-RS 2.0 they want, and configure it how they want
4. we will have two versions of the sample client applications, one that is unchanged and uses client-wink, and another that uses client-core and provide examples on how to use Jersey to create runnable code.
Any wink/jersey based client can work with any wink/jersey based server.
No client or server would need to support both wink and jersey
Do we all agree?
Jim Amsden, Senior Technical Staff Member
OSLC and Linked Lifecycle Data
919-525-6575
From: Jad El-Khoury <jad@xxxxxx>
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Date: 01/24/2019 06:30 PM
Subject: Re: [lyo-dev] Migrating lyo.client from Wink to Jersey
Sent by: lyo-dev-bounces@xxxxxxxxxxx
Jim
I believe the aim is to migrate OSLC4J from JAX-RS 1.1 to 2.0 (and not necessarily from Wink to Jersey). But we also wanted to remove the dependency on Wink, to allow developers using Lyo to adopt other JAX_RS implementations.
There was a particular desire to make “the client library dependencies were much smaller and simpler - today with Wink et al they are huge” – as formulated by Nick Crossley.
So, we should not be adding a dependency on Jersey, right?
I have not earlier used OslcOAuthClient nor JazzFormAuthClient, so I did not fully grasp all the details in your summary. My general understanding from your text is that (1) they are not up to date with oslc 3.0 (2) they are far from perfect.
As a small step that allows us to move away from Jax-rs 1.1, while maintaining Wink support, does it make sense to split the current client code into 2 projects:
1. client-core that offers the CRUD and lookup functionalities, but without having to deal with client (or clientConfig) constructors.
2. Client-Wink that offers the capabilities of creating clients for OslcOAuthClient, JazzFormAuthClient & OslcClient
The former is NOT dependent on Wink, but on Jax-rs2.0 only. The latter is dependent on Wink.
In future steps, we can then improve the 2nd to support clients for OpenIDConnect (for example). But we can also provide a Client-Jersey alternative.
regards
______________________________
Jad El-khoury, PhD
KTH Royal Institute of Technology
School of Industrial Engineering and Management, Mechatronics Division
Brinellvägen 83, SE-100 44 Stockholm, Sweden
Phone: +46(0)8 790 6877 Mobile: +46(0)70 773 93 45
jad@xxxxxx,
www.kth.se
From:lyo-dev-bounces@xxxxxxxxxxx[mailto:lyo-dev-bounces@xxxxxxxxxxx]
On Behalf Of Jim Amsden
Sent: 24 January 2019 22:15
To: Lyo project developer discussions <lyo-dev@xxxxxxxxxxx>
Subject: [lyo-dev] Migrating lyo.client from Wink to Jersey
There are technical, security and maintainability benefits for migration OSLC4J from Wink to Jersey. I won't discuss the motivation here. Rather I want to explore possible design strategies and API and implementation implications. For lyo.client, this will
be a significant API breaking change that may push responsibility for certain HTTP communication configuration up into the client applications. Although these changes will improve security, OSLC4J users may be challenged to migrate their existing client applications
to 4.0.0, especially if other parts of their technical architecture have related/coupled technical debt that they are not prepared to address. So we should make this change carefully.
Ideally OSLC4J 4.0.0 lyo.client would support Wink or Jersey, allowing users to migrate as needed in their applications. This not explores how the current OslcClient, its subclasses, and client applications work, and how this might change when migrating to
Jersey.
There are currently three OslcClient variants:
OslcClient: provides generic CRUD method for OSLC resource access supporting SSL with configurable trust managers, with a default trust manager that trusts all client/server interactions over https. This capability is often used during development and testing.
JazzFormAuthClient: Extends OslcClient to support Java EE Form based authentication as used by the
jazz.netapps. OSLC client applications typically use JazzRootServicesHelper
to read the jazz.netapp rootservices document, and provide the URLs to the
services' ServiceProviderCatalogs.
OslcOAuthClient: extends OslcClient to support OAuth authentication as used by the DOORS Web Client.
This perhaps identifies our first design issue. A lot has changed in authorization since lyo.client was originally developed. The
jazz.netapps now provide lots of ways to authentication including OpenIDConnect.
Also other OSLC servers like the iotp-adaptor on the Watson IoT Platform use HTTP Basic authentication of SSL and this is not directly supported, requiring an interceptor in order to add the authentication header.
Ideally lyo.client would at least support the two authentication mechanisms recommended by OSLC Core 3.0: Basic and OpenIDConnect, neither of which are supported now.
But ignoring that, let's look at how these current OslcClient variants work and what would be required to migrate them to Jersey.
Simple OSLC client applications
The simplest OSLC client applications that don't need SSL and don't require login. GenericRMSample is an example of such a client application.
These client applications can use the default OslcClient() constructor. This default constructor supports http and automatically trusts all client/server communications with all hosts over https. Alternatively client applications can use the OslcClient(TrustManager[],
X509HostnameVerifier) constructor to specify more specific and secure communications over SSL.
OslcClient supports:
-
create, get, update and delete resource
-
Lookup the ServiceProvider URL given a ServiceProviderCatalog URL and ServiceProvider title
-
Lookup the query capability URL given a ServiceProvider URL
-
Lookup the creation factory URL given the ServiceProvider URL and possibly the OSLC domain, resource type and OSLC usage
Here's a brief summary of the OslcClient(TrustManager[], X509HostnameVerifier) constructor:
Creates a TheadSaveClientConnManager DefaultHttpClient
-
Turns off redirects.
-
unregisters https from the SchemeRegistry of the httpClient
-
Creates a TrustManager[] that has an X509TrustManager that ignores checkClientTrusted and checkServerTrusted calls in order to trust all certificates - a security issue!!
-
uses this TrustManager[] if the one provided in the OslcClient constructor was null
-
uses SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER if the one provided by the OslcClient constructor is null. Also a security issue!!
-
Creates an sslContext from a list of secure socket protocols: TLSv1.2, TLS, SSL, SSL_TLS
-
initializes the sslContext with the trust managers
-
re-registers a new https Scheme in the SchemeRegistry with an SSLSocketFactory that uses the SSL context and host name verifier created above
-
Creates a javax.ws.rs.core.Application and adds the JenaProvidersRegistry and Json4JProvidersRegistry providers.
-
configures the clientConfig with the applications
-
uses this clientConfig to the Wink RestClient used to access all resources
There are a couple of additional methods in OslcClient that are only used by the OAuth clients. These will be discussed below.
jazz.netOSLC client applications
In the early days, all the jazz.netapplications used JEE Form based authentication.
lyo.client provides JazzFormAuthClient as an extension to OslcClient to handle the Form based authentication. However, client applications generally don't use JazzFormAuthClient directly, they instead use it indirectly through JazzRootServicesHelper. RTCFormSample
is an example of such a client application.
JazzRootServicesHelper:
-
Constructor figures out the namespace and service providers property to use to identify the ServiceProviderCatalog URLs in the
jazz.netapp rootservices document
-
uses new OslcClient() to read the root services document when it is constructed with JazzRootServicesHelper (String url, String catalogDomain). Authentication isn't required to read this document.
-
Then the client application calls helper.initFormClient(userid, password) which constructs a JazzFormAuthClient from the helper's baseUrl, and the userid, password.
-
The client application then calls client.login() to actually login using Jazz Form based authentication.
-
the client application can now use the JazzFormAuthClient subclass methods to do OSLC CRUD operations on
jazz.netapp resources, and use the service discovery method of an OslcClient
-
This immediately raises the issue that the JazzRootServicesHelper, and JazzFormAuthClient do not provide any means for the OSLC client application to specify trust managers or
host name validators, all SSL client/server interaction to any server is unchecked! SSL is essentially ignored.
So I'm not sure this could be used in a potentially insecure production environment.
OAuth OSLC client applications
Since DOORS Web Access required OAuth, lyo.client needed to provide OslcOAuthClient to extend OslcClient to provide OAuth 1.0a authentication (which was in common use at the time). DoorsOauthSample is an example of such a client application.
OSLC client applications that use OAuth also use JazzRootServicesHelper to not only get the ServiceProviderCatalog URLs, but also the various OAuth URLs including:
<jfs:oauthDomain>https://ce4iot.rtp.raleigh.ibm.com:9443/iotp</jfs:oauthDomain>
<jfs:oauthRequestConsumerKeyUrl rdf:resource="https://host:9443/iotp/services/oauth/requestKey"
/>
<jfs:oauthApprovalModuleUrl rdf:resource="https://host:9443/iotp/services/oauth/approveKey"
/>
<jfs:oauthRequestTokenUrl rdf:resource="https://host:9443/iotp/services/oauth/requestToken"/>
<jfs:oauthUserAuthorizationUrl rdf:resource="https://host:9443/iotp/services/oauth/authorize"
/>
<jfs:oauthAccessTokenUrl rdf:resource="https://host:9443/iotp/services/oauth/accessToken"/>
The OSLC client app then uses the helper.initOAuthClient with the consumer key and secret to construct an OslcOAuthClient instance that also has the OAuth URLs obtained from the rootservices document.
The OSLC client application then attempts to get a protected resource, catches any OAuthRedirectException, and then calls its validateTokens method to do the OAuth 1.0a dance, given the user id and password (there is no browser popup involved to access the
user's credentials).
This raises the same issue as JazzFormAuthClient. All the OslcOAuthClient constructors also use the default OslcClient() constructor which again trusts all client/server communications from all hosts over SSL. In addition, a client application that uses parameters
for user ids and passwords this way is achieving no additional security over HTTP Basic while incurring the additional complexity and overhead of OAuth and its variants. This is because there's no independent step between the client and the server where the
user gets a chance to decide if they want to provide their credentials in response to an authentication challenge - the client app just does it.
Potential conclusion
With this analysis, it appears lyo.client doesn't support the recommended OSLC Core 3.0 approaches to authentication, and interacts apps requiring JEE Form based or OAuth authentication without any SSL protections.
I'm inclined to see if there's a way to leave the current lyo.client Java client API on Wink and create a completely new client API and sample apps using Jersey that does support SSL, Basic and OpenIDConnect properly, while being sufficiently open with the
client configuration to support whatever the client application needs.
Thoughts?
Sorry for the long note. But this wasn't documented that I know of, and I couldn't reason about the situation without writing down what's there.
Jim Amsden, Senior Technical Staff Member
OSLC and Linked Lifecycle Data
919-525-6575_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
_______________________________________________
lyo-dev mailing list
lyo-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/lyo-dev
