Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » Eclipse EGit TFS Git Connection - Authentication not supported
Eclipse EGit TFS Git Connection - Authentication not supported [message #1841309] Tue, 11 May 2021 19:08 Go to next message
Aslam Ahmed is currently offline Aslam AhmedFriend
Messages: 4
Registered: May 2021
Junior Member
Hi,

We are facing this dreadful issue "authentication not supported" when using egit from eclipse. I have read many articles, questions but none of them seem to solve the issue at hand.

We are using Microsoft Team Foundation Server 2015 which can provide a git repository as version control system. But authentication mechanism does not seem to work using eclipse's egit plugin. We checked using different eclipse versions from eclipse 2020-06, 2020-12, 2021-03 with different jdk setup 1.8, 11 and 15 but no success.

The cloning, push/fetch works fine with git commandline (git bash - windows) even legacy eclipse kepler is working fine without issue, but not the latest ones. The issue seems to be related to NTLM authentication. In newer versions of eclipse, egit doesn't negotiate or send NTLM token even after the server respond with WWW-Authenticate: NTLM.

Below are details for both newer and older eclipse versions, tunneled through TCP/IP Monitor for traffic capture. TFS git repository clone protocol is http even though the portal is on https.

Eclipse 2021-03

Request:
GET /tfs/xxx/_git/xxx/info/refs?service=git-upload-pack HTTP/1.1
Accept-Encoding: gzip
Pragma: no-cache
User-Agent: JGit/5.11.0.202103091610-r
Accept: application/x-git-upload-pack-advertisement, */*
Git-Protocol: version=2
Host: tfs-xxx-xx:8090
Connection: Keep-Alive


Response:

HTTP/1.1 401 Unauthorized
Cache-Control: private
Content-Type: text/html
Server: Microsoft-IIS/8.5
X-TFS-ProcessId: 76ec3355-4bc2-498d-ba64-xxxxxxx
X-FRAME-OPTIONS: SAMEORIGIN
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 3600
Access-Control-Allow-Methods: OPTIONS,GET,POST,PATCH,PUT,DELETE
Access-Control-Expose-Headers: ActivityId,X-TFS-Session
Access-Control-Allow-Headers: authorization
Set-Cookie: Tfs-SessionId=3b370bd6-2197-xxxxxxxxx; path=/
Set-Cookie: Tfs-SessionActive=2021-05-11T17:25:24; path=/
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
P3P: CP="CAO xxxxxxxxxxxxxxxxx"
X-Content-Type-Options: nosniff
Date: Tue, 11 May 2021 17:25:24 GMT
Content-Length: 1293


Eclipse Kepler

First Request/Response is similar to above.


Second Request:

GET /tfs/xxx/_git/xxx/info/refs?service=git-upload-pack HTTP/1.1
Accept-Encoding: gzip
Pragma: no-cache
User-Agent: JGit/3.2.0.201312181205-r
Accept: application/x-git-upload-pack-advertisement, */*
Cache-Control: no-cache
Host: tfs-pmo-app:8090
Connection: keep-alive
Authorization: NTLM TlRMTVxxxxxxx - Length 72 chars


Second Response:

HTTP/1.1 401 Unauthorized
Content-Type: text/html; charset=us-ascii
WWW-Authenticate: NTLM TlRMxxxxxx== - Length 354 chars
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 11 May 2021 17:33:00 GMT
Content-Length: 341


Third Request:


GET /tfs/xxx/_git/xxx/info/refs?service=git-upload-pack HTTP/1.1
Accept-Encoding: gzip
Pragma: no-cache
User-Agent: JGit/3.2.0.201312181205-r
Accept: application/x-git-upload-pack-advertisement, */*
Cache-Control: no-cache
Host: tfs-pmo-app:8090
Connection: keep-alive
Authorization: NTLM TlRMTVNxxxxx - Length 256 chars


Third Response:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/x-git-upload-pack-advertisement
Server: Microsoft-IIS/8.5
X-TFS-ProcessId: 76ec3355-4bc2-xxxxxxxxxxxxxx
X-FRAME-OPTIONS: SAMEORIGIN
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 3600
Access-Control-Allow-Methods: OPTIONS,GET,POST,PATCH,PUT,DELETE
Access-Control-Expose-Headers: ActivityId,X-TFS-Session
Access-Control-Allow-Headers: authorization
Set-Cookie: Tfs-SessionId=xxxxxxxxxxxx; path=/
Set-Cookie: Tfs-SessionActive=2021-05-11T17:33:01; path=/
X-VSS-UserData: 69b58752-xxxxxxxxxxxxx
X-AspNet-Version: 4.0.30319
Persistent-Auth: true
X-Powered-By: ASP.NET
P3P: CP="CAO xxxxxxxxxxxxxxxxxxxxxxxxx"
X-Content-Type-Options: nosniff
Date: Tue, 11 May 2021 17:33:00 GMT
Content-Length: 1098


What we have tried so far, increasing connection timeout to 300, changing http client in Eclipse Preferences for Git, adding username/password when cloning repo - this doesn't have any visible impact, changing different variations of eclipse with jdk, changing git credentials helper to wincred, manager, store even resetting without any credentials helper, tried configuring cntlm but during test it says that you don't need proxying, none of this seems to work.

This probably means there is a bug in egit/jgit implementation. Looking at the current implementation source and tracing stack from error:

!ENTRY org.eclipse.egit.core 4 0 2021-05-08 04:56:02.018
!MESSAGE Pulling 1 repository
!SUBENTRY 1 org.eclipse.egit.core 4 0 2021-05-08 04:56:02.021
!MESSAGE http://xxxxx:8090/tfs/xxxx/_git/xxx: authentication not supported
!STACK 0
org.eclipse.jgit.api.errors.TransportException: http://xxxxx:8090/tfs/xxxx/_git/xxx: authentication not supported
    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:224)
    at org.eclipse.jgit.api.PullCommand.call(PullCommand.java:263)
    at org.eclipse.egit.core.op.PullOperation$PullJob.run(PullOperation.java:255)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: org.eclipse.jgit.errors.TransportException: http://xxxxx:8090/tfs/xxxx/_git/xxx: authentication not supported
    at org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:674)
    at org.eclipse.jgit.transport.TransportHttp.openFetch(TransportHttp.java:465)
    at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:142)
    at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:94)
    at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1309)
    at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:213)
    ... 3 more


https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/refs/tags/v5.11.0.202103091610-r/org.eclipse.jgit/src/org/eclipse/jgit/transport/TransportHttp.java

org.eclipse.jgit.transport.TransportHttp.connect(TransportHttp.java:674):

Line 671:   case HttpConnection.HTTP_UNAUTHORIZED:
Line 672:                       authMethod = HttpAuthMethod.scanResponse(conn, ignoreTypes);
Line 673:                       if (authMethod.getType() == HttpAuthMethod.Type.NONE)
Line 674:                           throw new TransportException(uri, MessageFormat.format(
Line 675:                                   JGitText.get().authenticationNotSupported, uri));



Looks like scanResponse method is returning none for auth method type which is causing the issue, so lets review HttpAuthMethod:

https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/refs/tags/v5.11.0.202103091610-r/org.eclipse.jgit/src/org/eclipse/jgit/transport/HttpAuthMethod.java

The code here basically checks for 3 authentication methods, BASIC, DIGEST and NEGOTIATE, in our case the header "WWW-Authenticate: NTLM" is none of the supported onces hence the exception.

But the confusing part is how the hell is kepler successfully authenticating for NTLM, the answer probably (through web-searches) is that actually jdk maybe responsible for that as its under HttpURLConnection class is inherently sending NTLM tokens which may not be the case for newer jdks (don't know why).

Conclusion: I don't seem to find the right answers on where does the problem lie?

Is it because of new jdks not inherently supporting NTML (under the hood)?
Shouldn't egit/jgit fall back to other mechanism like BASIC authentication?
Why can't I send my credentials pre-emptively using egit/jgit to bypass NTLM?

P.S. When writing this post, it looks like our server may not have been configured for BASIC authentication support, but I tried different authentication types using git command and they did work. I will check again with wireshark capture and see the underlying mechanism.

Thanks, if you read the whole post :).
Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841319 is a reply to message #1841309] Wed, 12 May 2021 07:04 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 363
Registered: August 2016
Senior Member
Your analysis is correct and summarizes very well the state of affairs with TFS and NTLM.

  1. JGit itself has no implementation for NTLM. It may work -- I've seen network traces showing so -- if the underlying URL connection does the authentication under the hood. JGit has two possible implementations, one using the JDK's HttpUrlConnection, the other using Apache HttpClient. I do not know under which circumstances NTLM happens to work; it may perhaps also depend on configuration outside of the Java process? In any case it might be worth trying to change the implementation used in the Eclipse Git preferences. (Default is Apache HttpClient.)
  2. Looking at the requests you show above, I think JGit doesn't fall back to BASIC because the server does not announce it.
  3. I've seen mentions that using the cntlm proxy may help.
  4. In JGit 5.11.0, you should be able to use pre-emptive BASIC auth; see the JGit New & Noteworthy page. But if the server doesn't accept BASIC auth, that won't help. EGit has no UI for using pre-emptive BASIC auth; but it should be used if the remote URL in the git config includes the username/password as shown on the N&N page.
  5. You seem to use a plain HTTP URL, not HTTPS. BASIC auth should not be used over plain HTTP. (Which is probably also why your TFS doesn't announce it. If you reconfigure the server to allow BASIC auth, also change it to be accessible over HTTPS only.)

None of the JGit maintainers use TFS or NTLM or Windows machines, so we can't track this down ourselves.
Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841354 is a reply to message #1841319] Wed, 12 May 2021 22:08 Go to previous messageGo to next message
Aslam Ahmed is currently offline Aslam AhmedFriend
Messages: 4
Registered: May 2021
Junior Member
Finally I managed to resolve the issue. First feedback on Thomas's points (BTW thanks for your response):


Quote:

1. JGit itself has no implementation for NTLM. It may work -- I've seen network traces showing so -- if the underlying URL connection does the authentication under the hood. JGit has two possible implementations, one using the JDK's HttpUrlConnection, the other using Apache HttpClient. I do not know under which circumstances NTLM happens to work; it may perhaps also depend on configuration outside of the Java process? In any case it might be worth trying to change the implementation used in the Eclipse Git preferences. (Default is Apache HttpClient.)

Yes, I already did this even tried different variations of eclipse and jdk but didn't work.

Quote:

2. Looking at the requests you show above, I think JGit doesn't fall back to BASIC because the server does not announce it.

Thats correct I also checked with git cmd using url based user/pass and analyze the traffic in wireshark, but it didn't force basic authentication and always used NTLM authentication.

Quote:

3. I've seen mentions that using the cntlm proxy may help.

Tried that too, first there is no good documentation since over vpn I was not able to find any proxy configuration. Second even after setting up a local proxy (ccproxy) which acts as parent proxy, testing using cntlm revealed that there is no need for another proxy as proxy is already open (didn't undestand).

Quote:

4. In JGit 5.11.0, you should be able to use pre-emptive BASIC auth; see the JGit New & Noteworthy page. But if the server doesn't accept BASIC auth, that won't help. EGit has no UI for using pre-emptive BASIC auth; but it should be used if the remote URL in the git config includes the username/password as shown on the N&N page.

Tried that and saw traffic, but I was not able to find any authorization header.

Quote:

5. You seem to use a plain HTTP URL, not HTTPS. BASIC auth should not be used over plain HTTP. (Which is probably also why your TFS doesn't announce it. If you reconfigure the server to allow BASIC auth, also change it to be accessible over HTTPS only.)

Yes, it seems we would have to go that way if we configure our server for BASIC auth, then https is a must. But since it will require some approvals :) so I kept this for last option.

Now since the kepler version (jdk 1.7) was able to communicate with tfs, so I looked to see if there is something changed in jdks after 1.7 and luckily there was:
https://stackoverflow.com/questions/54555127/ntlm-authentication-in-httpurlconnection-not-working-in-jre-but-works-in-jdk-env
https://stackoverflow.com/questions/54568189/how-to-provide-ntlm-authentication-while-calling-any-url

From jdk1.8.0_181 onwards http based ntlm transparent authentication is disabled by default so, just setting it to allHosts fixed the issue.


All you need to do is comment this line in jre/lib/net.properties file for Java 8, or <JAVA_HOME>/conf/net.properties for Java 11/15:

#jdk.http.ntlm.transparentAuth=disabled


and uncomment this one:
jdk.http.ntlm.transparentAuth=allHosts


Also since this works with HttpUrlConnection client so we also need to change git http client configuration to use "Java built-in HTTP" rather than "Apache HTTP".
Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841369 is a reply to message #1841354] Thu, 13 May 2021 09:11 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 363
Registered: August 2016
Senior Member
Great find! Looks like this can be set also as a system property on the command line: "-Djdk.http.ntlm.transparentAuth=allHosts".

Could you please check if it also works if you add this system property in eclipse.ini (leaving the JDK net.properties at "disabled")? If it does, I'll update the EGit User's Guide.

[Updated on: Thu, 13 May 2021 09:30]

Report message to a moderator

Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841371 is a reply to message #1841369] Thu, 13 May 2021 09:41 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 363
Registered: August 2016
Senior Member
I wonder why it doesn't work with Apache HttpClient. Apache does include an NTLM authentication implementation.
Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841383 is a reply to message #1841371] Thu, 13 May 2021 20:57 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 363
Registered: August 2016
Senior Member
Thomas Wolf wrote on Thu, 13 May 2021 09:41
I wonder why it doesn't work with Apache HttpClient. Apache does include an NTLM authentication implementation.

Still unclear. Looking at SystemDefaultCredentialsProvider (which is what JGit uses), it seems it should work if

  • the user name is given as "DOMAIN\\username", or
  • system property "http.auth.ntlm.domain" is set to the NTLM domain, and the username is given without domain.


Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841408 is a reply to message #1841383] Fri, 14 May 2021 17:12 Go to previous messageGo to next message
Aslam Ahmed is currently offline Aslam AhmedFriend
Messages: 4
Registered: May 2021
Junior Member
Quote:

Great find! Looks like this can be set also as a system property on the command line: "-Djdk.http.ntlm.transparentAuth=allHosts".

Could you please check if it also works if you add this system property in eclipse.ini (leaving the JDK net.properties at "disabled")? If it does, I'll update the EGit User's Guide.

This indeed works. Thanks for the hint.

Quote:

the user name is given as "DOMAIN\\username", or
system property "http.auth.ntlm.domain" is set to the NTLM domain, and the username is given without domain.

I tried both options, unfortunately none of them worked. I can further debug this if you want, I just need to know how to setup debugging for egit, let me know if that would be helpful.

[Updated on: Fri, 14 May 2021 17:35]

Report message to a moderator

Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841410 is a reply to message #1841408] Fri, 14 May 2021 18:16 Go to previous messageGo to next message
Aslam Ahmed is currently offline Aslam AhmedFriend
Messages: 4
Registered: May 2021
Junior Member
For Apache Http not using NTLM, I think the exception is thrown before reaching getCredentialsProvider check the following in TransportHttp:

				case HttpConnection.HTTP_UNAUTHORIZED:
					authMethod = HttpAuthMethod.scanResponse(conn, ignoreTypes);
					if (authMethod.getType() == HttpAuthMethod.Type.NONE)
						throw new TransportException(uri, MessageFormat.format(
								JGitText.get().authenticationNotSupported, uri));
					CredentialsProvider credentialsProvider = getCredentialsProvider();
					if (credentialsProvider == null)
						throw new TransportException(uri,
								JGitText.get().noCredentialsProvider);
					if (authAttempts > 1)
						credentialsProvider.reset(currentUri);
					if (3 < authAttempts
							|| !authMethod.authorize(currentUri,
									credentialsProvider)) {
						throw new TransportException(uri,
								JGitText.get().notAuthorized);
					}
					authAttempts++;
					continue;


Since authMethod only has 3 types BASIC, DIGEST, NEGOTIATE and we are receiving NTLM in authentication header, therefore exception is thrown first without reattempting credential provider.

[Updated on: Fri, 14 May 2021 18:17]

Report message to a moderator

Re: Eclipse EGit TFS Git Connection - Authentication not supported [message #1841414 is a reply to message #1841410] Fri, 14 May 2021 22:46 Go to previous message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 363
Registered: August 2016
Senior Member
If Apache doesn't do the NTLM auth transparently you're right. JDK apparently manages to do it all on its own under the hood without giving the application code any work to do.

Anyway, we're not likely to change anything in JGit for NTLM. The protocol is not nice, and Microsoft themselves state new applications shouldn't use NTLM. With that system property you found there is a way for users to get it to work using the JDK client, so that's good enough.

I've updated the EGit User's Guide.
Previous Topic:Using JGit as a web service with multiple users using same repository
Next Topic:Problem checking out large files with JGIT LFS BuiltinLFS
Goto Forum:
  


Current Time: Sat Jun 12 18:49:55 GMT 2021

Powered by FUDForum. Page generated in 0.02626 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top