Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » Git clone/pull with SSH from Azure DevOps repositories fails frequently(Error "remote end hung up unexpectedly")
Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1831933] Thu, 03 September 2020 08:21 Go to next message
Corny de Souza is currently offline Corny de SouzaFriend
Messages: 3
Registered: September 2020
Junior Member
Hello,

we recently moved our git repositories to Azure DevOps and since then we are having issues accessing them with git clone/pull.

Error "remote end hung up unexpectedly"

This error usually occurs on the first clone/pull on a specific repository. If the clone/pull is executed immediately after the failure, then the clone/pull usually succeeds.

If the SSH client is changed to JSch then the problem also occurs with the error "session is down"

If we use the plain old command line git executable (2.17.1.windows.2) then everything works as expected.

Eclipse Version: Eclipse 2020-06
EGit/JGit Version: 5.8.0
Backend: Apache MINA

Any help provided would be much appreciated.

Thanks
Corny
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832031 is a reply to message #1831933] Sun, 06 September 2020 16:33 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
Could you try this with the JGit command-line client with debug logging enabled? See bug 545920 comment 5 for how to do that. (But use the latest JGit command-line executable.) Without any logs I can't even begin to formulate any hypothesis.

The JGit command-line executable is a Unix shell script; I suppose it should be possible to run it in the git shell on Windows.

Also helpful for a comparison would be a log from git-for-windows doing the clone or pull. You should be able to capture such a log by setting environment variable GIT_SSH_COMMAND or git config core.sshCommand to ssh -vvv.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832032 is a reply to message #1832031] Sun, 06 September 2020 16:41 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
Also: is there anything in the Eclipse logs? "remote end hung up unexpectedly" should log an exception with a stack trace; maybe that gives some more hints.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832103 is a reply to message #1832032] Wed, 09 September 2020 06:25 Go to previous messageGo to next message
Corny de Souza is currently offline Corny de SouzaFriend
Messages: 3
Registered: September 2020
Junior Member
We could not reproduce the error with the JGit command-line client, log is however attached.

The error persists with Eclipse though.

Eclipse Log for Clone:
eclipse.buildId=4.16.0.I20200604-0540
java.version=1.8.0_202
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
Framework arguments:  -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -data D:\Development\LokalWS\OxygenWS_master -product org.eclipse.epp.package.java.product

org.eclipse.egit.ui
Error
Tue Sep 08 11:07:08 CEST 2020
Git repository clone failed.

org.eclipse.jgit.api.errors.TransportException: git@ssh.dev.azure.com:v3/storeit/dopix-products/dopix.core: remote hung up unexpectedly
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222)
	at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:302)
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:178)
	at org.eclipse.egit.core.op.CloneOperation.run(CloneOperation.java:194)
	at org.eclipse.egit.ui.internal.clone.AbstractGitCloneWizard.executeCloneOperation(AbstractGitCloneWizard.java:489)
	at org.eclipse.egit.ui.internal.clone.AbstractGitCloneWizard.access$2(AbstractGitCloneWizard.java:482)
	at org.eclipse.egit.ui.internal.clone.AbstractGitCloneWizard$5.run(AbstractGitCloneWizard.java:443)
	at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)
Caused by: org.eclipse.jgit.errors.TransportException: git@ssh.dev.azure.com:v3/storeit/dopix-products/dopix.core: remote hung up unexpectedly
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:270)
	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144)
	at org.eclipse.jgit.transport.FetchProcess.reopenConnection(FetchProcess.java:256)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:156)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211)
	... 7 more
Caused by: java.lang.IllegalStateException: Channel id=1 not registered because session is being closed: ClientConnectionService[JGitClientSession[git@ssh.dev.azure.com/52.236.147.103:22]]
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.handleChannelRegistrationFailure(AbstractConnectionService.java:437)
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.registerChannel(AbstractConnectionService.java:427)
	at org.apache.sshd.client.session.AbstractClientSession.createExecChannel(AbstractClientSession.java:330)
	at org.apache.sshd.client.session.ClientSession.createExecChannel(ClientSession.java:183)
	at org.eclipse.jgit.transport.sshd.SshdSession.exec(SshdSession.java:136)
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254)
	... 13 more


Eclipse Log for Fetch
eclipse.buildId=4.16.0.I20200604-0540
java.version=1.8.0_202
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
Framework arguments:  -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -data D:\Development\LokalWS\OxygenWS_master -product org.eclipse.epp.package.java.product

org.eclipse.egit.ui
Error
Tue Sep 08 11:12:48 CEST 2020
ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly

org.eclipse.core.runtime.CoreException: ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly
	at org.eclipse.egit.ui.internal.fetch.FetchOperationUI.execute(FetchOperationUI.java:113)
	at org.eclipse.egit.ui.internal.fetch.FetchOperationUI$1.performJob(FetchOperationUI.java:136)
	at org.eclipse.egit.ui.internal.jobs.RepositoryJob.run(RepositoryJob.java:59)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222)
	at org.eclipse.egit.core.op.FetchOperation.run(FetchOperation.java:134)
	at org.eclipse.egit.ui.internal.fetch.FetchOperationUI.execute(FetchOperationUI.java:110)
	... 3 more
Caused by: org.eclipse.jgit.errors.TransportException: ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:270)
	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144)
	at org.eclipse.jgit.transport.FetchProcess.reopenConnection(FetchProcess.java:256)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:156)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211)
	... 5 more
Caused by: java.lang.IllegalStateException: Channel id=1 not registered because session is being closed: ClientConnectionService[JGitClientSession[git@ssh.dev.azure.com/40.74.28.0:22]]
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.handleChannelRegistrationFailure(AbstractConnectionService.java:437)
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.registerChannel(AbstractConnectionService.java:427)
	at org.apache.sshd.client.session.AbstractClientSession.createExecChannel(AbstractClientSession.java:330)
	at org.apache.sshd.client.session.ClientSession.createExecChannel(ClientSession.java:183)
	at org.eclipse.jgit.transport.sshd.SshdSession.exec(SshdSession.java:136)
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254)
	... 11 more
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832128 is a reply to message #1832103] Wed, 09 September 2020 15:38 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
Does it still happen with EGit 5.9.0? (To be released 2020-09-16, but you can already use the 5.9.0 update site to install it in Eclipse.)

The log is not very helpful; strangely it doesn't show anything from Apache MINA sshd. (Also no timestamps and other things normally found in a log file.) How was that log generated?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832174 is a reply to message #1832128] Thu, 10 September 2020 21:58 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
I suspect the somewhat abrupt ssh channel closing in 5.8.1 might cause this. That was fixed in JGit 5.9.0 in bug 565854.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832348 is a reply to message #1832174] Wed, 16 September 2020 21:31 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
EGit 5.9.0 is now released officially; available for install from the update site or from Eclipse Marketplace.

Any news on this problem here? Does it still occur with EGit 5.9.0?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832627 is a reply to message #1832348] Tue, 22 September 2020 09:41 Go to previous messageGo to next message
Andreas Lück is currently offline Andreas LückFriend
Messages: 1
Registered: September 2020
Junior Member
Thomas Wolf wrote on Wed, 16 September 2020 21:31
EGit 5.9.0 is now released officially; available for install from the update site or from Eclipse Marketplace.

Any news on this problem here? Does it still occur with EGit 5.9.0?


We tried it with the same Eclipse release with updated EGit, and we tried it with the new Eclipse 2020-09. The issue still occurs. The person who updated EGit says that it "feels like" as if it occurs more rarely.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832689 is a reply to message #1832627] Wed, 23 September 2020 11:43 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
I think I really need a debug log showing the Apache MINA sshd logging. I don't see why the client-side session should be closed or closing at this point. You wouldn't have access to any server-side logs, would you?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832911 is a reply to message #1832689] Tue, 29 September 2020 07:35 Go to previous messageGo to next message
Corny de Souza is currently offline Corny de SouzaFriend
Messages: 3
Registered: September 2020
Junior Member
I doubt we will be able to get access to the log information, but I have submitted a request anyway just in case.

Is there any other way of debugging this?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832952 is a reply to message #1832911] Tue, 29 September 2020 21:42 Go to previous messageGo to next message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
Unfortunately getting a log from Eclipse with these Apache MINA sshd messages is indeed hard. With JGit, it should be possible in the way I wrote above, but if you can't reproduce with JGit, that won't help.

With Eclipse, you have to figure out which logging back-end, if any, is installed. My normal "Eclipse for Committers" apparently has none; so to get logging output I have to download one, for instance org.slf4j.impl.log4j12, and then copy that into my Eclipse's "dropins" folder. Then write a log4j configuration and pass that via a Java system property defined via the _JAVA_OPTIONS environment variable to Eclipse.

Here's a way to get DEBUG logging for Apache MINA sshd from the Eclipse for Java Developers package that you appear to be using. That one includes a logback logging back-end (via m2e), so write a config for logback:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="5 seconds">

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{40} - %msg%n</pattern>
        </encoder>
    </appender>

    <logger name="org.apache.sshd" level="DEBUG" additivity="false">
        <appender-ref ref="STDOUT" />
    </logger>

    <root level="WARN">
        <appender-ref ref="STDOUT" />
    </root>

</configuration>

Save it as file "logback-debug.xml" somewhere.

Then set the environment variable and start Eclipse from the command line. Then pull from your repo on Azure DevOps. You should get a lot of DEBUG logging in the terminal.

On my OS X machine, I just start
$ _JAVA_OPTIONS='-Dlogback.configurationFile=file:///absolute/path/to/logback-debug.xml' ./eclipse &


With some luck, such a log will perhaps show something that can give me a clue as to what might be going on.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832953 is a reply to message #1832952] Tue, 29 September 2020 21:45 Go to previous message
Thomas Wolf is currently offline Thomas WolfFriend
Messages: 406
Registered: August 2016
Senior Member
The most direct way to debug this is of course in the debugger. But Apache MINA sshd is highly asynchronous and difficult to debug by stepping through if you don't have a good idea of where to set breakpoints.
Previous Topic:pre-commit Hook "echo" message not appearing in popup
Next Topic:visit Ceph S3 with AmazonS3 class got 403 Error
Goto Forum:
  


Current Time: Sun Oct 24 02:29:51 GMT 2021

Powered by FUDForum. Page generated in 0.02683 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top