Skip to main content



      Home
Home » Eclipse Projects » EGit / JGit » Git clone/pull with SSH from Azure DevOps repositories fails frequently(Error "remote end hung up unexpectedly")
Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1831933] Thu, 03 September 2020 04:21 Go to next message
Eclipse UserFriend
Hello,

we recently moved our git repositories to Azure DevOps and since then we are having issues accessing them with git clone/pull.

Error "remote end hung up unexpectedly"

This error usually occurs on the first clone/pull on a specific repository. If the clone/pull is executed immediately after the failure, then the clone/pull usually succeeds.

If the SSH client is changed to JSch then the problem also occurs with the error "session is down"

If we use the plain old command line git executable (2.17.1.windows.2) then everything works as expected.

Eclipse Version: Eclipse 2020-06
EGit/JGit Version: 5.8.0
Backend: Apache MINA

Any help provided would be much appreciated.

Thanks
Corny
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832031 is a reply to message #1831933] Sun, 06 September 2020 12:33 Go to previous messageGo to next message
Eclipse UserFriend
Could you try this with the JGit command-line client with debug logging enabled? See bug 545920 comment 5 for how to do that. (But use the latest JGit command-line executable.) Without any logs I can't even begin to formulate any hypothesis.

The JGit command-line executable is a Unix shell script; I suppose it should be possible to run it in the git shell on Windows.

Also helpful for a comparison would be a log from git-for-windows doing the clone or pull. You should be able to capture such a log by setting environment variable GIT_SSH_COMMAND or git config core.sshCommand to ssh -vvv.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832032 is a reply to message #1832031] Sun, 06 September 2020 12:41 Go to previous messageGo to next message
Eclipse UserFriend
Also: is there anything in the Eclipse logs? "remote end hung up unexpectedly" should log an exception with a stack trace; maybe that gives some more hints.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832103 is a reply to message #1832032] Wed, 09 September 2020 02:25 Go to previous messageGo to next message
Eclipse UserFriend
We could not reproduce the error with the JGit command-line client, log is however attached.

The error persists with Eclipse though.

Eclipse Log for Clone:
eclipse.buildId=4.16.0.I20200604-0540
java.version=1.8.0_202
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
Framework arguments:  -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -data D:\Development\LokalWS\OxygenWS_master -product org.eclipse.epp.package.java.product

org.eclipse.egit.ui
Error
Tue Sep 08 11:07:08 CEST 2020
Git repository clone failed.

org.eclipse.jgit.api.errors.TransportException: git@ssh.dev.azure.com:v3/storeit/dopix-products/dopix.core: remote hung up unexpectedly
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222)
	at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:302)
	at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:178)
	at org.eclipse.egit.core.op.CloneOperation.run(CloneOperation.java:194)
	at org.eclipse.egit.ui.internal.clone.AbstractGitCloneWizard.executeCloneOperation(AbstractGitCloneWizard.java:489)
	at org.eclipse.egit.ui.internal.clone.AbstractGitCloneWizard.access$2(AbstractGitCloneWizard.java:482)
	at org.eclipse.egit.ui.internal.clone.AbstractGitCloneWizard$5.run(AbstractGitCloneWizard.java:443)
	at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)
Caused by: org.eclipse.jgit.errors.TransportException: git@ssh.dev.azure.com:v3/storeit/dopix-products/dopix.core: remote hung up unexpectedly
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:270)
	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144)
	at org.eclipse.jgit.transport.FetchProcess.reopenConnection(FetchProcess.java:256)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:156)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211)
	... 7 more
Caused by: java.lang.IllegalStateException: Channel id=1 not registered because session is being closed: ClientConnectionService[JGitClientSession[git@ssh.dev.azure.com/52.236.147.103:22]]
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.handleChannelRegistrationFailure(AbstractConnectionService.java:437)
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.registerChannel(AbstractConnectionService.java:427)
	at org.apache.sshd.client.session.AbstractClientSession.createExecChannel(AbstractClientSession.java:330)
	at org.apache.sshd.client.session.ClientSession.createExecChannel(ClientSession.java:183)
	at org.eclipse.jgit.transport.sshd.SshdSession.exec(SshdSession.java:136)
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254)
	... 13 more


Eclipse Log for Fetch
eclipse.buildId=4.16.0.I20200604-0540
java.version=1.8.0_202
java.vendor=Oracle Corporation
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE
Framework arguments:  -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -product org.eclipse.epp.package.java.product -product org.eclipse.epp.package.java.product -data D:\Development\LokalWS\OxygenWS_master -product org.eclipse.epp.package.java.product

org.eclipse.egit.ui
Error
Tue Sep 08 11:12:48 CEST 2020
ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly

org.eclipse.core.runtime.CoreException: ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly
	at org.eclipse.egit.ui.internal.fetch.FetchOperationUI.execute(FetchOperationUI.java:113)
	at org.eclipse.egit.ui.internal.fetch.FetchOperationUI$1.performJob(FetchOperationUI.java:136)
	at org.eclipse.egit.ui.internal.jobs.RepositoryJob.run(RepositoryJob.java:59)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:222)
	at org.eclipse.egit.core.op.FetchOperation.run(FetchOperation.java:134)
	at org.eclipse.egit.ui.internal.fetch.FetchOperationUI.execute(FetchOperationUI.java:110)
	... 3 more
Caused by: org.eclipse.jgit.errors.TransportException: ssh://git@ssh.dev.azure.com/v3/storeit/dopix-products/dopix.product: remote hung up unexpectedly
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:270)
	at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:144)
	at org.eclipse.jgit.transport.FetchProcess.reopenConnection(FetchProcess.java:256)
	at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:156)
	at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:91)
	at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1260)
	at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:211)
	... 5 more
Caused by: java.lang.IllegalStateException: Channel id=1 not registered because session is being closed: ClientConnectionService[JGitClientSession[git@ssh.dev.azure.com/40.74.28.0:22]]
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.handleChannelRegistrationFailure(AbstractConnectionService.java:437)
	at org.apache.sshd.common.session.helpers.AbstractConnectionService.registerChannel(AbstractConnectionService.java:427)
	at org.apache.sshd.client.session.AbstractClientSession.createExecChannel(AbstractClientSession.java:330)
	at org.apache.sshd.client.session.ClientSession.createExecChannel(ClientSession.java:183)
	at org.eclipse.jgit.transport.sshd.SshdSession.exec(SshdSession.java:136)
	at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:254)
	... 11 more
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832128 is a reply to message #1832103] Wed, 09 September 2020 11:38 Go to previous messageGo to next message
Eclipse UserFriend
Does it still happen with EGit 5.9.0? (To be released 2020-09-16, but you can already use the 5.9.0 update site to install it in Eclipse.)

The log is not very helpful; strangely it doesn't show anything from Apache MINA sshd. (Also no timestamps and other things normally found in a log file.) How was that log generated?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832174 is a reply to message #1832128] Thu, 10 September 2020 17:58 Go to previous messageGo to next message
Eclipse UserFriend
I suspect the somewhat abrupt ssh channel closing in 5.8.1 might cause this. That was fixed in JGit 5.9.0 in bug 565854.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832348 is a reply to message #1832174] Wed, 16 September 2020 17:31 Go to previous messageGo to next message
Eclipse UserFriend
EGit 5.9.0 is now released officially; available for install from the update site or from Eclipse Marketplace.

Any news on this problem here? Does it still occur with EGit 5.9.0?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832627 is a reply to message #1832348] Tue, 22 September 2020 05:41 Go to previous messageGo to next message
Eclipse UserFriend
Thomas Wolf wrote on Wed, 16 September 2020 21:31
EGit 5.9.0 is now released officially; available for install from the update site or from Eclipse Marketplace.

Any news on this problem here? Does it still occur with EGit 5.9.0?


We tried it with the same Eclipse release with updated EGit, and we tried it with the new Eclipse 2020-09. The issue still occurs. The person who updated EGit says that it "feels like" as if it occurs more rarely.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832689 is a reply to message #1832627] Wed, 23 September 2020 07:43 Go to previous messageGo to next message
Eclipse UserFriend
I think I really need a debug log showing the Apache MINA sshd logging. I don't see why the client-side session should be closed or closing at this point. You wouldn't have access to any server-side logs, would you?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832911 is a reply to message #1832689] Tue, 29 September 2020 03:35 Go to previous messageGo to next message
Eclipse UserFriend
I doubt we will be able to get access to the log information, but I have submitted a request anyway just in case.

Is there any other way of debugging this?
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832952 is a reply to message #1832911] Tue, 29 September 2020 17:42 Go to previous messageGo to next message
Eclipse UserFriend
Unfortunately getting a log from Eclipse with these Apache MINA sshd messages is indeed hard. With JGit, it should be possible in the way I wrote above, but if you can't reproduce with JGit, that won't help.

With Eclipse, you have to figure out which logging back-end, if any, is installed. My normal "Eclipse for Committers" apparently has none; so to get logging output I have to download one, for instance org.slf4j.impl.log4j12, and then copy that into my Eclipse's "dropins" folder. Then write a log4j configuration and pass that via a Java system property defined via the _JAVA_OPTIONS environment variable to Eclipse.

Here's a way to get DEBUG logging for Apache MINA sshd from the Eclipse for Java Developers package that you appear to be using. That one includes a logback logging back-end (via m2e), so write a config for logback:
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="5 seconds">

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{40} - %msg%n</pattern>
        </encoder>
    </appender>

    <logger name="org.apache.sshd" level="DEBUG" additivity="false">
        <appender-ref ref="STDOUT" />
    </logger>

    <root level="WARN">
        <appender-ref ref="STDOUT" />
    </root>

</configuration>

Save it as file "logback-debug.xml" somewhere.

Then set the environment variable and start Eclipse from the command line. Then pull from your repo on Azure DevOps. You should get a lot of DEBUG logging in the terminal.

On my OS X machine, I just start
$ _JAVA_OPTIONS='-Dlogback.configurationFile=file:///absolute/path/to/logback-debug.xml' ./eclipse &


With some luck, such a log will perhaps show something that can give me a clue as to what might be going on.
Re: Git clone/pull with SSH from Azure DevOps repositories fails frequently [message #1832953 is a reply to message #1832952] Tue, 29 September 2020 17:45 Go to previous message
Eclipse UserFriend
The most direct way to debug this is of course in the debugger. But Apache MINA sshd is highly asynchronous and difficult to debug by stepping through if you don't have a good idea of where to set breakpoints.
Previous Topic:pre-commit Hook "echo" message not appearing in popup
Next Topic:visit Ceph S3 with AmazonS3 class got 403 Error
Goto Forum:
  


Current Time: Thu Jun 12 08:14:36 EDT 2025

Powered by FUDForum. Page generated in 0.78119 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top