[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] [egit-dev] Git LFS

Thanks Markus. I will first try to support the native Git use case and then look at JGit builtin LFS support...

Am Do., 18. Apr. 2019 um 10:18ÂUhr schrieb Duft Markus <Markus.Duft@xxxxxxxxxxxxxxxx>:

Hey,


The summary sounds reasonable for me :)


If you are looking to implement HTTPS (auth) support for JGit LFS, you will want to start by looking atÂ/org.eclipse.jgit.lfs/src/org/eclipse/jgit/lfs/internal/LfsConnectionFactory.java - especially the methodÂgetLfsUrlâ will try to figure out everything required to connect - either by configuration (explicit URL set), or by discovery including SSH authentication if it is an SSH URL.


Cheers,

Markus


From: Bossert, Andre <anb0s@xxxxxxxx>
Sent: Thursday, April 18, 2019 10:03
To: Bossert, Andre
Cc: Duft Markus; JGit Developers list; EGit developer discussion (egit-dev@xxxxxxxxxxx); Andre Bossert
Subject: Re: [egit-dev] Git LFS
Â
Hello,

my investigations so far:
- native git hooks are not supported at Windows (without cygwin)
- Git for Windows (MSYS2 = cygwin fork) is not supported
- this explains whyÂpost-commit and pre-push actions are not executed
- this explains why "git lfs status" shows different output to native git and binary data is not transferred to LFS server
- there is no error message and no abort of push --> breaks consistency between LFS and Git server --> fetch in another clones shows missing LFS objects
- JGit builtin LFS works with embedded hooks, but push to HTTPS enabled server (Artifactory) is not supported

From my point of view LFS is not usable for us at Windows and there is no clear documentation howto use and what are the pitfalls.

I will try to adress the main points we need to proceed with JGit LFS.

If somebody has some more input to clarify and show how it can work, please write some hints. Thank You!

Regards,
Andre

Am Mi., 17. Apr. 2019 um 13:38ÂUhr schrieb Bossert, Andre <anb0s@xxxxxxxx>:
Hi,

Thanks :)

I've found that in the Mehod "org.eclipse.jgit.hooks.GitHook.doRun()" the call "org.eclipse.jgit.util.ProcessResult.isExecutedWithError()" returns false and the status is ProcessResult.Status.NOT_SUPPORTED.
So there is no error, exception and no log if hook was not executed :(
...
if (result.isExecutedWithError()) {
throw new AbortedByHookException(
new String(errorByteArray.toByteArray(), UTF_8),
getHookName(), result.getExitCode());
}
...

Why the hook is not supported, but Filters are supported is the question here. May be related how MSYS2 Git Bash at Windows is detected...

Regards,
Andre



Am Mi., 17. Apr. 2019 um 13:17ÂUhr schrieb Duft Markus <Markus.Duft@xxxxxxxxxxxxxxxx>:

Hi,


Hook handling is generic and should be used whenever useJGitBuiltin is not set...â We had situations where the native GIT LFS was used (and working) because we forgot to enable this :)


Unfortunately I don't know locations in the code from the top of my head...


Cheers,

Markus



From: Bossert, Andre <anb0s@xxxxxxxx>
Sent: Wednesday, April 17, 2019 11:28
To: Duft Markus
Cc: Andre Bossert; JGit Developers list; EGit developer discussion (egit-dev@xxxxxxxxxxx); Andre Bossert
Subject: Re: [egit-dev] Git LFS
Â
Hello Markus,

Thank You for the answer!

If HTTPS is not implemented or not working can we use native Git LFS with JGit? So then the pre-push hooks should etc. should just work... Do you have hints where to search, set breakpoints etc. to analyse why they are not executed...

Regards,
Andre

Duft Markus <Markus.Duft@xxxxxxxxxxxxxxxx> schrieb am Mi., 17. Apr. 2019, 11:18:

Hey,


I implemented parts of this...


We are using JGit LFS in production for some time now (~1Âyear) with thousands of users.


We are using authentication through SSH (using Gerrit and the Gerrit LFS plugin, which supports obtaining authentication tokens using 'ssh server git-lfs-authenticate project').


If nothing changed in the meantime, I guess HTTP(S) authentication is simply not supported (yet). Matthias did some tests against GitHub back then, and AFAIR they were successful, but I don't know what was the exact setup for the tests...


HTH,ÂCheers,

Markus




From: egit-dev-bounces@xxxxxxxxxxx <egit-dev-bounces@xxxxxxxxxxx> on behalf of Bossert, Andre <anb0s@xxxxxxxx>
Sent: Wednesday, April 17, 2019 10:31
To: JGit Developers list (jgit-dev@xxxxxxxxxxx); EGit developer discussion (egit-dev@xxxxxxxxxxx)
Cc: Andre Bossert
Subject: Re: [egit-dev] Git LFS
Â
I've found the option "lfs.useJGitBuiltin = true" that enables builtin LFS support. The pre-push hooks shows an error / exception, because of the autorization
...
!STACK 0
org.eclipse.jgit.errors.TransportException: When trying to open a connection to https://[server]/artifactory/[project]/objects/batch the server responded with an error code. rc=401
at org.eclipse.jgit.transport.Transport.push(Transport.java:1365)
at org.eclipse.egit.core.op.PushOperation.run(PushOperation.java:191)
at org.eclipse.egit.ui.internal.push.ConfirmationPage$2.run(ConfirmationPage.java:211)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:122)
Caused by: java.io.IOException: When trying to open a connection to https://[server]/artifactory/[project]/objects/batchÂÂthe server responded with an error code. rc=401
at org.eclipse.jgit.lfs.LfsPrePushHook.requestBatchUpload(LfsPrePushHook.java:214)
at org.eclipse.jgit.lfs.LfsPrePushHook.call(LfsPrePushHook.java:123)
at org.eclipse.jgit.transport.Transport.push(Transport.java:1363)
...

There is no dialog to enter the username and password. How can we enable this?

I've found only this information about LFS:
...

Eclipse JGit: Java implementation of Git 4.2

Support git-lfs extension

  • Add support for pre-push hooks

  • Add support for smudge filters.ÂIf defined in .gitattributes call smudge filter during checkout.
  • Add support for clean filters.ÂIf defined in .gitattributes call clean filter during add to index.
...

The pre-push hook is not executed in the case native git lfs is used, but it looks like the filters are executed... Is there any deeper information about JGit LFS ?

Am Di., 16. Apr. 2019 um 10:30ÂUhr schrieb Bossert, Andre <anb0s@xxxxxxxx>:
Hello,

we are using native Git with LFS support. The LFS files are pushed to an Artifactory server (https).
Filters (smudge and clean) and Hooks (post-checkout, pre-push etc.) are executed.

Now i'm trying to use Eclipse EGIt / JGit and it looks different:

- after a binary file (LFS tracked) is added to staging area "git lfs status" shows it in "Git LFS objects to be committed:" and "Git LFS objects not staged for commit:", but expected and seen in native Git it should be in "Git LFS objects to be committed:" only.

- push does not send the file to Artifactory, but there is no exception / error etc. The "oid sha256" from the git pointer file cannot be found at Artifactory server.

- it's also not clear for us if native git lfs is used or the embedded JGit LFS - are both ways still supprted? If yes how can we choose?

How can we analyse the issues? Is there any deep documentation about how it works or some FAQ where to search or enable some debug log etc.?

Thanks and Regards,
Andre



--
Regards
Andre (anb0s)

SSI SchÃfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht fÃr Zivilrechtssachen Graz

SSI SchÃfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht fÃr Zivilrechtssachen Graz


--
Regards
Andre (anb0s)


--
Regards
Andre (anb0s)

SSI SchÃfer IT Solutions GmbH | Friesachstrasse 15 | 8114 Friesach | Austria
Registered Office: Friesach | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht fÃr Zivilrechtssachen Graz


--
Regards
Andre (anb0s)
eMail: anb0s@xxxxxxxx