Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » [Solved] Clone submodules in JGit(Prerequisites? Add submodules to existing clone?)
[Solved] Clone submodules in JGit [message #1684662] Wed, 18 March 2015 14:24 Go to next message
Julian Enoch is currently offline Julian EnochFriend
Messages: 24
Registered: January 2014
Junior Member
Hi,

I am using JGit to clone, and I am trying to include the submodules. This is a snippet:

...
cloneCommand.setNoCheckout(true);
cloneCommand.setCloneSubmodules(true);
cloneCommand.call();

But the submodules weren't cloned. So I would like to ask about the prerequisites for submodules to be cloned. Does it have to be necessarily after the checkout?

also is there a simple way to add all submodules recursively to an existing clone?

Thanks,

[Updated on: Thu, 21 May 2015 15:50]

Report message to a moderator

Re: Clone submodules in JGit [message #1684801 is a reply to message #1684662] Wed, 18 March 2015 15:34 Go to previous messageGo to next message
Rüdiger Herrmann is currently offline Rüdiger HerrmannFriend
Messages: 581
Registered: July 2009
Senior Member
Julian,

AFAIK, to clone submodules, a repository must have a working directory.
Hence the setNoCheckout() call conflicts with setCloneSubmodules(). JGit
should probably _somehow_ flag that combining these options isn't possible.

To add submodules to an existing repository, the SubmoduleUpdateCommand
can be used. It will update all registered submodules, clone missing
submodules and checkout the commit specified in the configuration.

For more details you may want to read the summary [1] I wrote some time
ago that explains what submodule commands JGit offers and what they can
be used for.

HTH
Rüdiger

[1]
http://www.codeaffine.com/2014/04/16/how-to-manage-git-submodules-with-jgit/

On 18.03.2015 15:24, Julian Enoch wrote:
> Hi,
>
> I am using JGit to clone, and I am trying to include the submodules.
> This is a snippet:
>
> ...
> cloneCommand.setNoCheckout(true);
> cloneCommand.setCloneSubmodules(true);
> cloneCommand.call();
>
> But the submodules weren't cloned. So I would like to ask about the
> prerequisites for submodules to be cloned. Does it have to be
> necessarily after the checkout?
>
> also is there a simple way to add all submodules recursively to an
> existing clone?
>
> Thanks,
Re: Clone submodules in JGit [message #1688392 is a reply to message #1684801] Fri, 20 March 2015 15:54 Go to previous messageGo to next message
Julian Enoch is currently offline Julian EnochFriend
Messages: 24
Registered: January 2014
Junior Member
Thank you Rüdiger,

Using SubmoduleUpdateCommand in couple with SubmoduleInitCommand has worked for one of my submodule. However with the second one I am dealing with a strange exception:

org.eclipse.jgit.api.errors.JGitInternalException: Creating directories for /path/directory failed
at org.eclipse.jgit.api.SubmoduleUpdateCommand.call(SubmoduleUpdateCommand.java:210)
...
Caused by: java.io.IOException: Creating directories for /path/directory failed
at org.eclipse.jgit.util.FileUtils.mkdirs(FileUtils.java:316)
at org.eclipse.jgit.dircache.DirCacheCheckout.checkoutEntry(DirCacheCheckout.java:1225)
at org.eclipse.jgit.dircache.DirCacheCheckout.doCheckout(DirCacheCheckout.java:451)
at org.eclipse.jgit.dircache.DirCacheCheckout.checkout(DirCacheCheckout.java:396)
at org.eclipse.jgit.api.SubmoduleUpdateCommand.call(SubmoduleUpdateCommand.java:197)
... 14 more

After some debugging, I see that when FileUtils.mkdirs tries to create directory, the latter is present in /path/ and is a broken Link!!! I don't know where it comes from. Do you have any idea??

If I clone everything using git CLI and the option --recursive, everything goes well and the directory in question is just a normal plain folder and not a Link.
Re: Clone submodules in JGit [message #1688756 is a reply to message #1688392] Sat, 21 March 2015 16:38 Go to previous messageGo to next message
Rüdiger Herrmann is currently offline Rüdiger HerrmannFriend
Messages: 581
Registered: July 2009
Senior Member
Julian,

I remember that in the past, JGit did not support symbolic links [1].
This has changed but AFAIK still depends on which version of Java and
which OS you run on. Which OS/JGit/Java do you use?

If you look into the repository that was cloned with native Git, what
exactly is /path/directory?

Regards,
Rüdiger

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=354367

On 20.03.2015 16:54, Julian Enoch wrote:
> Thank you Rüdiger,
>
> Using SubmoduleUpdateCommand in couple with SubmoduleInitCommand has
> worked for one of my submodule. However with the second one I am dealing
> with a strange exception:
>
> org.eclipse.jgit.api.errors.JGitInternalException: Creating directories
> for /path/directory failed
> at
> org.eclipse.jgit.api.SubmoduleUpdateCommand.call(SubmoduleUpdateCommand.java:210)
>
> ...
> Caused by: java.io.IOException: Creating directories for /path/directory
> failed
> at org.eclipse.jgit.util.FileUtils.mkdirs(FileUtils.java:316)
> at
> org.eclipse.jgit.dircache.DirCacheCheckout.checkoutEntry(DirCacheCheckout.java:1225)
>
> at
> org.eclipse.jgit.dircache.DirCacheCheckout.doCheckout(DirCacheCheckout.java:451)
>
> at
> org.eclipse.jgit.dircache.DirCacheCheckout.checkout(DirCacheCheckout.java:396)
>
> at
> org.eclipse.jgit.api.SubmoduleUpdateCommand.call(SubmoduleUpdateCommand.java:197)
>
> ... 14 more
>
> After some debugging, I see that when FileUtils.mkdirs tries to create
> directory, the latter is present in /path/ and is a broken Link!!! I
> don't know where it comes from. Do you have any idea??
>
> If I clone everything using git CLI and the option --recursive,
> everything goes well and the directory in question is just a normal
> plain folder and not a Link.
Re: Clone submodules in JGit [message #1689513 is a reply to message #1688756] Mon, 23 March 2015 19:27 Go to previous messageGo to next message
Julian Enoch is currently offline Julian EnochFriend
Messages: 24
Registered: January 2014
Junior Member
Quote:
I remember that in the past, JGit did not support symbolic links [1].


It seems that this is related. In fact at some point the target of the symlink is deleted and the symlink is dangling, which causes the problem.

Quote:
This has changed but AFAIK still depends on which version of Java and
which OS you run on. Which OS/JGit/Java do you use?


os:
Red Hat Enterprise Linux Server release 6.4 (Santiago)
2.6.32-358.6.2.el6.x86_64

JGit:
org.eclipse.jgit (3.7.1.201502270113) "JGit Core"

Java:
java-1.7.0-openjdk-1.7.0.19.x86_64

Quote:
If you look into the repository that was cloned with native Git, what
exactly is /path/directory?


I am not sure I understand the point of this question. /path/directory is just a plain location path.

I am not a specialist but this is my idea of what is going on. This is complicated so hang on with me:


    On the commit 5f0a7db, we have a plain folder at location1
    On the master branch, the folder has been moved to location2 and a symlink was created instead at location1


Here comes JGit:


    If I clone the plain Git repository, the branch master is checked out and everything is fine

    But If I clone it as a submodule of another repository, using JGit submodule commands, here is what happens:



    JGit first clones the branch master: so in the working tree, we have a target folder at location2 and a symlink at location1

    Then JGit tries to checkout the commit 5f0a7db (I don't know why or if this is normal). So, it deletes the target folder at location2. This results in a dangling link at location1. Then it tries to create the folder in location1. which results in an exception because of the dangling link as you can see from this code:

    package org.eclipse.jgit.util;
    ...
    public class FileUtils {
    ...
    	public static void mkdirs(final File d, boolean skipExisting) throws IOException {
    		if (!d.mkdirs()) {
    			if (skipExisting && d.isDirectory()) return;
    			throw new IOException(MessageFormat.format(JGitText.get().mkDirsFailed, d.getAbsolutePath()));
    		}
    	}



In conclusion, I think that JGit should not fail because it couldn't create the folder. Instead it should be able to delete the broken link.

Please let me know your views.

Thanks,
Re: Clone submodules in JGit [message #1690066 is a reply to message #1689513] Tue, 24 March 2015 22:51 Go to previous messageGo to next message
Matthias Sohn is currently offline Matthias SohnFriend
Messages: 1268
Registered: July 2009
Senior Member
If you are using symlinks and JGit 3.7 you also need to have the org.eclipse.jgit.java7 bundle on the
class path otherwise there's no support for symlinks. Alternatively you can update to latest
4.0.0-SNAPSHOT where the java7 bundle has been merged into org.eclipse.jgit since we removed
support for Java 5 and 6.

If you can reproduce the problem with either the Java 7 bundle and running with Java 7 or higher
on a platform supporting symlinks then file a bug in Bugzilla describing the steps to reproduce the
problem.
Re: Clone submodules in JGit [message #1696090 is a reply to message #1690066] Thu, 21 May 2015 15:49 Go to previous message
Julian Enoch is currently offline Julian EnochFriend
Messages: 24
Registered: January 2014
Junior Member
Problem of symlinks gone away using JGit 3.7 and org.eclipse.jgit.java7
Thanks,
Previous Topic:Diff/Comparison from history for moved files does not work
Next Topic:Duplicate RevCommits in RevWalk
Goto Forum:
  


Current Time: Thu Apr 25 08:16:19 GMT 2024

Powered by FUDForum. Page generated in 0.03831 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top