Skip to main content


Eclipse Community Forums
Forum Search:

Search      Help    Register    Login    Home
Home » Eclipse Projects » EGit / JGit » Is there a way to use the Object Inserter and have it show up in the File system
Is there a way to use the Object Inserter and have it show up in the File system [message #1842450] Sun, 20 June 2021 00:08 Go to next message
Heavens Regent is currently offline Heavens RegentFriend
Messages: 7
Registered: May 2021
Junior Member
Update: Turns out the issue was somewhere else, I assumed since I wasn't seeing the files in the git repository it was sticking them somewhere random. However, is the main question is still valid. Is there a way to make the files show up in the file system while using ObjectInserter (as seen below)

------

Is there some way to use the ObjectInserter and save the files to a certain directory on the filesystem? When I've used ObjectInserter (see code below) I can't get it to save to a certain directory.

I don't care if I can see the actual files on the file system, but I need the files to be saved to a specific directory. I've only been able to do that with the second set of code. The first set of code runs much faster especially the retrieve file object part.

This code is working really fast but I can't find the generated files on the file system
Save Object w/ObjectInserter
        ObjectInserter objectInserter = repository.newObjectInserter();
        ObjectId blobId = objectInserter.insert(Constants.OBJ_BLOB, fileContents.getBytes());
        objectInserter.flush();

        TreeFormatter treeFormatter = new TreeFormatter();
        treeFormatter.append(fileName, FileMode.REGULAR_FILE, blobId);
        ObjectId treeId = objectInserter.insert(treeFormatter);
        objectInserter.flush();

        TreeWalk treeWalk = new TreeWalk(repository);
        treeWalk.addTree(treeId);
        treeWalk.next();

Retrieve object w/ObjectReader
        ObjectReader objectReader = git.getRepository().newObjectReader();
        ObjectLoader objectLoader = objectReader.open(blobId);
        return new String(objectLoader.getBytes());


This code is a lot slower, but I am able to find the files on the file system
Save Object w/Java FileWriter
        File myFile = new File(repository.getDirectory().getParent(), fileName);
        FileWriter myWriter = new FileWriter(filePath);
        myWriter.write(fileContents);
        myWriter.close();
        myFile.createNewFile();

Read object with Scanner
        File myObj = new File("/jgit/" + repoName + "/" + fileName);
        Scanner fileReader = new Scanner(myObj);
        String fileData = "";
        while (fileReader.hasNextLine()) {
            fileData += fileReader.nextLine();
        }

[Updated on: Sun, 20 June 2021 00:26]

Report message to a moderator

Re: Is there a way to use the Object Inserter and have it show up in the File system [message #1842467 is a reply to message #1842450] Mon, 21 June 2021 06:27 Go to previous messageGo to next message
Christian Halstrick is currently offline Christian HalstrickFriend
Messages: 274
Registered: July 2009
Senior Member
Hi,
the ObjectInserter will modify the object database of a git repositiory. This db is persisted under <repo>/.git/objects/... . So when you use this class you will see that files change in this folder.
The folder you inspect and where you expect files to pop up is the working tree. The working tree is updated by [j]git when you do checkout (or reset) operations. What you could do is:
Add objects to the object database using objectinserter and afterwards checkout the tree you have created.

You have the same layers in native git. See https://git-scm.com/book/en/v2/Git-Internals-Git-Objects : using ObjectInserter would be similar to the calls to 'git hash-object ...'. And similar to JGit 'git hash-object ...' will not update the working tree but only .../.git/objects/...


Ciao
Chris
Re: Is there a way to use the Object Inserter and have it show up in the File system [message #1842737 is a reply to message #1842467] Wed, 30 June 2021 08:42 Go to previous messageGo to next message
Heavens Regent is currently offline Heavens RegentFriend
Messages: 7
Registered: May 2021
Junior Member
Somehow I'm losing access to the objects I am inserting with ObjectInserter. I'm storing the blobId returned from objectInserter.insert(...) in the database.

It seems like I'm either doing something that removes the objects from the .git/objects/ or somehow I'm not saving the objects to the repository.

Are there any basic git commands that would overwrite the stuff there? For example:

git.add().addFilepattern(fileName).call();


On the other side, this is how I retrieve the git repository. Is there something about the way I'm doing this that doesn't actually reference this repository?
The reason this is important is that directory is the only safe directory on the system

Git git = Git.open(new File("/jgit/repoName/"))
Repository repository = git.getRepository();

[Updated on: Wed, 30 June 2021 08:54]

Report message to a moderator

Re: Is there a way to use the Object Inserter and have it show up in the File system [message #1842740 is a reply to message #1842737] Wed, 30 June 2021 09:20 Go to previous messageGo to next message
Christian Halstrick is currently offline Christian HalstrickFriend
Messages: 274
Registered: July 2009
Senior Member
You are going a not so often used way when adding content to the repo directly with ObjectInserter instead of using the standard way of writing new content to the worktree, adding and committing it. What you are trying to do is more following: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects instead of using slow git high level commands. Nevertheless: Your way will be faster and of course it should work with JGit.

The files which you write with ObjectInserter are never updated. If you have a file to version with the content 'test content' then it will be stored in the object database in a file called .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4. That's because the SHA1 of 'test content' happens to be d670460b4b4aece5915caf5c68d12f560a9fe3e4. When you modify the file content to 'hello world' then that new content will be stored in the object database in a file called .git/objects/3b/18e512dba79e4c8300dd08aeb37f8e728b8dad (beacause the sha1 of 'hello world' is 3b18... ) . So: it is very unlikely that somebody wants to write something different than 'test content' in a file named .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4.
But what can easily happen, and I guess that happens to you is that git itself deletes the files because it thinks that they are not needed anymore. If you store something in the obectDatabase then you have to take care that references (branches, tags) are pointing to that new content. Otherwise git will do a garbage collection and blobs
not referenced by any refs are thrown away. Means the files underneath the .git/objects folder corresponding to these blobs will be deleted.

The code in https://git-scm.com/book/en/v2/Git-Internals-Git-Objects does all that. It creates blobs, trees, commits, refs without using the working tree with low level commands.


Ciao
Chris
Re: Is there a way to use the Object Inserter and have it show up in the File system [message #1842741 is a reply to message #1842740] Wed, 30 June 2021 09:37 Go to previous messageGo to next message
Heavens Regent is currently offline Heavens RegentFriend
Messages: 7
Registered: May 2021
Junior Member
If I retrieve a file, make updates to the file, then save that file, does that mean that file v2 has no correlation to file v1? also meaning the blobId will be different?

Edit:
Also doesn't the below code do what you are saying I need to do?
I apologize if I seem dense, I've read through that page about many times over the course of the last 2 months and I'm just barely beginning to understand the git plumbing commands and how that relates to what I'm doing in JGit
        TreeFormatter treeFormatter = new TreeFormatter();
        treeFormatter.append(fileName, FileMode.REGULAR_FILE, blobId);
        ObjectId treeId = objectInserter.insert(treeFormatter);
        objectInserter.flush();

        TreeWalk treeWalk = new TreeWalk(repository);
        treeWalk.addTree(treeId);
        treeWalk.next();

[Updated on: Wed, 30 June 2021 09:51]

Report message to a moderator

Re: Is there a way to use the Object Inserter and have it show up in the File system [message #1842750 is a reply to message #1842741] Wed, 30 June 2021 13:16 Go to previous message
Christian Halstrick is currently offline Christian HalstrickFriend
Messages: 274
Registered: July 2009
Senior Member
That's true: If you version a file 'f.txt' containing the content 'test content' then the text 'test content' will be stored in a blob with name .git/objects/d6/70460b4b4aece5915caf5c68d12f560a9fe3e4. When you then modify the file f.txt and change the content to 'hello world' and add and commit that, then there will be a new blob persisted in .git/objects/3b/18e512dba79e4c8300dd08aeb37f8e728b8dad containing content 'hello world'. And from the blobs you can't detect that they both belong to 'f.txt' and one is the successor content of the same versioned file. If you afterwards add and commit a new file 'g.txt' with the content 'hello world' then ... no new blobs will be written to the object database. We will have a new tree object saying that the root folder has now two entries (f.txt, g.txt) and that both files have content stored in .git/objects/3b/18e512dba79e4c8300dd08aeb37f8e728b8dad. Two completely different files with different pathes do share the same content and therefore there is only one blob persisted. It's the tree object which finally maps filenames to blob objects.

Regarding whether your code is correct: No, you care about creating trees and blobs ... but both are not yet referenced by any ref (commit, tag). Means, after your code is executed and you wait long enough then git may run a garbage collection and will find out that neither the tree nor the blobs are reachable by any ref. Therefore they are garbage and can be deleted. You need to create a commit referencing you tree (and by that the blob) and additionally you need a ref which points to your commit. Otherwise the commit is also garbage and therfore tree and blob also.

I would suggest you start with a working tree based version of your code which uses high level commands like add, commit, checkout and inspect how the object database changes. Then, if this works, start optimizing by getting rid of the need to write content to the working tree.


Ciao
Chris
Previous Topic:Having Not Responding messages while using Git Staging view
Next Topic:PostCommitHook
Goto Forum:
  


Current Time: Fri Apr 19 22:27:12 GMT 2024

Powered by FUDForum. Page generated in 0.03300 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.2.
Copyright ©2001-2010 FUDforum Bulletin Board Software

Back to the top