Eclipse Community Forums: EGit / JGit » MissingObjectException after NAS failover

Help

Home

Home » Eclipse Projects » EGit / JGit » MissingObjectException after NAS failover

Show: Today's Messages :: Show Polls :: Message Navigator

MissingObjectException after NAS failover [message #1697474]

Thu, 04 June 2015 09:15

Gerard Soldevila

Messages: 4
Registered: June 2015

Junior Member

We have a system that uses GIT-based file storage in a NAS through CIFS, accessed through JGIT from a Java Application Server (crazy, I know)

To speed up things, we keep a reference to the "Repository" JGIT object, so that to access a given file we just do:

Repository r = ... // already existing instance in memory
ObjectLoader loader = r.open("<given_file_hash>");
...
InputStream is = loader.getInputStream();
Long size = loader.getSize();
...
repo.close();

The problem occurs upon NAS failover (some of the physical disks replaced).
This operation should be transparent from the Application standpoint, but after it happens, we start having errors of the form:

Quote:

org.eclipse.jgit.errors.MissingObjectException: Missing unknown 1bf28962a9e3495b57a4eb78b7fd2ba7f29783d5
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:148)
at org.eclipse.jgit.lib.ObjectDatabase.open(ObjectDatabase.java:177)
at org.eclipse.jgit.lib.ObjectDatabase.open(ObjectDatabase.java:147)
at org.eclipse.jgit.lib.Repository.open(Repository.java:250)

The reportedly missing unkown objects actually exist in the NAS (I checked manually through command line).
When we drop our reference to "Repository" object and we re-instantiate it, things get back to normal.

I would like to know if someone knows/can tell me why this is happening

Report message to a moderator

Re: MissingObjectException after NAS failover [message #1713208 is a reply to message #1697474]

Mon, 02 November 2015 11:40

Junyi Bei

Messages: 4
Registered: November 2015

Junior Member

It happened to me a few times. Maybe the object is partially corrupted?

When you open the file with command line, does it show the whole file correctly?

Report message to a moderator

Re: MissingObjectException after NAS failover [message #1713253 is a reply to message #1713208]

Mon, 02 November 2015 19:45

Matthias Sohn

Messages: 1268
Registered: July 2009

Senior Member

When JGit hits an exception when accessing a pack file which it doesn't recognize and handle explicitly
it may decide to consider this pack corrupt and hence remove it from the pack list.
This may result in MissingObjectExceptions which should disappear if you recreate the Repository object.

Report message to a moderator

Re: MissingObjectException after NAS failover [message #1714539 is a reply to message #1697474]

Fri, 13 November 2015 13:15

Gerard Soldevila

Messages: 4
Registered: June 2015

Junior Member

@Junyi: Yes, from command line the whole file is displayed.
Also, after restarting the application server ... the file is correctly retrieved again.

@Matthias: I don't think we have any corrupt pack. Could it be a problem if we are simultaneously:

- adding files in GIT
- retrieving them with the aforementioned logic

Report message to a moderator

Re: MissingObjectException after NAS failover [message #1715066 is a reply to message #1714539]

Wed, 18 November 2015 22:49

Matthias Sohn

Messages: 1268
Registered: July 2009

Senior Member

As I said this can happen if JGit happens to see an unexpected exception when accessing packs which
it doesn't know and then it may erroneously assume the pack is corrupt and it will then remove it from
the pack list. If you could find out if there is such an exception we could improve JGit to handle also
your case.

See http://git.eclipse.org/c/jgit/jgit.git/tree/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/file/ObjectDirectory.java#n556

Report message to a moderator

Re: MissingObjectException after NAS failover [message #1715104 is a reply to message #1715066]

Thu, 19 November 2015 09:32

Gerard Soldevila

Messages: 4
Registered: June 2015

Junior Member

Thanks! Now I understood your answer a bit better.

Let us imagine the unexpected exception is an IOException due to filesystem access (network problem or contention problem).

Say one of the packs is removed from the pack list (erroneously considered corrupt). Is there something we can do to reset / rebuild this pack list?

Note that I upon detection of the "MissingObjectException" I am already trying to rebuild the Repository object, by doing:

Git.open(path).getRepository()

But so far the only solution seems to be JVM restart.

For information, we are using
org.eclipse.jgit-3.5.3.201412180710-r

Report message to a moderator

Re: MissingObjectException after NAS failover [message #1716757 is a reply to message #1715104]

Tue, 08 December 2015 00:02

Matthias Sohn

Messages: 1268
Registered: July 2009

Senior Member

Did you close the repository so that its useCnt is 0 (Repository uses refcouting) ?
Did you unregister the repository from the RepositoryCache before trying to recreate the Repository instance ?

Report message to a moderator