Home » Eclipse Projects » EGit / JGit » Eclipse JGit(Different SHA1Id generated for Windows and Linux file due to EOL difference )
| Eclipse JGit [message #1799391] |
Thu, 06 December 2018 01:26  |
Eclipse User |
|
|
|
I am using JGit jar to identify if file is updated or not depending upon SHA1Id generation. Observed, if file contents are same but file is generated on different OS (here windows and Linux) due to EOL difference SHA1Id is different.
I would like to know, is there any way to ignore the EOL while computing the SHA1Id in JGit jar pro-grammatically.
Jar version is : org.eclipse.jgit-5.1.3.201810200350-r
|
|
| | |
| Re: Eclipse JGit [message #1799408 is a reply to message #1799406] |
Thu, 06 December 2018 05:40  |
| Eclipse User |
|
|
|
To compute a sha1 of a local file in a way you can compare it to sha1 which you find in git commit/blobs you would have to follow:
https://stackoverflow.com/a/24283352/3967955
The code in JGit which does this can be looked up here: https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/v5.1.3.201810200350-r/org.eclipse.jgit/src/org/eclipse/jgit/treewalk/WorkingTreeIterator.java#1176
If you want to compute a SHA1 which has autoCRLF semantics then you should feed to the method above not a FileInputStream but JGits AutoCRLFInputStream
like
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.JGitInternalException;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.util.sha1.SHA1;
public class CalcSHA1 {
public static void main(String args[]) throws IOException, GitAPIException, JGitInternalException {
try (InputStream in = new FileInputStream(args[0])) {
System.out.println(
"Git hash for file " + args[0] + " is " + ObjectId.fromRaw(computeHash(in, 1172)).getName());
}
}
private static final byte[] digits = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' };
protected static final byte[] zeroid = new byte[Constants.OBJECT_ID_LENGTH];
private static final byte[] hblob = Constants.encodedTypeString(Constants.OBJ_BLOB);
private static byte[] computeHash(InputStream in, long length) throws IOException {
SHA1 contentDigest = SHA1.newInstance();
final byte[] contentReadBuffer = new byte[4096];
contentDigest.update(hblob);
contentDigest.update((byte) ' ');
long sz = length;
if (sz == 0) {
contentDigest.update((byte) '0');
} else {
final int bufn = contentReadBuffer.length;
int p = bufn;
do {
contentReadBuffer[--p] = digits[(int) (sz % 10)];
sz /= 10;
} while (sz > 0);
contentDigest.update(contentReadBuffer, p, bufn - p);
}
contentDigest.update((byte) 0);
for (;;) {
final int r = in.read(contentReadBuffer);
if (r <= 0)
break;
contentDigest.update(contentReadBuffer, 0, r);
sz += r;
}
if (sz != length)
return zeroid;
return contentDigest.digest();
}
}
|
|
|
| Re: Eclipse JGit [message #1799409 is a reply to message #1799408] |
Thu, 06 December 2018 05:46 |
| Eclipse User |
|
|
|
sorry, example code was not cleaned. See here
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.JGitInternalException;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.util.sha1.SHA1;
public class CalcSHA1 {
public static void main(String args[]) throws IOException, GitAPIException, JGitInternalException {
File file = new File(args[0]);
file.length();
try (InputStream in = new FileInputStream(file)) {
System.out.println(
"Git hash for file " + args[0] + " is " + ObjectId.fromRaw(computeHash(in, file.length())).getName());
}
}
private static final byte[] digits = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' };
protected static final byte[] zeroid = new byte[Constants.OBJECT_ID_LENGTH];
private static final byte[] hblob = Constants.encodedTypeString(Constants.OBJ_BLOB);
private static byte[] computeHash(InputStream in, long length) throws IOException {
SHA1 contentDigest = SHA1.newInstance();
final byte[] contentReadBuffer = new byte[4096];
contentDigest.update(hblob);
contentDigest.update((byte) ' ');
long sz = length;
if (sz == 0) {
contentDigest.update((byte) '0');
} else {
final int bufn = contentReadBuffer.length;
int p = bufn;
do {
contentReadBuffer[--p] = digits[(int) (sz % 10)];
sz /= 10;
} while (sz > 0);
contentDigest.update(contentReadBuffer, p, bufn - p);
}
contentDigest.update((byte) 0);
for (;;) {
final int r = in.read(contentReadBuffer);
if (r <= 0)
break;
contentDigest.update(contentReadBuffer, 0, r);
sz += r;
}
if (sz != length)
return zeroid;
return contentDigest.digest();
}
} |
|
| | |
| Re: Eclipse JGit [message #1799563 is a reply to message #1799525] |
Mon, 10 December 2018 05:58 |
| Eclipse User |
|
|
|
I added crlf conversion to the java example and added a script which shows which sha1's to expect (should be 7594b8c63dc99a37e9b27e6f27e405701413d09f for both files):
> git init
Initialized empty Git repository in /tmp/init/.git/
> ls
linux.txt windows.txt
> git config core.autocrlf input
> git add linux.txt windows.txt
warning: CRLF will be replaced by LF in windows.txt.
The file will have its original line endings in your working directory
> git ls-files -sv
H 100644 7594b8c63dc99a37e9b27e6f27e405701413d09f 0 linux.txt
H 100644 7594b8c63dc99a37e9b27e6f27e405701413d09f 0 windows.txt
> git rm --cached linux.txt windows.txt
rm 'linux.txt'
rm 'windows.txt'
> git config core.autocrlf false
> git add .
> git ls-files -sv
H 100644 7594b8c63dc99a37e9b27e6f27e405701413d09f 0 linux.txt
H 100644 0be486bd774c11a47e2476daaaf082a91d7877c4 0 windows.txt
Java Code:
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.api.errors.JGitInternalException;
import org.eclipse.jgit.lib.Constants;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.util.io.AutoLFInputStream;
import org.eclipse.jgit.util.sha1.SHA1;
public class CalcSHA1 {
static byte[] buffer = new byte[4096];
private static final byte[] digits = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' };
protected static final byte[] zeroid = new byte[Constants.OBJECT_ID_LENGTH];
private static final byte[] hblob = Constants.encodedTypeString(Constants.OBJ_BLOB);
public static void main(String args[]) throws IOException, GitAPIException, JGitInternalException {
for (String a : args) {
File file = new File(a);
long l = 0, length = 0;
try (InputStream in = new AutoLFInputStream(new FileInputStream(file), true)) {
while ((l = in.read(buffer)) != -1) {
length += l;
}
}
try (InputStream in = new AutoLFInputStream(new FileInputStream(file), true)) {
System.out.println(
"Git hash for file " + a + " with length "+length+ " is " + ObjectId.fromRaw(computeHash(in, length)).getName());
}
}
}
private static byte[] computeHash(InputStream in, long length) throws IOException {
SHA1 contentDigest = SHA1.newInstance();
contentDigest.update(hblob);
contentDigest.update((byte) ' ');
long sz = length;
if (sz == 0) {
contentDigest.update((byte) '0');
} else {
final int bufn = buffer.length;
int p = bufn;
do {
buffer[--p] = digits[(int) (sz % 10)];
sz /= 10;
} while (sz > 0);
contentDigest.update(buffer, p, bufn - p);
}
contentDigest.update((byte) 0);
for (;;) {
final int r = in.read(buffer);
if (r <= 0)
break;
contentDigest.update(buffer, 0, r);
sz += r;
}
if (sz != length)
return zeroid;
return contentDigest.digest();
}
}
|
|
| | | | |
| Re: Eclipse JGit [message #1802968 is a reply to message #1802707] |
Tue, 19 February 2019 15:48 |
| Eclipse User |
|
|
|
Hey, we have 2019 and the current Java version is Java 11 and Java 12 is about to be shipped next month.
Why should a JGit committer waste time to backport the code he wrote for you to Java 7 which is out of maintenance as well?
Maybe that's something you could try on your own ?
|
|
|
Goto Forum:
Current Time: Wed Jun 18 07:47:06 EDT 2025
Powered by FUDForum. Page generated in 0.10768 seconds |
Home