[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [geogig-dev] Stacktrace when cloning a repo on Windows
|
I'm using Windows 7 64-bit.
David.
-----Original Message-----
From: geogig-dev-bounces@xxxxxxxxxxxxxxxx [mailto:geogig-dev-bounces@xxxxxxxxxxxxxxxx] On Behalf Of geogig-dev-request@xxxxxxxxxxxxxxxx
Sent: 04 May 2017 17:00
To: geogig-dev@xxxxxxxxxxxxxxxx
Subject: geogig-dev Digest, Vol 32, Issue 2
Message: 1
Date: Wed, 3 May 2017 11:16:27 -0500
From: Erik Merkle <emerkle@xxxxxxxxxxxxxxxx>
To: GeoGig project <geogig-dev@xxxxxxxxxxxxxxxx>
Subject: Re: [geogig-dev] Stacktrace when cloning a repo on Windows
Message-ID:
<CAK+3T0OBarKw0N-ZFz-B=33RXG7J5X0SKv6YZB6G5JTO2VSCYw@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="utf-8"
David, are you using Windows 7 32-bit, or 64-bit?
Erik Merkle
Software Engineer | Boundless <http://boundlessgeo.com/>
<http://d32wfbeasdaw38.cloudfront.net/img/Boundless_Logo.png>
On Wed, May 3, 2017 at 9:29 AM, Herbert, David J. <darb1@xxxxxxxxx> wrote:
> Hi,
>
>
>
> I am using Geogig 1.0 on Windows 7, Oracle JRE 1.8.0_121. I have a
> remote repository served via Geoserver/PostGIS, containing approx 1GB
> of data for Antarctica.
>
>
>
> From the documentation, I think I should be able to clone the remote
> repository into a directory on the local machine. I tried to do this
> first by issuing the command:
>
>
>
> geogig.bat clone http://<servername>/geoserver/geogig/repos/master
> master
>
>
>
> in the intended directory. I get the following output:
>
>
>
> Cloning into 'master'...
>
>
>
> Fetching objects from refs/heads/master
>
> 10:55:42.835 [main] ERROR o.locationtech.geogig.cli.GeogigCLI - An
> unhandled error occurred: org.rocksdb.RocksDBException: ?. See the log
> for more details.
>
> java.lang.RuntimeException: org.rocksdb.RocksDBException: ?
>
> at
> com.google.common.base.Throwables.propagate(Throwables.java:160)
> ~[guava-18.0.jar:na]
>
> at org.locationtech.geogig.rocksdb.RocksdbObjectStore.
> putAll(RocksdbObjectStore.java:382) ~[geogig-rocksdb-1.0.jar:1.0]
>
> at org.locationtech.geogig.storage.impl.ForwardingObjectDatabase.
> putAll(ForwardingObjectDatabase.java:162) ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.di.ObjectDatabasePutInterceptor$
> GraphUpdatingObjectDatabase.putAll(ObjectDatabasePutInterceptor.java:9
> 8)
> ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.storage.impl.ForwardingObjectDatabase.
> putAll(ForwardingObjectDatabase.java:162) ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.remote.BinaryPackedObjects.
> ingest(BinaryPackedObjects.java:224) ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.remote.HttpRemoteRepo.
> fetchMoreData(HttpRemoteRepo.java:456) ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.remote.HttpRemoteRepo.
> fetchNewData(HttpRemoteRepo.java:208) ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.FetchOp._call(FetchOp.java:257)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.FetchOp._call(FetchOp.java:48)
> ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.repository.AbstractGeoGigOp.
> call(AbstractGeoGigOp.java:153) ~[geogig-api-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.CloneOp._call(CloneOp.java:163)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.CloneOp._call(CloneOp.java:39)
> ~[geogig-core-1.0.jar:1.0]
>
> at org.locationtech.geogig.repository.AbstractGeoGigOp.
> call(AbstractGeoGigOp.java:153) ~[geogig-api-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.porcelain.Clone.runInternal(Clone.java:164
> )
> ~[geogig-cli-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.AbstractCommand.run(AbstractCommand.java:6
> 8)
> ~[geogig-cli-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.GeogigCLI.executeInternal(GeogigCLI.java:5
> 28)
> ~[geogig-cli-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.GeogigCLI.execute(GeogigCLI.java:367)
> ~[geogig-cli-1.0.jar:1.0]
>
> at org.locationtech.geogig.cli.app.CLI.main(CLI.java:80)
> [geogig-cli-app-1.0.jar:1.0]
>
> Caused by: org.rocksdb.RocksDBException: ?
>
> at org.rocksdb.RocksDB.write0(Native Method)
> ~[rocksdbjni-4.13.4.jar:na]
>
> at org.rocksdb.RocksDB.write(RocksDB.java:602)
> ~[rocksdbjni-4.13.4.jar:na]
>
> at org.locationtech.geogig.rocksdb.RocksdbObjectStore.
> putAll(RocksdbObjectStore.java:371) ~[geogig-rocksdb-1.0.jar:1.0]
>
> ... 17 common frames omitted
>
> An unhandled error occurred: org.rocksdb.RocksDBException: ?. See the
> log for more details.
>
>
>
> The error happens consistently after fetching around 300,000 objects
> (out of about 1,130,000). The logfile gives no extra information over
> what?s pasted above.
>
>
>
> I have tried:
>
>
>
> geogig.bat init
>
>
>
> in the directory as well, in case it was missing an initialised empty
> repo, but get the same response. With identical software versions on
> a Centos 7 Linux Virtual Box VM on the same Windows box, I get:
>
>
>
> geogig clone http://<servername>/geoserver/geogig/repos/master master
>
>
>
> Cloning into 'master'...
>
>
>
> Fetching objects from refs/heads/master
>
> 1,129,655
>
> Processed 1,129,697 objects. Inserted: 1,129,697. Existing: 0. Time:
> 21.46 min. Compressed size: 630,289,302 bytes. Uncompressed size:
> 1,257,346,509 bytes.
>
> 100%
>
> Done.
>
>
>
> The created local repo is as expected. It looks like I might have hit
> some size or complexity limit in the Windows case? It looks like I am
> using rocksdb 4.13.4.
>
>
>
> Any help on this much appreciated.
>
>
>
> Thanks,
>
>
>
> David Herbert
>
> British Antarctic Survey.
> ------------------------------
> This message (and any attachments) is for the recipient only. NERC is
> subject to the Freedom of Information Act 2000 and the contents of
> this email and any reply you make may be disclosed by NERC unless it
> is exempt from release under the Act. Any material supplied to NERC
> may be stored in an electronic records management system.
> ------------------------------
>
> _______________________________________________
> geogig-dev mailing list
> geogig-dev@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://dev.locationtech.org/mailman/listinfo/geogig-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dev.locationtech.org/mailman/private/geogig-dev/attachments/20170503/875cb885/attachment.html>
------------------------------
Message: 2
Date: Wed, 3 May 2017 16:27:11 +0000
From: "Herbert, David J." <darb1@xxxxxxxxx>
To: "geogig-dev@xxxxxxxxxxxxxxxx" <geogig-dev@xxxxxxxxxxxxxxxx>
Subject: Re: [geogig-dev] Stacktrace when cloning a repo on Windows
(ErikMerkle)
Message-ID:
<DB4PR06MB25412B2AC2564CE3F76C00BB2160@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset="us-ascii"
Hi Erik,
Thanks for your input. I replaced the rocksdb jar with 4.11.2 and repeated the clone - I now get:
Fetching objects from refs/heads/master
17:12:32.171 [main] ERROR o.locationtech.geogig.cli.GeogigCLI - An unhandled error occurred: org.rocksdb.RocksDBException: IO error: Failed to FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Documents\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst: The process cannot access the file because another process has locked a portion of the file.
. See the log for more details.
java.lang.RuntimeException: org.rocksdb.RocksDBException: IO error: Failed to FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Documents\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst: The process cannot access the file because another process has locked a portion of the file.
at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-18.0.jar:na]
at org.locationtech.geogig.rocksdb.RocksdbObjectStore.putAll(RocksdbObjectStore.java:382) ~[geogig-rocksdb-1.0.jar:1.0]
at org.locationtech.geogig.storage.impl.ForwardingObjectDatabase.putAll(ForwardingObjectDatabase.java:162) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.di.ObjectDatabasePutInterceptor$GraphUpdatingObjectDatabase.putAll(ObjectDatabasePutInterceptor.java:98) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.storage.impl.ForwardingObjectDatabase.putAll(ForwardingObjectDatabase.java:162) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.remote.BinaryPackedObjects.ingest(BinaryPackedObjects.java:224) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.remote.HttpRemoteRepo.fetchMoreData(HttpRemoteRepo.java:456) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.remote.HttpRemoteRepo.fetchNewData(HttpRemoteRepo.java:208) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.porcelain.FetchOp._call(FetchOp.java:257) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.porcelain.FetchOp._call(FetchOp.java:48) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.repository.AbstractGeoGigOp.call(AbstractGeoGigOp.java:153) ~[geogig-api-1.0.jar:1.0]
at org.locationtech.geogig.porcelain.CloneOp._call(CloneOp.java:163) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.porcelain.CloneOp._call(CloneOp.java:39) ~[geogig-core-1.0.jar:1.0]
at org.locationtech.geogig.repository.AbstractGeoGigOp.call(AbstractGeoGigOp.java:153) ~[geogig-api-1.0.jar:1.0]
at org.locationtech.geogig.cli.porcelain.Clone.runInternal(Clone.java:164) ~[geogig-cli-1.0.jar:1.0]
at org.locationtech.geogig.cli.AbstractCommand.run(AbstractCommand.java:68) ~[geogig-cli-1.0.jar:1.0]
at org.locationtech.geogig.cli.GeogigCLI.executeInternal(GeogigCLI.java:528) ~[geogig-cli-1.0.jar:1.0]
at org.locationtech.geogig.cli.GeogigCLI.execute(GeogigCLI.java:367) ~[geogig-cli-1.0.jar:1.0]
at org.locationtech.geogig.cli.app.CLI.main(CLI.java:80) [geogig-cli-app-1.0.jar:1.0] Caused by: org.rocksdb.RocksDBException: IO error: Failed to FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Documents\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst: The process cannot access the file because another process has locked a portion of the file.
at org.rocksdb.RocksDB.write0(Native Method) ~[rocksdbjni-4.11.2.jar:na]
at org.rocksdb.RocksDB.write(RocksDB.java:555) ~[rocksdbjni-4.11.2.jar:na]
at org.locationtech.geogig.rocksdb.RocksdbObjectStore.putAll(RocksdbObjectStore.java:371) ~[geogig-rocksdb-1.0.jar:1.0]
... 17 common frames omitted
An unhandled error occurred: org.rocksdb.RocksDBException: IO error: Failed to FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Documents\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst: The process cannot access the file because another process has locked a portion of the file.
. See the log for more details.
The log contains nothing more than this, apart from three occurrences of:
2017-05-03 17:17:07,712 INFO [main] o.l.g.p.CreateDeduplicator [CreateDeduplicator.java:40] No DeduplicationService service found, using default heap based one
Before the stacktrace. I guess this new error reveals it's some kind of file locking issue.
Thanks,
David.
________________________________
This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dev.locationtech.org/mailman/private/geogig-dev/attachments/20170503/2689ea9b/attachment.html>
------------------------------
Message: 3
Date: Wed, 3 May 2017 14:14:39 -0500
From: Erik Merkle <emerkle@xxxxxxxxxxxxxxxx>
To: GeoGig project <geogig-dev@xxxxxxxxxxxxxxxx>
Subject: Re: [geogig-dev] Stacktrace when cloning a repo on Windows
(ErikMerkle)
Message-ID:
<CAK+3T0Moj3qV-7V5mmoS0z2352KG0KRY9MD3sinXYoU-mPGA_g@xxxxxxxxxxxxxx>
Content-Type: text/plain; charset="utf-8"
Thanks for the info David. I've created an issue in GitHub for this:
https://github.com/locationtech/geogig/issues/302
We'll try to reproduce the issue and see if we can fix it inside GeoGig itself, or determine if it might be a bug in the Windows implementation of the RocksDB library.
I'm not sure I can offer a suitable workaround, but if you want/need to have this data in a GeoGig repository on Windows, exporting the data from the remote (GeoServer) repository and importing that into a freshly init'ed RocksDB repository on Windows might work (though I can't say for sure if the issue you are having won't affect import in the same way). Also, if you choose to go this route for now, understand that the import may not import any revision history from the remote repository.
Erik Merkle
Software Engineer | Boundless <http://boundlessgeo.com/>
<http://d32wfbeasdaw38.cloudfront.net/img/Boundless_Logo.png>
On Wed, May 3, 2017 at 11:27 AM, Herbert, David J. <darb1@xxxxxxxxx> wrote:
> Hi Erik,
>
>
>
> Thanks for your input. I replaced the rocksdb jar with 4.11.2 and
> repeated the clone ? I now get:
>
>
>
> Fetching objects from refs/heads/master
>
> 17:12:32.171 [main] ERROR o.locationtech.geogig.cli.GeogigCLI - An
> unhandled error occurred: org.rocksdb.RocksDBException: IO error:
> Failed to
> FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Doc
> uments\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst:
> The process cannot access the file because another process has locked
> a portion of the file.
>
> . See the log for more details.
>
> java.lang.RuntimeException: org.rocksdb.RocksDBException: IO error:
> Failed to FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Doc
> uments\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst:
> The process cannot access the file because another process has locked
> a portion of the file.
>
>
>
> at
> com.google.common.base.Throwables.propagate(Throwables.java:160)
> ~[guava-18.0.jar:na]
>
> at
> org.locationtech.geogig.rocksdb.RocksdbObjectStore.putAll(RocksdbObjec
> tStore.java:382)
> ~[geogig-rocksdb-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.storage.impl.ForwardingObjectDatabas
> e.putAll(ForwardingObjectDatabase.java:162) ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.di.ObjectDatabasePutInterceptor$Grap
> hUpdatingObjectDatabase.putAll(ObjectDatabasePutInterceptor.java:98)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.storage.impl.ForwardingObjectDatabas
> e.putAll(ForwardingObjectDatabase.java:162) ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.remote.BinaryPackedObjects.ingest(BinaryPacked
> Objects.java:224)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.remote.HttpRemoteRepo.fetchMoreData(HttpRemote
> Repo.java:456)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.remote.HttpRemoteRepo.fetchNewData(HttpRemoteR
> epo.java:208)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.FetchOp._call(FetchOp.java:257)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.FetchOp._call(FetchOp.java:48)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.repository.AbstractGeoGigOp.call(AbstractGeoGi
> gOp.java:153)
> ~[geogig-api-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.CloneOp._call(CloneOp.java:163)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.porcelain.CloneOp._call(CloneOp.java:39)
> ~[geogig-core-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.repository.AbstractGeoGigOp.call(AbstractGeoGi
> gOp.java:153)
> ~[geogig-api-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.porcelain.Clone.runInternal(Clone.java:164
> )
> ~[geogig-cli-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.AbstractCommand.run(AbstractCommand.java:6
> 8)
> ~[geogig-cli-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.GeogigCLI.executeInternal(GeogigCLI.java:5
> 28)
> ~[geogig-cli-1.0.jar:1.0]
>
> at
> org.locationtech.geogig.cli.GeogigCLI.execute(GeogigCLI.java:367)
> ~[geogig-cli-1.0.jar:1.0]
>
> at org.locationtech.geogig.cli.app.CLI.main(CLI.java:80)
> [geogig-cli-app-1.0.jar:1.0]
>
> Caused by: org.rocksdb.RocksDBException: IO error: Failed to
> FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Doc
> uments\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst:
> The process cannot access the file because another process has locked
> a portion of the file.
>
>
>
> at org.rocksdb.RocksDB.write0(Native Method)
> ~[rocksdbjni-4.11.2.jar:na]
>
> at org.rocksdb.RocksDB.write(RocksDB.java:555)
> ~[rocksdbjni-4.11.2.jar:na]
>
> at
> org.locationtech.geogig.rocksdb.RocksdbObjectStore.putAll(RocksdbObjec
> tStore.java:371)
> ~[geogig-rocksdb-1.0.jar:1.0]
>
> ... 17 common frames omitted
>
> An unhandled error occurred: org.rocksdb.RocksDBException: IO error:
> Failed to FlushViewOfFile: C:\Users\DARB1.BS-DARB1-L5\Doc
> uments\projects\add\data\working\master\.geogig\objects.rocksdb/000015.sst:
> The process cannot access the file because another process has locked
> a portion of the file.
>
> . See the log for more details.
>
>
>
> The log contains nothing more than this, apart from three occurrences of:
>
>
>
> 2017-05-03 17:17:07,712 INFO [main] o.l.g.p.CreateDeduplicator
> [CreateDeduplicator.java:40] No DeduplicationService service found,
> using default heap based one
>
>
>
> Before the stacktrace. I guess this new error reveals it?s some kind
> of file locking issue.
>
>
>
> Thanks,
>
>
> David.
>
>
> ------------------------------
> This message (and any attachments) is for the recipient only. NERC is
> subject to the Freedom of Information Act 2000 and the contents of
> this email and any reply you make may be disclosed by NERC unless it
> is exempt from release under the Act. Any material supplied to NERC
> may be stored in an electronic records management system.
> ------------------------------
>
> _______________________________________________
> geogig-dev mailing list
> geogig-dev@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://dev.locationtech.org/mailman/listinfo/geogig-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dev.locationtech.org/mailman/private/geogig-dev/attachments/20170503/f3a9a0b1/attachment.html>
------------------------------
_______________________________________________
geogig-dev mailing list
geogig-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.locationtech.org/mailman/listinfo/geogig-dev
End of geogig-dev Digest, Vol 32, Issue 2
*****************************************
________________________________
This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
________________________________
