Re: [geowave-dev] Ingest shapefile from Java API

[Jose M <soloninguno@xxxxxxxxxxx>]

Hi, sorry about that, it seems there was some issue with my network.

I already had the s3 bucket in the repositories section, but they were not donwloaded at first (that's why I switch to 0.9.0 for them).

Now I tried again and they download ok, so it's working :)

Thanks and sorry once again, my mistake.

Regards,

---

De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Rich Fecher <rfecher@xxxxxxxxx>
Enviado: jueves, 04 de febrero de 2016 03:50 p.m.
Para: geowave-dev@xxxxxxxxxxxxxxxx
Asunto: Re: [geowave-dev] Ingest shapefile from Java API

 

Jose,

The NoClassDefFound is because you are mixing 0.9.0-SNAPSHOT with 0.9.1-SNAPSHOT.  You definitely have to choose a version of geowave and be consistent with that choice.

Regarding not seeing two of the dependencies available for version 0.9.1-SNAPSHOT, I looked and they're in our s3 bucket.  Check out this regarding where we publish to s3.  So for example an artifact for geowave-format-vector is available here.  There is no index for these artifacts so you can't browse, but they should be resolved through maven if you setup the pom according to the documentation here.

Alternatively, you could build it yourself from source, but I definitely recommend '-DskipTests' if you don't want it to hit all of our unit and integration tests.  '-Dfindbugs.skip=true' is another favorite, otherwise it will run findbugs on all of the source.

And it looks like you have your dependencies well-covered there for what you appear to be doing.

Good luck and let us know if you have more questions.

Rich

On Thu, Feb 4, 2016 at 1:21 PM, Jose M <soloninguno@xxxxxxxxxxx> wrote:

Hi Rich,

Thanks! I have one more question for you, hope you don't mind. I'm creating a simple example looking at your recommendation but I'm having some issues, don't know if it's because of the transition you mention yesterday, or is may fault this time.

This is my example, really easy as you can see

package mil.nga.giat.geowave.test;

import mil.nga.giat.geowave.core.cli.GeoWaveMain;
import org.apache.commons.lang.StringUtils;

public class GeoWaveTest {
    public static void main(String[] args) {
        String ingestFilePath = "./new_50m_airports.shp";
        System.out.println("Ingesting '" + ingestFilePath + "' - this may take several minutes...");
        final String[] args1 = StringUtils.split(
                "-localingest -datastore accumulo -f geotools-vector -b ./ingest -gwNamespace geowave.palenqueTest -instance accumulo -user geowave -password geowave -zookeeper localhost:2181",
            ' ');
        GeoWaveMain.run(args1);
    }
}

If I try to run it with 0.9.1-SNAPSHOT (the one that I build yesterday), i get this error

Ingesting './new_50m_airports.shp' - this may take several minutes...
Exception in thread "main" java.lang.NoClassDefFoundError: mil/nga/giat/geowave/core/geotime/IndexType
    at mil.nga.giat.geowave.format.geotools.vector.GeoToolsVectorDataStoreIngestPlugin.<init>(GeoToolsVectorDataStoreIngestPlugin.java:57)
    at mil.nga.giat.geowave.format.geotools.vector.GeoToolsVectorDataStoreIngestFormat.getLocalFileIngestPlugin(GeoToolsVectorDataStoreIngestFormat.java:42)
    at mil.nga.giat.geowave.core.ingest.local.LocalFileIngestDriver.runInternal(LocalFileIngestDriver.java:77)
    at mil.nga.giat.geowave.core.ingest.AbstractIngestCommandLineDriver.runOperation(AbstractIngestCommandLineDriver.java:68)
    at mil.nga.giat.geowave.core.cli.GeoWaveMain.run(GeoWaveMain.java:56)
    at mil.nga.giat.geowave.test.GeoWaveTest.main(GeoWaveTest.java:16)
Caused by: java.lang.ClassNotFoundException: mil.nga.giat.geowave.core.geotime.IndexType

And I can't find that class IndexType.

When adding geowave artifacts to my pom, i saw that two of them doesn't have a 0.9.1-SNAPSHOT, geowave-analytic-mapreduce and geowave-format-vector so I use the 0.9.0-SNAPSHOT for them.

Because of that I decide to try running my class but with all artifacts in the 0.9.0-SNAPSHOT version. But with it I got this error

Ingesting './new_50m_airports.shp' - this may take several minutes...
2016-02-04 15:09:53 ERROR GenericStoreCommandLineOptions:314 - Cannot find selected datastore 'accumulo'
2016-02-04 15:09:53 FATAL AbstractIngestCommandLineDriver:188 - Error parsing commandline
org.apache.commons.cli.ParseException: Cannot find selected datastore 'accumulo'
    at mil.nga.giat.geowave.core.cli.GenericStoreCommandLineOptions.parseOptions(GenericStoreCommandLineOptions.java:315)

These are the artifacts I'm using in the POM, maybe I'm missing something.
-geowave-adapter-vector
-geowave-adapter-raster
-geowave-analytic-mapreduce
-geowave-datastore-accumulo
-geowave-core-ingest
-geowave-format-vector

Thanks!

---

De: geowave-dev-bounces@xxxxxxxxxxxxxxxx <geowave-dev-bounces@xxxxxxxxxxxxxxxx> en nombre de Rich Fecher <rfecher@xxxxxxxxx>
Enviado: jueves, 04 de febrero de 2016 12:46 p.m.
Para: geowave-dev@xxxxxxxxxxxxxxxx
Asunto: Re: [geowave-dev] Ingest shapefile from Java API

 

Our integration tests do a lot of programmatic ingest.  Here is our programmatic ingest of shapefiles or anything supported by that pluggable ingest format under the current API:

Example of local shapefile ingest

Example of mapreduce gpx ingest

Unfortunately at this time we don't wrap the commandline tool with a very clean java API interface...yet. A revamp of our commandline tools is one of the bigger tasks associated with 0.9.1 (issue #546).  We plan to document a proposed approach on our wiki and work from there.  The DataStore interface is the main API, but it is completely decoupled from format parsers.  These format plugins hook directly into our ingest framework (right now exposed through commandline tools).

It wouldn't be much effort to wrap calls to GeoWaveMain with some syntactic sugar, but this is going to be rolled into a significant commandline tool undertaking that is moving all of our commons-cli library usage to the jcommander library.

Rich

On Thu, Feb 4, 2016 at 10:24 AM, Jose M <soloninguno@xxxxxxxxxxx> wrote:

Hi guys,

I'm trying to ingest a shapefile in accumulo/geowave (local or hdfsingest) through Java API, but I'm having some issues trying to get all the pieces together.

Is there any test/example I can follow to get that working?

Thanks!

_______________________________________________
geowave-dev mailing list
geowave-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geowave-dev

_______________________________________________
geowave-dev mailing list
geowave-dev@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geowave-dev