Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geowave-dev] geowave-tools -hdfsingest

Since v0.8.7 GeoWave has a command line tools framework that relies on Java SPI discoverable libraries to provide functionality. 

When we package GeoWave for installation via RPM the file structure looks as follows:

[andy@c1-app-02 ~]$ tree /usr/local/geowave/tools
├── geowave-tools.jar
└── plugins
    ├── geowave-format-geolife-
    ├── geowave-format-gpx-
    ├── geowave-format-raster-
    ├── geowave-format-tdrive-
    └── geowave-format-vector-

The /usr/local/geowave/tools/ script is then soft-linked into the path of all users using the name "geowave" and takes care of setting the classpath to include anything in the plugins directory

Snippet from the script
exec $JAVA $GEOWAVE_TOOL_JAVA_OPT -cp $CLASSPATH mil.nga.giat.geowave.core.cli.GeoWaveMain "$@"

This setup allows us to reference a "geowave" command to perform any of the configured actions. An example of an ingest operation on an RPM installed system is shown at

If you want to run tools commands from your local repo in development you'd build the application and then set up the classpath to include the various jars yourself.

Set your desired dependencies
export BUILD_ARGS="-Daccumulo.version=1.6.0-cdh5.1.4 -Dhadoop.version=2.6.0-cdh5.4.0 -Dgeotools.version=13.2 -Dgeoserver.version=2.7.2 -Dvendor.version=cdh5 -P cloudera"
export EXTRA_ARGS="-Dfindbugs.skip=true -DskipFormat=true -DskipITs=true -DskipTests=true"

Build the project and the tools.jar
mvn clean install $BUILD_ARGS $EXTRA_ARGS 
mvn package -P geowave-tools-singlejar $BUILD_ARGS $EXTRA_ARGS

Ensure all the plugins you want are on the classpath along with the tools.jar
java -cp deploy/target/geowave-deploy-0.8.9-SNAPSHOT-tools.jar:extensions/formats/geotools-vector/target/geowave-format-vector-0.8.9-SNAPSHOT.jar mil.nga.giat.geowave.core.cli.GeoWaveMain -localingest -l

Once you've verified that the plugins can be discovered by the tools.jar you'd use the remaining arguments "-b, -i etc." the same as the instructions show for a packaged installation.


Let’s download the Admin 0 - Countries shapefile:

$ mkdir ingest
$ mv ingest/
$ cd ingest
$ unzip
$ rm
$ cd ..
$ geowave -localingest \
      -b ./ingest \
      -n geowave.50m_admin_0_countries \ 
      -f geotools-vector \ 
      -u USERNAME \
      -p PASSWORD \

One last note is that there is no csv plugin. The examples I've shown use the geotools-vector plugin which supports a number of formats


On Mon, Sep 14, 2015 at 11:22 AM, Marcel <m.jacob@xxxxxxxxxxx> wrote:
I want to ingest some data via the geowave-tools operation hdfsingest using the following command:

java -cp geowave-deploy-0.8.9-SNAPSHOT-tools.jar mil.nga.giat.geowave.core.cli.GeoWaveMain -hdfsingest \-b /home/marcel/testData/verysmall.csv \-hdfs node1-scads02:8020 \-i accumulo \-n geowave \-p secret \-resourceman node1-scads02:8032 \-u root \-z node1-scads02:2181

FATAL [main] ingest.AbstractIngestCommandLineDriver ( - There were no ingest format plugin providers found

In your documentation it says that there were plugin jars needed. What do you mean by that? Where can I find them?

Marcel Jacob.

geowave-dev mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit

Back to the top