Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[geomesa-users] Code adapted from geomesa-gdelt-master for different data doesn't update Accumulo

After getting the geomesa-gdelt-master code to work with one of data files at ( I decided to adapt the, and classes to work with a different data-set (see sensor.csv).


I believe I have made only minimal changes to the “GDELT*.java” files producing, (enclosed) and (not included). The pom.xml and core-site.xml files are unchanged.


My data (see sensor.csv) is defined in as:


static List<String> attributes = Lists.newArrayList(



                                "Time:Date",                                      // Date => a "Unix" time (e.g., millis since 1970.01.01)







                                "*geom:Point:srid=4326"           // SRID 4326 => standard coordinate system



The critical fields are “Time:Date” (which is a Unix-like time), and the event lat/long (“SensorGeo_Lat/Long:Float”). I note that the GDELT data uses an “SQLDATE” field of the form “YYYYMMDD”. However, the lat/long fields are similar. I even renamed them to “SensorGeo_Lat” and “SensorGeo_Long” on the theory that “…Geo_...” is a significant naming convention.


I am running on an RHEL 6.5 server (VMWare Workstation VM), using Accumulo 1.5.1 and Hadoop 2.4.0.


Using the enclosed “./bin/data” Bash script, I run the command “data upload” to load the data into Hadoop, and then “data ingest” to ingest it into Accumulo (using Hadoop Map-Reduce).


I see no errors, but when I use the “accumulo shell” commands “table sensor” and “scan”, I only see 4 (metadata) entries in the target “sensor” table, rather than 124 “SimpleFeature” entries:


~METADATA_sensor attributes: []    EventID:Integer,Sensor:Integer,Time:Date,Latitude:Float,Longitude:Float,Altitude:Integer,MinAxis:Float,MajAxis:Float,Orientation:Float,geom:Point:srid=4326

~METADATA_sensor dtgfield: []    Time

~METADATA_sensor featureEncoding: []    avro

~METADATA_sensor schema: []    %~#s%99#r%sensor#cstr%0,3#gh%yyyyMMdd#d::%~#s%3,2#gh::%~#s%#id


I’m perplexed as to what I have not done or done wrong here.


Any thoughts or suggestions would be greatly appreciated!


Bob Barnhart

Chief Systems Engineer | 858 826 5596 (Office) | 619 972 9489 (Mobile) | Robert.M.Barnhart@xxxxxxxxxx



Attachment: pom.xml
Description: pom.xml

Attachment: core-site.xml
Description: core-site.xml



Attachment: sensor.csv
Description: sensor.csv

Attachment: data
Description: data

Back to the top