[geomesa-users] R: Error on import data from nifi

[Francesco Gusmeroli <cisco-90@xxxxxxxxxx>]

Hi,

thanks a lot Emilio! You saved my day. I didn't think about conversion in CSVReader and now, following what you have suggested, works all right.

Thanks again,

Francesco

---

Da: geomesa-users <geomesa-users-bounces@xxxxxxxxxxx> per conto di Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>
Inviato: venerdì 21 maggio 2021 14:48
A: geomesa-users@xxxxxxxxxxx <geomesa-users@xxxxxxxxxxx>
Oggetto: Re: [geomesa-users] Error on import data from nifi

 

Hello,

The default date column would just be configured with the name of your date attribute, i.e. "time" (although if you only have one date-type field then you can leave it empty). However, it doesn't automatically convert the attribute to a date-type field, that has to be done with the record reader you configure. If you're using TSV, presumably you're using the CSVReader controller service record reader with 'inferred schema'. In the configuration for the controller service, you need to make the 'Date Format' property match the format of the dates in your data, so that the reader infers that it's a date column. If you're using a different schema strategy or controller service, the exact steps would be different but the end goal would be the same: to make the record reader type that column as a date.

I opened a ticket here to automatically force the specified default date field into a date-type field, to make this easier in the future: https://geomesa.atlassian.net/browse/GEOMESA-3080

Thanks,

Emilio

On 5/21/21 3:25 AM, Francesco Gusmeroli wrote:

Hi, 

 

i’ve a trobule using https://github.com/geomesa/geomesa-nifi v3.2.0 component to ingest data:

Premise:
I use this command to import data via cli:
./bin/geomesa-redis ingest -u redis://:nY6F.Ju8G.4xN2CjRT@10.246.0.10:18637 -c tpl -f water -s time:Date,geom:Point:srid=4326,vehicleID:int /home/assets/geomesa-ingestion/test.tsv
it works well and I can see data with datetime, point and vehicleID

Example of data:
time,geom,vehicleID 

2021-03-12T15:02:56Z,POINT(0,0),7

I tried to import the same file with PutGeomesaRedisRecord: “Geometry columns” setted up geom:Point:srid=4326 works fine, connection is ok, but I cannot figure out how setup “Default date columns” to accept my date column. I tried several way, column name is “time” and I tried with these values: “time”, “time:Date”, “time:DATE”, “time:date”, “time:timestamp”, “time:Timestamp”, “time.TIMESTAMP”. I also tried to use different values for time: with or without single/double quota, yyyy-MM-ddTHH:mm:ssZ yyyy-MM-dd HH:mm:ss timestamp numeric but all of these tries lead up to the same error:



If I leave “Default date columns” empty, data is written on redis but with string/numeric values depends on which values I use.

After I tried PutGeoMesaRedis, I saved the sft when I import it with cli and I try to use via processor. This lead up to that error. 

I also tried with HBase but I just found the same problems

Could you please help me?

Thanks, 

Francesco Gusmeroli

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxx
To unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/geomesa-users