Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Use of Landsat 8 imagery with GeoMesa
Hi Miguel,

From looking at the data you provided and reading through our code, I think I have identified the issue.  The raster code is picking up some of the raster chunks at different GeoHash levels.  Sadly, this is fairly particular, internal error, and it may take us a bit to think through and fix.  I've entered a ticket for this issue here:  https://geomesa.atlassian.net/browse/GEOMESA-943.

Sorry I don't have better news.

Jim

On 10/30/2015 08:17 AM, Miguel Ángel Esbrí wrote:
Hi Jim,

Please, find attached the output from the two commands you indicated in your previous email.

I have enclosed also an screenshot of the current table status provided by Accumulo in case it might be of help as well.

Btw, I was wondering whether the raster ingestion tool takes into account the temporal dimension of the raster images in order to build the indexes. Every 16 days there are new available scenes for the same areas, which we would like to add to the existing layers (B1,... B11) - instead of having to create separate layers per date -  so we can use the temporal capabilities of the WMS and WCS for requesting the data afterwards.

This is something I couldn't test yet due to the previous problem.


Thanks,

Miguel Ángel

Inline image 1

Miguel A. Esbrí

On Thu, Oct 29, 2015 at 7:16 PM, Jim Hughes <jnh5y@xxxxxxxx> wrote:
Hi Miguel,

Thanks for the logs.  The exception is happening during some metadata retrieval. 

If you are willing to do a little more digging, providing the data in that table before you wipe it might help us out.  There should be an combiner set on that table which would aggregate the keys for which there was an issue.  The two diagnostics you could do are...

1.  Scan the bounds table and send along the output.*

2.  Send along the output of 'config -t GEOMESA_RASTER_BOUNDS'.

With that info, we'd be able to see if the issue is with the combiner or with the data ingest.  I have put in a ticket in our JIRA here https://geomesa.atlassian.net/browse/GEOMESA-938.

Depending on what the underlying cause is, we should be able to cook up a work around.

Thanks,

Jim

* If the 'scan' command returns several screens, you can do the following:

Create a file (say /tmp/script) with lines like this:

table GEOMESA_RASTER_BOUNDS
scan -np
exit

And then run
> accumulo shell -u user -p pass -f /tmp/script > /tmp/rasterOutput.txt

Accumulo's shell will execute the commands in the file given, and the -np on the scan will turn off pagination.



On 10/29/2015 01:42 PM, Miguel Ángel Esbrí wrote:
Hi Jim,

I uploaded the Lansat 8 image example here: https://www.dropbox.com/s/nhagv7xwundp28e/LC82020322015275LGN00_B3.TIF?dl=0

The geoserver logs are attached to the email.

Thank you very much for the support :)


Miguel A. Esbrí

On Thu, Oct 29, 2015 at 4:25 PM, Jim Hughes <jnh5y@xxxxxxxx> wrote:
Hi Miguel,

Thanks for the question.  In order to help, I've got two quick questions:

First, would you be willing to send along your geoserver.log or the log from your J2EE container running GeoServer?  It should have more of the stack trace you are seeing, and with that stack trace, we'll be able to understand what's going on a little better.

Second, is there a public link to the Landsat 8 Imagery you are using?  If there, I'll try and find some time to reproduce the error locally. 

And you are right, we don't have support for multiband rasters sorted at the minute.

Thanks,

Jim


On 10/29/2015 11:16 AM, Miguel Ángel Esbrí wrote:
Hi,

As part of our agriculture-based project we need to store and process Landsat 8 imagery for large areas in various countries in Europe.

We are interested in using GeoMesa as solution since it seems to provide many of the features we need to store, visualize and process (using the additional Geoserver plugins) the large amount raster images  we have to consider in the project .

I managed to set up a working environment (Apache Accumulo + Geomesa + Geoserver) and followed the raster ingestion example as indicated here: http://www.geomesa.org/geomesa-raster/

The natural earth example can be visualized without problem, however when it comes to visualizing the ingested Landsat 8 image via a WMS request I am getting the following:



Error:
<?xml version="1.0" encoding="UTF-8" standalone="no"?><!DOCTYPE ServiceExceptionReport SYSTEM "http://5.79.69.49:28080/geoserver/schemas/wms/1.1.1/WMS_exception_1_1_1.dtd"> <ServiceExceptionReport version="1.1.1" >   <ServiceException>
      Error rendering coverage on the fast path
org.locationtech.geomesa.shade.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: duplicate key: 0.002526
java.lang.IllegalArgumentException: duplicate key: 0.002526
duplicate key: 0.002526
</ServiceException></ServiceExceptionReport>

Somethings regarding the steps I followed for ingesting the Landsat rasters:

- We need to store the 11 (B1 to B11) bands that compose a Landsat 8 scene, therefore we created one layer per band (AFAIK multiband rasters are not supported yet by Geomesa)

- The Landsat 8 scene is reprojected to EPSG:4326 (from EPSG:32630): gdalwarp -t_srs EPSG:4326 LC82020322015275LGN00_B3.TIF LC82020322015275LGN00_B3_4326.TIF

- Image pyramid creation: gdal_retile.py -v -pyramidOnly -r lanczos -levels 4 -ps 675 675 -targetDir ./ LC82020322015275LGN00_B3_4326.TIF

- Rasters ingestion: geomesa ingestraster -u geomesa -p geomesa -t LANDSAT8_B3 -f "./1/" -fmt geotiff 

Same procedure for pyramid levels 2, 3 and 4

Do you have any idea about what may be causing the duplicate key problem? 

I am attaching also the output from gdalinfo command applied to the LC82020322015275LGN00_B3.TIF  image in case the info might be of help.

Best regards


Miguel A. Esbrí


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users

Back to the top