Re: [geomesa-users] StatsCombiner

[Nathan Mercer <nmercer@xxxxxxxxxxxx>]

I did get the classpath working, but I still did see the StatsCombiner errors and I reverted back to 1.2.1. 

However, recently I upgraded to 1.2.6 with a namespace and I am not seeing the errors. Maybe the first time I still had the state cached. But I did restart Accumulo a bunch of times so I am not so sure about that. I think it was actually the fact that there were some tables created with 1.2.1 and some with the newer version and there was conflicts. After I reverted back to 1.2.1 I still got errors until I removed all tables that were created with version 1.2.6.

Note that I am not sure how to deal with GeoServer however because it does namespaces so it appears that you can only install one version of the GeoServer plugin at a time. And I cannot get GeoServer to work with version 1.2.6.

-----Original Message-----
From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Emilio Lahr-Vivaz
Sent: Friday, October 7, 2016 6:37 AM
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] StatsCombiner

Hi Jeff,

Yes, I believe Nathan got the classpath working. The main things to check are:

* permissions for the file on hdfs - try to make sure it's owned by the hdfs user used by accumulo if you are having problems
* namespace configuration parameters are correct
* your table is in the correct namespace

Accumulo tends to cache state, so even after e.g. fixing permissions, I often have to restart to pick up the changes.

As a backup plan, you can always put the jars into lib/ext on each tablet server.

Thanks,

Emilio

On 10/06/2016 08:03 PM, Jeff Gold wrote:
> GeoMesa Team-
>     Was the StatsCombiner issue from Nathan from July ever resolved? I 
> don't see a solution to that part of his question in the thread below 
> and I'm hitting the same issue in my upgrade from 1.2.1 to 1.2.6.
>
> I'm using namespaces, I've done the Accumulo restart...
>
>
>
>
> Thanks,
> Jeff
>
>
>
>
>
> From: Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>
> Date: Fri, 15 Jul 2016 16:04:51 -0400
> Delivered-to: geomesa-users@xxxxxxxxxxxxxxxx
> User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
> Thunderbird/38.8.0
>
> ________________________________
> Hi Nathan,
>
> We've got a fix for the contains issue here:
> https://github.com/locationtech/geomesa/pull/995
>
> If you get a chance, could you try it out and let us know if it fixes 
> your problem? You'll have to re-ingest your polygons so that the fix 
> takes effect.
>
> Thanks,
>
> Emilio
>
> On 07/14/2016 04:50 PM, Emilio Lahr-Vivaz wrote:
>
> Hi Nathan,
>
> For your contains issue: we've identified a bug in the way we index 
> polygons. Basically at very fine resolutions (e.g. a point), there are 
> some gaps in our index. We're working on a fix, but in the meantime 
> you might avoid the issue by querying a small polygon instead of a 
> point - e.g. 'contains(geom, POLYGON((-1 -1, -1 1, 1 1, 1 -1, -1 
> -1)))'. To apply the point filtering, you can manually filter the 
> results coming back - e.g.
>
> Filter filter = ECQL.toFilter("contains(geom, POINT(0 0))");
>   ...
> SimpleFeature feature = reader.next();
>
> if (filter.evaluate(feature)) {
> ...
> }
>
> Thanks,
>
> Emilio
>
> On 07/11/2016 03:43 PM, Nathan Mercer wrote:
>
> Thanks again Jim,
>
>
>
> Yes I have restarted Accumulo. Still getting the errors. Continuing to 
> try to figure it out.
>
>
>
> And since you asked, I have another issue. When I upgraded to 1.2.3 it 
> appears that the contains method is not working. I have a MULTIPOLYGON 
> type in geomesa and I want to get back the polygons that contain a 
> point I pass in. I have an application that does this through a Spark 
> RDD and it used to work fine in 1.2.1, but not after the upgrade. Same 
> thing if I manually try to do an export using the geomesa tools with a 
> CQL query using contains. This proves to me that it can’t be my code 
> and there must be a problem with geomesa update. Also note that I am 
> positive my polygons contain the point. My polygons make up the 
> boundaries of the US. And I use a point that is right in the middle of 
> the US.
>
>
>
> Cheers,
>
> Nathan
>
>
>
>
>
> From: geomesa-users-bounces@xxxxxxxxxxxxxxxx
> [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim 
> Hughes
> Sent: Friday, July 8, 2016 4:44 PM
> To: geomesa-users@xxxxxxxxxxxxxxxx
> Subject: Re: [geomesa-users] StatsCombiner
>
>
>
> Hi Nathan,
>
> I'll respond inline since there are two different questions.
>
> On 07/08/2016 05:18 PM, Nathan Mercer wrote:
>
> Thank you Jim.
>
>
>
> I followed the instructions for setting up Accumulo namespace to 
> install the distributed runtime jar within HDFS as per the 
> instructions 
> herehttp://www.geomesa.org/documentation/user/installation_and_configu
> ration.html#installing-the-accumulo-distributed-runtime-library
>
>
>
> However, I am still getting these errors, although it doesn’t appear 
> on ingest any more, but it is happening on my querying it appears. But 
> I am sure I am querying using 1.2.3 so it should be compatible. Isn’t 
> installing this runtime library equal to deploying the iterators? Or 
> is there something I am missing. I have also removed all jar files 
> from the Accumulo lib/ext path so these should not be getting in the 
> way. But what do I do about old tables that were ingested using an 
> older version of geomesa? Will I need to re-ingest these? With 
> namespaces, in the future I should simply have to specify the table 
> with the namespace prefix and all should be well, but all my old 
> tables were ingested without the use of namespaces.
>
>
> As a long shot, after you setup the namespace and removed the jars 
> from lib/ext, did you restart Accumulo?  If not, the old jars could 
> still be in the Accumulo tablet servers JVMs.
>
>
> Finally, I am also seeing another strange thing with version 1.2.3. I 
> have an application that creates an AccumuloDataStore object and when 
> the application ends, it doesn’t shut down. It just stays open. I have 
> narrowed it down to the data store object. If I revert back to 1.2.1 
> this does not happen (I skipped from 1.2.1 to 1.2.3).
>
>
> This is a bug, and we've since fixed it.  The issue is that the new 
> pre-computed stats writer was hanging on in a static context.  We've 
> moved the stats writing to be managed by each datastore, and we are 
> cleaning it up via the DataStore.dispose() method.
>
> Good questions!  Keep them coming!
>
> Jim
>
>
> Cheers,
>
> Nathan
>
>
>
> From: geomesa-users-bounces@xxxxxxxxxxxxxxxx
> [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Jim 
> Hughes
> Sent: Wednesday, July 6, 2016 4:00 PM
> To: geomesa-users@xxxxxxxxxxxxxxxx
> Subject: Re: [geomesa-users] StatsCombiner
>
>
>
> Hi Nathan,
>
> GeoMesa 1.2.3 has the ability to gather stats on the data during 
> ingest.  In order to do this, the StatsCombiner is configured on any 
> new tables (assuming you don't turn the collectStats flag off).
>
> You should just have to deploy the GeoMesa 1.2.3 iterators on your 
> Accumulo classpath (or on a namespace's classpath).
>
> Generally, during an update, the GeoMesa code for ingest clients, 
> Accumulo iterators, and query clients (like GeoServer) all should 
> match.  Since coordinating that may be complicated, I'd call attention 
> to Accumulo namespaces; I've used namespaces like geomesa120, 
> geomesa121, etc. to help keep things straight on our internal dev 
> cluster.
>
> Cheers,
>
> Jim
>
> On 07/06/2016 05:38 PM, Nathan Mercer wrote:
>
> Hi. I am trying to upgrade to Geomesa 1.2.3
>
>
>
> I have a program which does some data ingest using a FeatureWriter.
> When I try ingesting, I get the following error
> java.lang.ClassNotFoundException:
> org.locationtech.geomesa.accumulo.data.stats.StatsCombiner
>
>
>
> Is there something new I need to be adding to my pom file or something?
>
>
>
> I have been playing around with this for a while and can’t figure it out.
>
>
>
> Thanks,
>
> Nathan
>
>
>    ­­
>
>
> _______________________________________________
>
> geomesa-users mailing list
>
> geomesa-users@xxxxxxxxxxxxxxxx
>
> To change your delivery options, retrieve your password, or 
> unsubscribe from this list, visit
>
> https://www.locationtech.org/mailman/listinfo/geomesa-users
> _______________________________________________
> geomesa-users mailing list
> geomesa-users@xxxxxxxxxxxxxxxx
> To change your delivery options, retrieve your password, or 
> unsubscribe from this list, visit 
> https://www.locationtech.org/mailman/listinfo/geomesa-users

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://www.locationtech.org/mailman/listinfo/geomesa-users

*************************************************************************
PRIVILEGE AND CONFIDENTIALITY NOTICE:

The information in this email is intended for the named recipients only.
It may contain privileged and confidential information.  If you have
received this communication in error, any use, copying or dissemination
of its contents is strictly prohibited.  Please erase all copies of the
message along with any included attachments and notify Intermap
Technologies or the sender immediately by telephone at the number
indicated on this page.

*************************************************************************