Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-dev] [geomesa-users] GeoMesa Accumulo Custom Iterator
  • From: "Lahr-Vivaz, Emilio" <Emilio.Lahr-Vivaz@xxxxxxxxxxx>
  • Date: Tue, 2 Jul 2024 19:58:14 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ga-ccri.com; dmarc=pass action=none header.from=ga-ccri.com; dkim=pass header.d=ga-ccri.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector5401; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3yLhlEwRWxfcJkoqUMrEuvc9bebzf4dg6XyhGNah5yw=; b=JfeGlPWIF8ePRj9/f58mCFpBISbiAJcbPG4p1CPCMRlQj5oCxvkReHLgaB6DSht/LIBB8tJmLtjjg6IqSfXrn2xCf0EtQbP5C5VzCWpETWdcalYaKaV4e9xtta8PDFLGPWK/YA1TyeY01WloC3pcSlqQeWc26oIMcln+O0X48Gvq8GeiwzSl/vihouPfomlyH9CgEXT4egDtf20zpLntkC3Kb2XLQ/SOYGplU0dBqDevAdBevzSxnrzqQ+Vyb+mq+HTvHyT8a7olfPrZzKlesJaNvix9fOGo6CvvTog8vtzjmSqJJpg4+YXtEKeTZzXLHi9K4C+lFTIGYKGK3FquPA==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector5401; d=microsoft.com; cv=none; b=rATR5Nge4dAbGdwLjpvh0VP+rhl+fYOL5ZXegCjZBrZHbe+HKZEttnBI5kwIFr1nyJzv6Dy/xJsdwM6flEM+dQogMk0NvxHq28VNDO6IA0XRlkv2n1Yenl+TudWf2/0XxPANVLd6g7a8/y4wlc6h/HJmXLtNieFBdzH8PoBOa4vy+L3r449Sth/unm3RyZIznM84AIXqswu9yEn+KaGUaA97jm7r2YD6qEDyFbt/2SkTk3CSg6DxMsXzkU0lwucoSpjmPWtI4oBHo9b8FlO+xynGHe26T33G4lHWiol32OiGhXjuagYaSFMNnTJ33jKRteT7XbsSiReEPi3lwjXf7w==
  • Delivered-to: geomesa-dev@xxxxxxxxxxx
  • List-archive: <https://www.eclipse.org/mailman/private/geomesa-dev/>
  • List-help: <mailto:geomesa-dev-request@eclipse.org?subject=help>
  • List-subscribe: <https://www.eclipse.org/mailman/listinfo/geomesa-dev>, <mailto:geomesa-dev-request@eclipse.org?subject=subscribe>
  • List-unsubscribe: <https://www.eclipse.org/mailman/options/geomesa-dev>, <mailto:geomesa-dev-request@eclipse.org?subject=unsubscribe>
  • Msip_labels:
  • Thread-index: AQHazIBkVDchyp0CNkaj5nMKNmp7SLHjbaqAgABsJto=
  • Thread-topic: [geomesa-users] GeoMesa Accumulo Custom Iterator
I would suggest trying to get the attribute level visibilities working - others have used them successfully, and that should be simpler than writing a custom iterator. GeoMesa doesn't have a pluggable way to add iterators - if you want to implement one, the steps would be:

  • Fork the repo
  • Implement your iterator
  • Extend the code for query handling to check for your custom query hint, and configure your iterator appropriately
  • Bundle your iterator into the geomesa-accumulo-distributed-runtime jar and install it into Accumulo
  • Deploy your custom query handling code to your environment (spark, geoserver, etc)
  • Keep your fork up-to-date with any upstream changes

Thanks,

Emilio Lahr-Vivaz
General Atomics, CCRi
From: Patel, Meet RTX <meet.2.patel@xxxxxxx>
Sent: Tuesday, July 2, 2024 9:25 AM
To: Lahr-Vivaz, Emilio <Emilio.Lahr-Vivaz@xxxxxxxxxxx>
Subject: RE: [geomesa-users] GeoMesa Accumulo Custom Iterator
 

Thank you for replying.

 

The goal of the custom iterator is to remove/filter key/value pairs based on the user visibility that we pass in.

I have tried to implement the attribute level visibilities on geomesa-accumulo 2.12 4.0.1 version however unable to store the attribute with a proper visibility. So, I want to create a GeoMesa custom iterator that will allow key/value filtering on the Accumulo tablet server level and return only pairs that the user is able to see. I will pass in a Hint from the GeoTools API that will tell the iterator to filter based on the value.

Does this make sense? Is there a better way to process?

Sincerely,

Meet Patel

 

From: geomesa-users <geomesa-users-bounces@xxxxxxxxxxx> On Behalf Of Lahr-Vivaz, Emilio via geomesa-users
Sent: Tuesday, July 2, 2024 9:05 AM
To: geomesa-dev@xxxxxxxxxxx; geomesa-users@xxxxxxxxxxx
Cc: Lahr-Vivaz, Emilio <Emilio.Lahr-Vivaz@xxxxxxxxxxx>
Subject: [External] Re: [geomesa-users] GeoMesa Accumulo Custom Iterator

 

Hello,

 

What is your iterator meant to do?

I see you posted the same question on https://gis.stackexchange.com/questions/483276/geomesa-custom-iterator - why don't we move the discussion there.

 

For questions about running Accumulo on k8s, I would ask the Accumulo lists.


Thanks,

 

Emilio Lahr-Vivaz

General Atomics, CCRi

From: geomesa-users <geomesa-users-bounces@xxxxxxxxxxx> on behalf of Patel, Meet RTX via geomesa-users <geomesa-users@xxxxxxxxxxx>
Sent: Thursday, June 27, 2024 10:22 AM
To: geomesa-dev@xxxxxxxxxxx <geomesa-dev@xxxxxxxxxxx>; geomesa-users@xxxxxxxxxxx <geomesa-users@xxxxxxxxxxx>
Cc: Patel, Meet RTX <meet.2.patel@xxxxxxx>
Subject: -EXT-[geomesa-users] GeoMesa Accumulo Custom Iterator

 

WARNING:  This message is from an external source.  Evaluate the message carefully BEFORE clicking on links or opening attachments.

Hello,

 

I am looking to create a custom iterator under GeoMesa’s library in Scala. Will this iterator run at Accumulo’s tablet server level? I want to ensure that the processing of the custom iterator runs at the tserver level.

 

Also, the accumulo database is running on a pod in K8. Is there a way to enable logging? If so how am I able to do that and where am I able to see the logging for custom iterator?

 

Thanks,

Meet

 

Back to the top