Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Syncing data between GeoMesa instances
Hi Jon,

This is an interesting question.  Generally, I think it should work.  The big gotcha would be handling any updates/deletes which are in progress. 

To mitigate various issues, I'd suggest something like this...

1.  Allow writes / deletes to finish.
2.  Manually compact the GeoMesa Accumulo tables.
3.  (Optional) Call clonetable to create 'copies' of the existing tables.
4.  Sync over those tables to the new Accumulo instance. 
** Note:  The root 'catalog' table contains metadata entries which tell GeoMesa which tables are used and for what purpose.  In the new system, you will want to use the existing table names.  If you must change, we'll need to walk through updating that metadata.  It wouldn't be hard, but there isn't an existing tool to help.

I'm suggesting the clonetable command so you can begin ingesting into the original GeoMesa instance if needed.  I believe that Accumulo does the obvious lazy-evaluation/clever thing with the clone tables to save space.  Your sync would then be using static rfiles.

I'd love to hear how this works out for you; feel free to let us know how it goes!  If this approach doesn't work easily, the GeoMesa export/ingest tools will serve as another slower method to try: http://www.geomesa.org/geomesa-tools-ingest-export/.

Thanks,

Jim

On 09/03/2015 01:32 PM, Parise, Jonathan wrote:

Hi,

 

I have a situation where I will likely need to synchronize some data between GeoMesa instances. I would like to do this down at the Accumulo level by synchronizing the tables that GeoMesa is using.

 

What I would like to know is how GeoMesa would respond to this type of change. If the data in the Accumulo tables changes while the system is running, will it break? I know that GeoMesa uses several tables for each datastore defined. If all of these tables were synchronized(which basically means copied from one GeoMesa Accumulo instance to another), would that result in a working system or a broken system.

 

I know this is a bit of a strange question. I am just trying to understand how much we can change the underlying Accumulo data without breaking GeoMesa.

 

Is all of the information GeoMesa needs stored in Accumulo or is some of it elsewhere(in memory, other files on disk, ect)?

 

Thanks,

 

Jon



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users

Back to the top