Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] how do you upgrade the catalog if you are upgrading geomesa versions
If you're hitting oddities, in almost every case it won't actually corrupt your data. The best thing to do is try to ensure that all of your geomesa jars are installed correctly and match versions across your whole environment (ingest/accumulo/geoserver/etc). The migration job isn't likely to help in this case.

When running the migration job, usually the easiest thing is to point it at a different table with the same simple feature type name. Then, once you've verified that the migration works, you can just archive the old tables (by renaming and offlining it) and then rename the new tables back to the old ones. One note: if you rename any of the non-catalog tables, you'll have to update the appropriate metadata entry by hand to point to the correct table. You can do all that through the accumulo shell.

For adding attributes, you can manually update the schema string in the metadata as you say. You need to ensure that the new attributes are *after* any existing attributes. We don't support removing attributes.

There is no problem with using a single catalog table for multiple feature types. By default they will share most of the same tables (the exception is the z3 table). Even if you share a catalog table, you can use separate data tables by setting the 'geomesa.table.sharing' user data in your simple feature type user data to the string 'false'. The only downside to sharing tables is storing an extra byte per row of data - for large-scale applications this is sometimes undesirable.

There is no easy way that I know of to merge tables in accumulo - but even if you could, certain meta-data is calculated per-catalog-table, so merging multiple existing catalogs would cause conflicts.

Thanks,

Emilio

On 07/19/2016 12:06 PM, Diane Griffith wrote:

Yes I have an unknown thing around how would one add and/or remove attributes of an existing simple feature type/schema. 

 

I also feel I was hitting oddities of trying to ingest data an old feature type/schema once I upgraded geomesa between 1.2.0 and 1.2.3 (not running the update index job) and how to recover without re-ingesting.  So if one hits an oddity is the recommended course of action to run that job to upgrade the indexing to see if it clears the issue?  When one uses that update index job on upgrade, does one use the geomesa command line tool to rename the new tables to the old table names or does one just use the accumulo shell to do that?

 

Regarding how to add and/or remove attributes of a schema, if it is just manually update the schema string in the catalog metadata (as it comes out from the accumulo shell) then good to know.  Not sure if removing is the same thing as adding. 

 

Previously I was just re-ingesting which now is becoming more of a pain point for me. 

 

Is there a simple path that would let me combine separate simple feature types that were originally ingested into separate tables/catalogs into one tablename/catalog?  Maybe you all don’t recommend putting different feature types together in one table/catalog.

 

Thanks,

Diane

 

 

 

From: geomesa-users-bounces@xxxxxxxxxxxxxxxx [mailto:geomesa-users-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Emilio Lahr-Vivaz
Sent: Tuesday, July 19, 2016 9:49 AM
To: geomesa-users@xxxxxxxxxxxxxxxx
Subject: Re: [geomesa-users] how do you upgrade the catalog if you are upgrading geomesa versions

 

Hi Diane,

1. No, the old caveats still apply - your data will not be modified, just the way we store the metadata in the catalog table. Updating the data is left up to the user, via the index update jobs, as many users have large amounts of data and don't want to re-write it all.
2. You can still manually append new attributes to a schema by modifying the catalog metadata, and any new or updated records will have that attribute. Old data will be null for that attribute.

Is your confusion about adding new attributes to an existing schema? We don't have any documentation for that at the moment, as it's an 'unsupported' feature. I can explain the steps here if that's what you're asking about.

Thanks,

Emilio

On 07/19/2016 09:34 AM, Diane Griffith wrote:

So it took me a bit to understand the terminology.  In the 1.2.4 release notes you mention once you upgrade to 1.2.4 one cannot revert a catalog back to 1.2.3 or older without doing some manual steps which you all can provide.

 

So that begs the following questions:

 

1.       Can one upgrade an existing catalog that maybe was created in version 1.2.0, 1.2.2, or 1.2.3 to an newer version and get any indexing improvements without dropping and re-ingesting?

2.       Can one update the catalog to have new fields (i.e. add a second geometry or just new ifelds) and then any new ingests of the existing records as well as new records will fill out those new columns? 

 

I understand now how to manually update one of the rows in the catalog now but many of the rows are related and as I can see from schemas when I ingested under 1.2.0 and 1.2.3 that the attributes row looks different depending on the geomesa version I created the simple feature attribute type with.

 

So I am not sure if I can upgrade an existing simple feature type/catalog or how to do it. 

 

Any explanation or pointer to an area of the documentation would be appreciated.

 

Diane




_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users

 



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.locationtech.org/mailman/listinfo/geomesa-users

Back to the top