Re: [che-dev] Che operator: backup / restore

[Mykola Morhun <mmorhun@xxxxxxxxxx>]

Thank you for your feedback.

@Thomas Maeder there is an epic [1] to support level 3 of the operator framework capability level. Backup / restore is a subtask.

Regarding the question about scenario, it is more disaster recovery, as the actual job should be done by the operator. However, it should be possible to move Che to another cluster by deploying a fresh Che instance and restoring backup from the first cluster. At least I see no blocker now for that.

@Sergii Kabashniuk good question. We have no final decision yet, but taking into account the fact that Che operator should do the backup/restore it would be logical to use its CR to store backup configuration (as we have the operator configuration in one place, but not a separate CRD for configuration of Che server, database, identity provider, etc.). But we are open to proposals and will consider any idea which can bring some benefits to the implementation.

[1] https://github.com/eclipse/che/issues/18043

On Tue, Feb 23, 2021 at 2:43 PM Sergii Kabashniuk <skabashn@xxxxxxxxxx> wrote:

Hello.

@Mykola Morhun  thanks for sharing.

Have you considered to the user a separate CR to manipulate with the 

maintenance process instead of adding an extra part to exist CheCluster CR? 

On Tue, Feb 23, 2021 at 12:45 PM Thomas Mäder <tmader@xxxxxxxxxx> wrote:

Thanks for the heads up.

Reading through the description, I was wondering what scenario we're trying to cover with the backup. Disaster recovery or migration maybe? Also, is there an issue or epic covering this work?

/Thomas

On 23/02/2021 11:32, Mykola Morhun wrote:

Hello all.
Deploy team is working on backup / restore feature for Che operator.
Backup/restore should be triggered by setting a CR field to "backup" or "restore". Also destination storage server (another field) and credentials for it (a secret) have to be configured as well.

The order of the backup procedure is planned to be:
 1. Ensure Che is up and running and backup storage with credentials is configured
 2. Gather all resources to backup:
  - Postgres databases
  - Che cluster CR and CRD
  - Che related secrets and CAs configmaps
 3. Send the resources to an external backup/storage server
 
Operator can get all needed resources yamls directly from the cluster except databases. To backups databases it is planned to use exec into postgres pod and dump them.
To send/retrieve back collected data we plan to use restic [1] cli tool:
 - open source under BSD 2-rd clause licence
 - written in golang, no external dependencies
 - supports many destinations: AWS, Azure, GCS, OpenStack Swift, NFS, SSH, SFTP and many more
 - used as an extesion in Velero to deliver backups to storage servers.
The only downside we see is that we have to include the binary into the operator image, that will increase its size for ~20Mb.

If anyone has concerns / suggestions regarding this topic, please let us know.

[1] https://restic.net/

--

Mykola Morhun

Software engineer

Red Hat

_______________________________________________
che-dev mailing list
che-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/che-dev

_______________________________________________
che-dev mailing list
che-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/che-dev

--

Sergii Kabashniuk

Principal Software Engineer, DevTools 

Red Hat

skabashniuk@xxxxxxxxxx    

--

Mykola Morhun

Software engineer

Red Hat