Migrating from Gravity to Bring Your Own Kubernetes (BYOK8s)#

Contact the Anaconda implementation team before you begin for assistance migrating your version of Data Science & AI Workbench from Gravity to BYOK8s. Follow along with these instructions as an Anaconda implementation team member guides you through the migration process.

Note

If you are using network file storage (NFS) or dynamic storage for both your anaconda-storage and anaconda-persistence volumes within your Gravity cluster, it is not necessary to perform these steps, as you can unmount these volumes from your Gravity cluster and remount them on your BYOK8s cluster. An Anaconda implementation team member will guide you through this process, if necessary.

Caution

Have all users save their work, stop any open sessions and deployments, and log out of the platform so no data is lost during the migration process.

Prerequisites#

The following conditions must be met to complete this task successfully:

You must have sudo access.
The jq conda package must be installed in your base environment.

OR

Optionally, you can install this ae5-conda environment, which already contains the necessary packages.

Install the backup and restore tool#

Migrating your data from Gravity to BYOK8s involves using the Workbench backup and restore tool, which you can obtain using conda.

Note

If you chose to install the ae5-conda environment listed in the prerequisites, skip ahead to verify your installation.

Standard environment#

For a standard environment, you can install the ae5_backup_restore package into your base conda environment by opening a terminal and running the following command:

conda install -c ae5-admin ae5_backup_restore

Once complete, verify the installation.

Air-gapped environment#

For Airgapped networks, download the latest ae5_conda installer file and move it to your master node.

Set the installer file to be executable, then install the ae5_conda environment by running the following commands:

chmod +x ae5-conda-latest-Linux-x86_64.sh
./ae5-conda-latest-Linux-x86_64.sh

Once complete, verify the installation.

Verify your installation#

Verify you’ve successfully installed the backup and restore tool by running the following command:

ae_backup.sh -h

If your terminal returns the usage help text for the backup script, your installation was successful.

Run the backup script#

When running the backup script, include the --config-db or -c command line argument to instruct the backup script to only capture your Workbench configuration data. User and project data is migrated separately later in this process.

To create a backup file of your cluster in your current working directory, run the command:

bash ae_backup.sh -c

To create a backup file of your cluster and store it in a specific directory, run the command:

# Replace <DIRECTORY_PATH> with the path to the directory you want to contain your backup
bash ae_backup.sh -c <DIRECTORY_PATH>

The backup script creates a tarball file that stores your Kubernetes resources and Postgres data with the following naming scheme:

ae5_config_db_YYYYMMDDHHMM.tar.gz

Note

YYYYMMDDHHMM is the format for the date timestamp of your backup data.

Migrate user data#

Before you can migrate user data, you must prevent users from making edits to the disk.

Open a terminal.
SSH into your Workbench master node.
Enter Gravity by running the following command:
```
gravity enter
```

Scale down the Anaconda Platform pods, identified by their ap- prefix, by running the following command:

kubectl get deploy | grep ap- | cut -d' ' -f1 | xargs kubectl scale deploy --replicas=0

Verify that the ap- prefixed pods have stopped by running the following command:
```
watch kubectl get pods
```
Once the ap- prefixed pods have stopped, migrate your user data by moving the following directories into the new cluster:

/opt/anaconda/storage/git/repositories/anaconda

/opt/anaconda/storage/projects

This can be done by either directly mounting the /opt/anaconda/storage volume onto a workstation with access to the BYOK8s cluster, or by compressing the directories separately, then copying them into the storage pod.

Caution

Migrating user data requires that both the Gravity and BYOK8s clusters have identical UID/GID permissions set. If permissions are not correctly aligned, your user data will not migrate.

To directly mount the /opt/anaconda/storage volume onto a workstation with access to the BYOK8s cluster, move both directories directly into place on top of the pre-existing directories on the BYOK8s cluster.

If you are unable to directly mount the /opt/anaconda/storage volume, you will instead need to copy both tarballs you have taken to the /tmp directory by running the following commands:

# Replace <PATH_TO_GIT_TARBALL> with the path to your /opt/anaconda/storage/git/repositories/anaconda tarball
# Replace <PATH_TO_PROJECTS_TARBALL> with the path to your /opt/anaconda/storage/projects tarball
# Replace <WORKBENCH_GIT_POD> with the ID of the Workbench git storage pod (the pod prefix is "anaconda-enterprise-ap-git-storage-")
# Replace <WORKBENCH_PROJECTS_POD> with the ID of the Workbench project storage pod (the pod prefix is "anaconda-enterprise-ap-storage")
kubectl cp <PATH_TO_GIT_TARBALL> <WORKBENCH_GIT_POD>:/tmp
kubectl cp <PATH_TO_PROJECTS_TARBALL> <WORKBENCH_PROJECTS_POD>:/tmp

Once you have copied the tarballs to /tmp, exec into the pods, then move them into place by running the following commands:

# Replace <WORKBENCH_GIT_POD> with the ID of the Workbench git storage pod (the pod prefix is "anaconda-enterprise-ap-git-storage-")
kubectl exec -it <WORKBENCH_GIT_POD> /bin/bash
mv /tmp/anaconda /opt/anaconda/storage/git/repositories/anaconda
# Replace <WORKBENCH_PROJECTS_POD> with the ID of the Workbench project storage pod (the pod prefix is "anaconda-enterprise-ap-storage")
kubectl exec -it <WORKBENCH_PROJECTS_POD> /bin/bash
mv /tmp/projects /opt/anaconda/storage/projects

Run the restore script#

Restoring to a different host without a hostname change#

Use this method if you are migrating to a cluster with a URL that differs from the URL of your Gravity cluster.

Restored data:

Kubernetes secrets (non-ssl)
Postgres
ConfigMaps
SSL certs
Secrets

Non-restored data:

Hostname
Ingress

Restore your data by running the following command:

bash ae_restore.sh ae5_config_db_YYYYMMDDHHMM

Restoring to a different host with a hostname change#

Use this method if you are migrating to a new cluster but you want to retain the Gravity cluster’s URL.

Restore your data by running the following command:

bash ae_restore.sh -u ae5_config_db_YYYYMMDDHHMM

After you have moved user data over, and have completed running the Restore script, the migration is complete and you can confirm that your BYOK8s cluster contains all data from the old Gravity cluster.