Migrating from Gravity to Bring Your Own Kubernetes (BYOK8s)#
Contact the Anaconda implementation team before you begin for assistance migrating your version of Data Science & AI Workbench from Gravity to BYOK8s. Follow along with these instructions as an Anaconda implementation team member guides you through the migration process.
Note
If you are using network file storage (NFS) or dynamic storage for both your anaconda-storage
and anaconda-persistence
volumes within your Gravity cluster, it is not necessary to perform these steps, as you can unmount these volumes from your Gravity cluster and remount them on your BYOK8s cluster. An Anaconda implementation team member will guide you through this process, if necessary.
Caution
Have all users save their work, stop any open sessions and deployments, and log out of the platform so no data is lost during the migration process.
Prerequisites#
The following conditions must be met to complete this task successfully:
You must have
sudo
access.The
jq
conda package must be installed in your base environment.
OR
Optionally, you can install this ae5-conda environment, which already contains the necessary packages.
Install the backup and restore tool#
Migrating your data from Gravity to BYOK8s involves using the Workbench backup and restore tool, which you can obtain using conda.
Note
If you chose to install the ae5-conda
environment listed in the prerequisites, skip ahead to verify your installation.
Standard environment#
For a standard environment, you can install the ae5_backup_restore
package into your base conda environment by opening a terminal and running the following command:
conda install --channel ae5-admin ae5_backup_restore
Once complete, verify the installation.
Air-gapped environment#
For air-gapped networks, download the latest ae5_conda
installer file and move it to your master node.
Set the installer file to be executable, then install the ae5_conda
environment by running the following commands:
chmod +x ae5-conda-latest-Linux-x86_64.sh
./ae5-conda-latest-Linux-x86_64.sh
Once complete, verify the installation.
Verify your installation#
Verify you’ve successfully installed the backup and restore tool by running the following command:
ae_backup.sh -h
If your terminal returns the usage help text for the backup script, your installation was successful.
Run the backup script#
When running the backup script, include the --config-db
or -c
command line argument to instruct the backup script to only capture your Workbench configuration data. User and project data is migrated separately later in this process.
To create a backup file of your cluster in your current working directory, run the command:
bash ae_backup.sh -c
To create a backup file of your cluster and store it in a specific directory, run the command:
# Replace <DIRECTORY_PATH> with the path to the directory you want to contain your backup
bash ae_backup.sh -c <DIRECTORY_PATH>
The backup script creates a tarball file that stores your Kubernetes resources and Postgres data with the following naming scheme:
ae5_config_db_YYYYMMDDHHMM.tar.gz
Note
YYYYMMDDHHMM is the format for the date timestamp of your backup data.
Migrate user data#
Before you can migrate user data, you must prevent users from making edits to the disk.
Open a terminal.
SSH into your Workbench master node.
Enter Gravity by running the following command:
gravity enter
Scale down the Anaconda Platform pods, identified by their
ap-
prefix, by running the following command:kubectl get deploy | grep ap- | cut -d' ' -f1 | xargs kubectl scale deploy --replicas=0
Verify that the
ap-
prefixed pods have stopped by running the following command:watch kubectl get pods
Once the
ap-
prefixed pods have stopped, migrate your user data by moving the following directories into the new cluster:/opt/anaconda/storage/git/repositories/anaconda
/opt/anaconda/storage/projects
This can be done by either directly mounting the
/opt/anaconda/storage
volume onto a workstation with access to the BYOK8s cluster, or by compressing the directories separately, then copying them into the storage pod.Caution
Migrating user data requires that both the Gravity and BYOK8s clusters have identical UID/GID permissions set. If permissions are not correctly aligned, your user data will not migrate.
To directly mount the /opt/anaconda/storage
volume onto a workstation with access to the BYOK8s cluster, move both directories directly into place on top of the pre-existing directories on the BYOK8s cluster.
If you are unable to directly mount the /opt/anaconda/storage
volume, you will instead need to copy both tarballs you have taken to the /tmp
directory by running the following commands:
# Replace <PATH_TO_GIT_TARBALL> with the path to your /opt/anaconda/storage/git/repositories/anaconda tarball
# Replace <PATH_TO_PROJECTS_TARBALL> with the path to your /opt/anaconda/storage/projects tarball
# Replace <WORKBENCH_GIT_POD> with the ID of the Workbench git storage pod (the pod prefix is "anaconda-enterprise-ap-git-storage-")
# Replace <WORKBENCH_PROJECTS_POD> with the ID of the Workbench project storage pod (the pod prefix is "anaconda-enterprise-ap-storage")
kubectl cp <PATH_TO_GIT_TARBALL> <WORKBENCH_GIT_POD>:/tmp
kubectl cp <PATH_TO_PROJECTS_TARBALL> <WORKBENCH_PROJECTS_POD>:/tmp
Once you have copied the tarballs to /tmp
, exec into the pods, then move them into place by running the following commands:
# Replace <WORKBENCH_GIT_POD> with the ID of the Workbench git storage pod (the pod prefix is "anaconda-enterprise-ap-git-storage-")
kubectl exec -it <WORKBENCH_GIT_POD> /bin/bash
mv /tmp/anaconda /opt/anaconda/storage/git/repositories/anaconda
# Replace <WORKBENCH_PROJECTS_POD> with the ID of the Workbench project storage pod (the pod prefix is "anaconda-enterprise-ap-storage")
kubectl exec -it <WORKBENCH_PROJECTS_POD> /bin/bash
mv /tmp/projects /opt/anaconda/storage/projects
Run the restore script#
Restoring to a different host without a hostname change#
Use this method if you are migrating to a cluster with a URL that differs from the URL of your Gravity cluster.
Restored data:
Kubernetes secrets (non-ssl)
Postgres
configmaps
SSL certs
Secrets
Non-restored data:
Hostname
Ingress
Restore your data by running the following command:
bash ae_restore.sh ae5_config_db_YYYYMMDDHHMM
Restoring to a different host with a hostname change#
Use this method if you are migrating to a new cluster but you want to retain the Gravity cluster’s URL.
Restore your data by running the following command:
bash ae_restore.sh -u ae5_config_db_YYYYMMDDHHMM
After you have moved user data over, and have completed running the Restore script, the migration is complete and you can confirm that your BYOK8s cluster contains all data from the old Gravity cluster.