Upgrading between versions of AE5¶

To get assistance upgrading your version of AE5, contact the Anaconda implementation team.

After you have determined the topology for your Anaconda Enterprise cluster, and verified that your system meets all of the installation requirements, you’re ready to upgrade the cluster.

Before you begin:

Configure your A record in DNS for the master node with the actual domain name you will use for your Anaconda Enterprise installation.
If you are using a firewall for network security, we recommend you temporarily disable it while you upgrade Anaconda Enterprise.
When installing Anaconda Enterprise on a system with multiple nodes, verify that the clock of each node is in sync with the others prior to starting the installation process, to avoid potential issues. We recommend using the Network Time Protocol (NTP) to synchronize computer system clocks automatically over a network. See instructions here.

Create a backup of your anaconda-enterprise-anaconda-platform.yml ConfigMap by running the following command:

kubectl get cm anaconda-enterprise-anaconda-platform.yml -o json | jq -r '.data["anaconda-platform.yml"]' > configmap-backup.yml

Caution

Project sessions are terminated during the upgrade process! Because of this, it is important to stop all sessions prior to upgrading. If you do not, sessions that are terminated as part of the upgrade process must be restarted manually post upgrade.
Sessions can be stopped programmatically using ae5-tools by running the following command in a terminal that has access your Workbench cluster over the network:
ae5 session list --columns=id --no-header | xargs -n1 ae5 session stop --yes

Anaconda Enterprise supports in-place upgrades. The upgrade process varies slightly, depending on your current version and which version you’re installing. To update an existing Anaconda Enterprise installation to a newer version, follow the process that corresponds to your particular scenario:

Upgrading from AE 5.4.x to 5.5.0
Upgrading from AE 5.3.x to 5.5.x
Upgrading from AE 5.3.0/5.3.1 to 5.4.x
Upgrading from AE 5.2.x/5.3.0 to 5.3.1

Upgrading from AE 5.4.x to 5.5.0¶

Download and decompress the installer file onto the master node running Anaconda Enterprise software, then cd into the directory. Open a terminal and run the following commands:

# Replace <LOCATION_OF_INSTALLER> with the location of the installer file
curl -O <LOCATION_OF_INSTALLER>.tar.gz
# Replace <VERSION> with your installer version
tar xvzf anaconda-enterprise-<VERSION>.tar.gz
cd anaconda-enterprise-<VERSION>

Run the following command to verify your environment is properly prepared:
```
sudo ae-preflight
```
If the check returns an overall result of WARN, you can view the results of the check by running the command:
```
cat results.txt
```
If necessary, make applicable corrections as seen on our Installation requirements page.
Run the following command to start the upload and upgrade process:
```
sudo ./upload
sudo ./gravity upgrade
```
Depending on your implementation, the upgrade process may take an hour or more to complete. This is primarily due to the upload step. You can check the status of the upgrade process by running the following:
```
watch sudo ./gravity status
OR
watch sudo ./gravity plan
```
Even after the upgrade process is completed, it will still take some time for the pods to come up. You can monitor those with the following command:
```
watch kubectl get pods
```

If you encounter errors while upgrading, you can check the status of the operation by running sudo ./gravity plan. You can then roll back any step in the upgrade process by running the rollback command against the name of the phase, as it’s listed in the Phase column:

# Replace <NAME_OF_PHASE> with the name listed in the Phase column
sudo ./gravity rollback --phase=/<NAME_OF_PHASE>

After addressing the error(s), you can resume the upgrade by running the following command:

sudo ./gravity upgrade --resume --force

After the upgrade process completes, follow the steps to verify that your upgrade was successful.

After you’ve confirmed that your upgrade was successful—and everything works as expected—you can run a script to remove images leftover from the previous installation and free up space. This will help prevent the cluster from running out of disk space on the master node.

Upgrading from AE 5.3.x to 5.5.x¶

Anaconda recommends performing a fresh reinstall of Anaconda Enterprise software when upgrading from version 5.3.x. As a precautionary measure, create a backup of your current instance of Anaconda Enterprise before performing an upgrade.

Before upgrading, you’ll need to retain your configurations and settings.

Create a folder to contain the data and name it reinstall, then run the extract_config.sh script. Run the following commands:
```
mkdir reinstall
cd reinstall
bash extract_config.sh
```
The extract_config script retains the following .yaml files:
- anaconda-enterprise-certs.yaml
- anaconda-enterprise-keycloak.yaml
- helm_values.yaml
- values.yaml

Run the following commands to export your SSL Certificates, user secrets, and ConfigMap to .yaml files:

kubectl get cm -o yaml --export > configmap.yaml
kubectl get secret anaconda-enterprise-certs -o yaml --export > certs.yaml
kubectl get secrets | grep user-creds | cut -d ' ' -f 1 | xargs kubectl get secret -o yaml > user-secrets.yaml

Once your configurations and settings are backed up, you can move forward with your upgrade.

Uninstall Anaconda Enterprise on all nodes.
Reboot your instance.

Download and decompress the installer file onto the master node running Anaconda Enterprise software, then cd into the directory. Open a terminal and run the following commands:

# Replace <LOCATION_OF_INSTALLER> with the location of the installer file
curl -O <LOCATION_OF_INSTALLER>.tar.gz
# Replace <VERSION> with your installer version
tar xvzf anaconda-enterprise-<VERSION>.tar.gz
cd anaconda-enterprise-<VERSION>

Run the following command to verify your environment is properly prepared:
```
sudo ae-preflight
```
If the check returns an overall result of WARN, you can view the results of the check by running the command:
```
cat results.txt
```
If necessary, make applicable corrections as seen on our Installation requirements page.
Install the AE5 cluster.
Replace your SSL Certificates and secrets by running the following commands:
```
kubectl replace -f certs.yaml
kubectl create -f user-secrets.yaml
```
Manually restore the configurable sections of the config map (resource profiles, affinity settings, shared volumes, conda settings, git settings).

Update the Git repository for existing projects in Postgres, if necessary.

# Replace <SESSION_ID> with your postgres session ID
kubectl exec -it anaconda-enterprise-postgres-<SESSION_ID> /bin/bash
psql -U postgres
\c anaconda_storage

Update your projects.

SET repo_url = REPLACE (repo_url,'https://anaconda-enterprise-ap-git-storage/platform/git/anaconda/','http://anaconda-enterprise-ap-git-storage/anaconda/')

Replace http://anaconda-enterprise-ap-git-storage/anaconda/ with your external git repository URL, if necessary.

Add worker nodes.

After the upgrade process completes, follow the steps to verify that your upgrade was successful.

Upgrading from AE 5.3.0/5.3.1 to 5.4.x¶

Anaconda Enterprise 5.3.0 and 5.3.1 support in-place upgrades, so you can follow these simple steps to update your 5.3.0 or 5.3.1 installation to the latest version.

Download and decompress the installer file onto the master node running Anaconda Enterprise software, then cd into the directory. Open a terminal and run the following commands:

# Replace <LOCATION_OF_INSTALLER> with the location of the installer file
curl -O <LOCATION_OF_INSTALLER>.tar.gz
# Replace <VERSION> with your installer version
tar xvzf anaconda-enterprise-<VERSION>.tar.gz
cd anaconda-enterprise-<VERSION>

Download the shim file and unpack it, then cd into the direcotry. Run the following commands:

# Replace <LOCATION_OF_SHIM> with the location of the shim file
curl -O <LOCATION_OF_SHIM>.tar.gz
# Replace <VERSION> with your shim version
tar xvzf anaconda-enterprise-<VERSION>-gravity-only.tar.gz
cd anaconda-enterprise-<VERSION>-gravity-only

Run the following command to upload the installer to the AE environment:
```
sudo ./upload
```
When the upload process finishes, run the following command to start the upgrade process:
```
sudo ./gravity upgrade
```

Change directory to the install directory and rerun the following commands:

# Replace <VERSION> with your installer version
cd ../anaconda-enterprise-<VERSION>
sudo ./upload
sudo ./gravity upgrade

Depending on your implementation, the upgrade process may take an hour or more to complete. You can check the status of the upgrade process by running sudo ./gravity status.

# Replace <NAME_OF_PHASE> with the name listed in the Phase column
sudo ./gravity rollback --phase=/<NAME_OF_PHASE>

After addressing the error(s), you can resume the upgrade by running the following command:

sudo ./gravity upgrade --resume --force

After the upgrade process completes, follow the steps to verify that your upgrade was successful.

Upgrading from AE 5.2.x/5.3.0 to 5.3.1¶

Anaconda Enterprise 5.2.x and 5.3.0 support in-place upgrades, so you can follow these simple steps to update your 5.2.x or 5.3.0 installation to the latest version.

Download and decompress the installer file onto the master node running Anaconda Enterprise software, then cd into the directory. Open a terminal and run the following commands:

# Replace <LOCATION_OF_INSTALLER> with the location of the installer file
curl -O <LOCATION_OF_INSTALLER>.tar.gz
# Replace <VERSION> with your installer version
tar xvzf anaconda-enterprise-<VERSION>.tar.gz
cd anaconda-enterprise-<VERSION>

Run the following command to upload the installer to the AE environment:
```
sudo ./upload
```
When the upload process finishes, run the following command to start the upgrade process:
```
sudo ./gravity upgrade
```
The upgrade process may take up to an hour to complete. You can check the status of the upgrade process by running sudo ./gravity status.

# Replace <NAME_OF_PHASE> with the name listed in the Phase column
sudo ./gravity rollback --phase=/<NAME_OF_PHASE>

After addressing the error(s), you can resume the upgrade by running the following command:

sudo ./gravity upgrade --resume --force

After the upgrade process completes, follow the steps to verify that your upgrade was successful.

Verify installation¶

After you’ve verified that all pods are running and updated the Anaconda Enterprise URLs, you can confirm that your upgrade was successful by doing the following:

Return to the Authentication Center and select Users in the Manage menu on the left.
Click View all users and verify that all user data has also been restored.
Access the Anaconda Enterprise user console by visiting this URL in your browser: https://example.anaconda.com/—replacing example.anaconda.com with the FQDN of your server—and logging in using the same credential you used in your previous installation.
Review the Projects list to verify that all project data has been restored.

Note

If you didn’t configure SSL certificates as part of the post-install configuration, do so now. See Updating TLS/SSL certificates for more information.

If you’re upgrading a cluster with external Git configured:

Note

The git section of the anaconda-enterprise-anaconda-platform.yml file used to configure Anaconda Enterprise 5.3.1 includes parameter changes. If you backed up your Anaconda Enterprise ConfigMap before upgrading, and copied it onto the newly-updated master node, you’ll need to update your ConfigMap with the new information as described here.

If you’re upgrading a Spark/Hadoop configuration:

After you successfully restore your Anaconda Enterprise data, run the following commands on the master node of the newly-installed Anaconda Enterprise server:

kubectl replace -f <PATH_TO_ANACONDA_CONFIG_FILES_SECRETS.yaml>

To verify that your configuration upgraded correctly:

Log in to Anaconda Enterprise.
If your configuration uses Kerberos authentication, open a Hadoop terminal and authenticate yourself through Kerberos using the same credentials you used previously. For example, kinit <USERNAME>.
Open a Jupyter Notebook that uses Sparkmagic, and verify that it behaves as expected. For example, run the sc command to connect to Sparkmagic and start Spark.

After you’ve confirmed that your upgrade was successful, we recommend you run the following command to remove all unused packages and images from previous versions of the application, and repopulate the registry to include only those images required by the current version of the application:

sudo gravity gc

The command’s progress is displayed in the terminal, so you can watch as it marks packages associated with the latest version as required, and deletes older versions.

If running the command generates an error, you can resume the command (after you fix the issue that caused the error) by running the following command:

sudo gravity gc —-resume