Updating TLS/SSL certificates#

You can replace the self-signed certificates Data Science & AI Workbench generates as part of the initial post-install configuration at any time.

To minimize downtime, make sure that you have completed the preparation steps below before scheduling a maintenance interval to update the certificates. When you are ready, you can update the certificates using either the admin console or the command line. We strongly recommend the use of the admin console to minimize the potential for error.

Caution

Instruct users to save their work and log out before updating the certificates. While sessions, deployments, and jobs will still continue to function properly, there will be a brief interruption of front-end service during the update process.

Preparing for the update#

Anaconda recommends that you gather the following information and files before you proceed. If you are using the admin console, have your files present in a location where you can enter their contents into the applicable fields within the UI. If you are updating your certificates using the command line, you’ll need to copy the files to your server.

Most installations will need the following items:

  • The fully qualified domain name of the server.

  • The public SSL certificate for the domain <FQDN>: tls.crt

  • The private SSL key for the domain <FQDN>: tls.key.

  • If applicable, the intermediate certificate bundle: intermediate.pem.

  • If your certificate was issued by a private root CA, the public certificate for that CA: rootca.crt.

If you are using LetsEncrypt, the filenames will be different:

  • The public SSL certificate: cert.pem

  • The private SSL key for the domain: privkey.pem.

  • The intermediate certificate bundle: chain.pem.

  • No root CA file is needed in this case.

If you are using a different domain and/or SSL certificate for the session/deployment subdomains, you will need this additional information:

  • The wildcard subdomain. We will refer to this as <DEPLOYMENT-FQDN> below.

  • The public SSL certificate for *.<DEPLOYMENT-FQDN>: wildcard.crt

  • The private SSL key for *.<DEPLOYMENT-FQDN>: wildcard.key

Note that Workbench assumes that the intermediate certificate and root CA, if applicable, are identical for both certificates.

Finally, if you intend to use the command-line approach to updating the certificates, you will need a copy of the the latest set of publicly-trusted root certificates. A copy of this file, current to when the Workbench installer package was built, can be found at DIY-SSL-CA/CA/pubCA.crt in your unpacked installer assets.

Note

Workbench version 5.6+ automatically applies the updated SSL certificates to all running sessions and deployments.

Using the admin console#

  1. Log in to Workbench as an administrator.

  2. Open the My Account dropdown menu menu and select Admin Console.

  3. Select Web Certificates cert_icon from the left-hand menu.

  4. Enter your certificate and key information from the files you gathered during preparation into the appropriate fields. Make sure to paste the content of each file into the appropriate box, not the filenames.

    • Domain name: <FQDN>

    • SSL Certificate: tls.crt / cert.pem

    • SSL Private Key: tls.key / privkey.pem

    • Root Certificate: rootca.crt, if applicable

    • Intermediate Certificate: intermediate.pem / chain.pem, if applicable

    • Wildcard Domain: <DEPLOYMENT-FQDN>, or <FQDN> if identical. Do not include an asterisk.

    • Wildcard Certificate: wildcard.crt if different; tls.crt / cert.pem if identical.

    • Wildcard Private Key: wildcard.key if different; tls.key / privkey.pem if identical.

  5. Click Save to update the platform with your changes.

  6. If your root CA has changed, you will need to restart the Workbench system pods to make sure that the pods that use this certificate will pick up the new copy.

    kubectl get pods | grep ap- | cut -d ' ' -f 1 | xargs kubectl delete pods
    

Using the command line#

  1. Log in to the master node and make sure all of the files mentioned above have been transferred there, including the root CA trust file cacert.pem. To download the latest version of the root CA trust file, run the command:

    curl -OL https://curl.se/ca/cacert.pem
    
  2. Make sure you have downloaded a copy of cacert.pem per the instructions above. If you have a private root CA, append it to that file:

    cat rootca.crt >> cacert.pem
    
  3. If you have an intermediate bundle, you will need a file that combines that bundle with your main public certificate:

    cat tls.crt intermediate.pem > fullchain.pem
    cat wildcard.crt intermediate.pem > fullchain_wildcard.pem
    

    If you are using LetsEncrypt, you may skip this step, as fullchain.pem has already been provided for you.

  4. Create a file named certificates.yaml, then add the following information to it:

    apiVersion: v1
    kind: Secret
    metadata:
      name: anaconda-enterprise-certs
    type: kubernetes.io/tls
    data:
        tls.crt: TLS_CRT
        tls.key: TLS_KEY
        rootca.crt: ROOT_CA
    ---
    apiVersion: v1
    kind: Secret
    metadata:
      name: anaconda-enterprise-wildcard
    type: kubernetes.io/tls
    data:
        tls.crt: WILDCARD_CRT
        tls.key: WILDCARD_KEY
    

    This file will be used to provide your updated TLS/SSL certificate and key to access the kubernetes cluster. To keep your information secure, use placeholders for the actual values of the certificate and key.

  5. Save your work and close the file.

  6. Replace the placeholders in the certificates.yaml file with an encoded version of the actual certificate and key. Run the commands that fit your environment:

    If you have a single certificate with an intermediate, run the following commands:

    TLS_CRT=$(base64 -i --wrap=0 fullchain.pem)
    TLS_KEY=$(base64 -i --wrap=0 tls.key)
    ROOT_CA=$(base64 -i --wrap=0 cacert.pem)
    echo "s@TLS_CRT@$TLS_CRT@;" "s@TLS_KEY@$TLS_KEY@;" "s@WILDCARD_CRT@$TLS_CRT@;" "s@WILDCARD_KEY@$TLS_KEY@;" "s@ROOT_CA@$ROOT_CA@" | sed -i.bak -f - certificates.yaml
    

    If you are using a LetsEncrypt certificate, run the following commands:

    TLS_CRT=$(base64 -i --wrap=0 fullchain.pem)
    TLS_KEY=$(base64 -i --wrap=0 privkey.pem)
    ROOT_CA=$(base64 -i --wrap=0 cacert.pem)
    echo "s@TLS_CRT@$TLS_CRT@;" "s@TLS_KEY@$TLS_KEY@;" "s@WILDCARD_CRT@$TLS_CRT@;" "s@WILDCARD_KEY@$TLS_KEY@;" "s@ROOT_CA@$ROOT_CA@" | sed -i.bak -f - certificates.yaml
    

    If you have separate certificates for the main and wildcard domains, run the following commands:

    TLS_CRT=$(base64 -i --wrap=0 fullchain.pem)
    TLS_KEY=$(base64 -i --wrap=0 tls.key)
    WILDCARD_CRT=$(base64 -i --wrap=0 fullchain_wildcard.pem)
    WILDCARD_KEY=$(base64 -i --wrap=0 wildcard.key)
    ROOT_CA=$(base64 -i --wrap=0 cacert.pem)
    echo "s@TLS_CRT@$TLS_CRT@;" "s@TLS_KEY@$TLS_KEY@;" "s@WILDCARD_CRT@$WILDCARD_CRT@;" "s@WILDCARD_KEY@$WILDCARD_KEY@;" "s@ROOT_CA@$ROOT_CA@" | sed -i.bak -f - certificates.yaml
    
  7. Verify that your certificates.yaml file now contains the correct certificate and key by running the following command:

    grep -E 'TLS_(CRT|KEY)' certificates.yaml
    

    If the command returns nothing, your certificates.yaml file has been successfully created.

    If the command returns a message, re-verify the file name and its contents are correct, then run the command again.

  8. Run the following command to create a backup of your existing secret:

    kubectl get secret anaconda-enterprise-certs anaconda-enterprise-wildcard -o yaml --export > certificates.orig
    
  9. Remove the existing secrets, then recreate them from the new file by running the commands:

    kubectl delete secret anaconda-enterprise-certs anaconda-enterprise-wildcard
    kubectl create -f certificates.yaml
    
  10. Restart all Workbench system pods by running the command:

    kubectl get pods | grep ap- | cut -d ' ' -f 1 | xargs kubectl delete pods
    

If you are using a gravity cluster, you will need to perform the following additional steps:

  1. Create a file named gravity-certificates.yaml, then add the following information to it:

    apiVersion: v1
    kind: Secret
    metadata:
      name: cluster-tls
    type: Opaque
    data:
        certificate: TLS_CRT
        privatekey: TLS_KEY
    

    This file will be used to provide your updated TLS/SSL certificate and key to access the gravity cluster. To keep your information secure, use placeholders for the actual values of the certificate and key.

  2. Save your work and close the file.

  3. Replace the placeholders in the gravity-certificates.yaml file with an encoded version of the actual certificate and key by running the following commands:

    TLS_CRT=$(base64 -i --wrap=0 fullchain.pem)
    TLS_KEY=$(base64 -i --wrap=0 tls.key)
    echo "s@TLS_CRT@$TLS_CRT@;" "s@TLS_KEY@$TLS_KEY@" | sed -i.bak -f - gravity-certificates.yaml
    

    If you have a separate set of certificates for the ops-center domain, the command will look something like this:

    TLS_CRT=$(base64 -i --wrap=0 fullchain_opscenter.pem)
    TLS_KEY=$(base64 -i --wrap=0 opscenter.key)
    echo "s@TLS_CRT@$TLS_CRT@;" "s@TLS_KEY@$TLS_KEY@" | sed -i.bak -f - gravity-certificates.yaml
    
  4. Verify that your gravity-certificates.yaml file now contains the correct certificate and key by running the following command:

    grep -E 'TLS_(CRT|KEY)' gravity-certificates.yaml
    

    If the command returns nothing, your gravity-certificates.yaml file has been successfully created.

    If the command returns a message, re-verify the file name and its contents are correct, then run the command again.

  5. Run the following command to create a backup of your existing secret:

    kubectl get secret -n kube-system cluster-tls -o yaml --export > gravity-certificates.orig
    
  6. Remove the existing secrets by running the command:

    kubectl delete secret -n kube-system cluster-tls
    
  7. Create new secrets using the gravity-certificates.yaml file you created by running the command:

    kubectl create -n kube-system -f gravity-certificates.yaml