Generating custom Anaconda installers¶
As an Anaconda Enterprise Administrator, you can create custom environments for other users to access. These environments include specific packages and their dependencies. You can then create a custom installer for the environment, that can be shipped to HDFS and used in Spark jobs.
You can also provide users with standard Anaconda and Miniconda installers, Cloudera Manager parcels, and Hortonworks Data Manager management packs. Parcels are prebuilt distributions of Anaconda specifically created to be used inside a Cloudera CDH cluster, and management packs are created to be used inside a Hortonworks (Apache Ambari) cluster.
Custom installers enable IT and Hadoop administrators to maintain close control of a Hadoop cluster while also making these tools available to data scientists who need Python and R libraries. They provide an easy way to ship multiple custom Anaconda distributions to multiple Hadoop clusters.
You distribute installers to users via channels. After you add an installer to a channel, authorized users will be able to access it from their Channels list. Because each custom installer includes the specific packages and their dependencies, when users download them, they’ll have everything they need to use them. See Using installers, parcels and management packs for more information.
To create an environment:
Log in to the console using the Administrator credentials configured after installation.
Select Environments in the top menu to view the list of existing environments, version and package information for each, and when the environment was last updated.
Click on an environment name to view details about the packages included in the environment.
Click Create ** in the upper right corner, give the environment a unique name and click **Save.
NOTE: Environment names can contain alphanumeric characters and underscores only.
Check the channel you want to choose packages from, then select the specific packages–and version of each–you want to include in the installer.
- Click Save in the window banner to create the environment. Anaconda Enterprise resolves all the package dependencies and displays the environment in the list. If there is an issue resolving the dependencies, you’ll be notified and prompted to edit the environment.
You can now use the environment as a basis for creating additional versions of the environment or other environments.
To edit an existing environment:
- Click on an environment name to view it’s details, then click Edit.
- Change the channels and/or packages included in the environment, and enter a version number for the updated package before clicking Save. The new version is displayed in the list of environments.
To copy an environment:
- Enter a unique name for the environment and click Save. The new environment is diplayed in the list of environments.
Now that you’ve created an environment, you can create an installer for it.
To create a custom installer for an environment:
- Select the environment in the list, click the Create installer icon , and select the type of installer you want to create:
Anaconda Enterprise creates the installer and displays it in the Installers list:
- To view the relevant logs, download or delete the installer, click the icon and choose the appropriate command.
If you created a management pack, you’ll need to install it on your Hortonworks HDP cluster and add it to your local Ambari server to make it available to users. For more information, see this blog post about generating custom management packs.
If you created a parcel, you’ll need to install it on your Cloudera CDH cluster to make it available to users:
NOTE: If you are using CDH 5.x, you’ll need to manually download the parcel, move it to the Cloudera Manager node, then configure Cloudera Manager for a local parcel repository. This is because CDH 5.x does not work with TLS 1.2 that Anaconda Enterprise uses to serve the parcel, so you’ll see a protocol version error if you attempt to use AE as a remote parcel repository with CDH 5.x.
If you are using CDH 6.x with parcels, you can configure Anaconda Enterprise as a remote parcel repository, or you can manually download the parcel and configure a local parcel repository.
- In the Installers list, click the parcel name to view it’s details.
- Depending on the version of CDH you are using (see NOTE above), either copy the path to the parcel or download the parcel installer.
- From the Cloudera Manager Admin Console, click the Parcels indicator in the top navigation bar.
- Click the Configuration button on the top right of the Parcels page to display the Parcel Settings.
If you downloaded the parcel from AE in Step 2 above, copy it to the Local Parcel Repository Path you’ve configured for Cloudera Manager.
To configure AE as a remote parcel repository, add the URL you copied in Step 2 above to the Remote Parcel Repository URLs section and click Save Changes.
If automatic downloading and distribution are not enabled, go to the Parcels page and select Distribute to install the parcel across your CDH cluster. The custom-generated Anaconda parcel is now ready to use with Spark or other distributed frameworks on your Cloudera CDH cluster.
For more information, see these instructions from Cloudera.
To access standard Anaconda and Miniconda installers:
Download the installers from https://repo.anaconda.com/ then follow the instructions below to add them to a channel and share them with others.
Adding an installer to a channel
Select Channels to view the list of existing channels, each channel’s owner and when it was last updated.
Select a channel—or create a new channel—and click Upload in the upper right corner to browse for and upload the installer package to the channel.
To make the installer available to others, click Share in the menu on the left and add them as a collaborator to the channel.
NOTE: You can share the installer with multiple users at once by adding their group or role. See Managing roles and groups for more information.
When the user logs in to Anaconda Enterprise, they’ll be able to access the installer by clicking the Installers link from within their Channels list. For more information, see Using installers, parcels and management packs.