10.1. Automated Installation on a YARN-Based Cluster

Deploying Presto on a YARN-Based Cluster

The installation procedures assume that you have a basic knowledge of Presto and the configuration files and properties it uses.

Using Ambari Slider View to Install Presto on a YARN-Based Cluster

Ambari supports deploying Slider application packages using Slider View and provides Slider integration. Slider View for Ambari allows you to deploy and manage Slider apps from Ambari Web.

Use Ambari Slider View and the following steps to deploy Presto on YARN:

1. Install the Ambari server. See: http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_Installing_HDP_AMB/content/ch_Installing_Ambari.html.
2. Download the Apache Slider package. See: https://slider.incubator.apache.org/
3. Copy the Presto app package presto-yarn-package-<version>-<presto-version>.zip to /var/lib/ambari-server/resources/apps/ directory on your Ambari server node.
4. Restart ambari-server.
5. Log on to Apache Ambari, http://ambariserver_ip:8080 #username-admin password-admin
6. Name your cluster, provide the configuration of the cluster, and follow the steps on the WebUI.
7. Customize/configure the services and install them. A minimum of HDFS, YARN, Zookeeper is required for Slider to work. You must also select Slider to be installed.
8. For the Slider client installed, you need to update its configuration if you are not using the default installation paths for Hadoop and Zookeeper. Thus slider-env.sh should point to your JAVA_HOME and HADOOP_CONF_DIR

export JAVA_HOME=/usr/lib/jvm/java
export HADOOP_CONF_DIR=/etc/hadoop/conf

9. For zookeeper, if you are using a different installation directory from the default one at /usr/lib/zookeeper:

   • Add a custom property to the slider-client section in Slider configuration with the key: zk.home and value: path_to_your_zookeeper.
   • If using a different port from the default 2181, then add the key slider.zookeeper.quorum and value: master:5181, where master is the node and 5181 is the port.

10. Once you have all the services up and running on the cluster, you can configure Slider in Ambari to create and manage your application by creating a “View”.

    1. Go to admin (top right corner) -> Manage Ambari.
    2. From the left pane, select Views.
    3. Create a Slider View by populating all the necessary fields with a preferred instance name (for example, Slider). ambari.server.url can be of the format http://<ambari-server-url>:8080/api/v1/clusters/<clustername>, where <clustername> is what you have named your Ambari cluster.
    4. Select the “Views” control icon in the upper right.
    5. Select the instance you created in the previous step (for example, “Slider”).
    6. Click Create App to create a new Presto YARN application.

11. Provide details of the Presto service. By default, the UI will be populated with the values you have in the *-default.json files in your presto-yarn-package-*.zip.

12. The app name should be of lower case. For example: presto1.

13. You can set the configuration property fields required for your cluster. For example, if you want to set a connector for Presto, you can update the global.catalog property. See the following for an explanation of each configuration property.

    Presto Configuration Options for YARN-Based Clusters

14. Prepare HDFS for Slider. The user directory you create here should be for the same user you set in the global.app_user field. If the app_user is going to be yarn then do the following:

su hdfs hdfs dfs -mkdir -p /user/yarn
su hdfs hdfs dfs -chown yarn:yarn /user/yarn

15. Change the global.presto_server_port from 8080 to another unused port, for example, 8089, since Ambari by default uses 8080.
16. Pre-create the data directory in the UI (added in appConfig-default.json eg: /var/lib/presto/) on all nodes. The directory must be owned by global.app_user, otherwise Slider will fail to start Presto due to permission errors.

mkdir -p /var/lib/presto/data
chown -R yarn:hadoop /var/lib/presto/data

18. If you want to add any additional custom properties, use the Custom property section. Additional properties currently supported are:

    • site.global.plugin
    • site.global.additional_config_properties
    • site.global.additional_node_properties.

    For the requirements and format of these properties, see:

    Presto Configuration Options for YARN-Based Clusters

19. Click Finish. This is the equivalent of package  --install and create performed with the bin/slider script. If successfully deployed, you will see the YARN application started for Presto. You can do the following:

    • Click app launched and monitor the status from Slider view.
    • Click``Quick Links``, which should take you to the YARN WebUI.

    If your application is running successfully, it should always be available in the YARN resource manager as a “RUNNING” application.

20. If the job fails, check the job history’s logs and the logs on the node’s disk. See Debugging and Logging for YARN-Based Clusters.

21. You can manage the application lifecycle (for example: start, stop, flex, and destroy) from the View UI.

Additional Configuration Options

After you install Presto and Slider, you can reconfigure Presto or perform additional configuration.

Debugging and Logging

For more information, see Debugging and Logging for YARN-Based Clusters.

Links

• https://github.com/prestodb/presto-yarn/blob/master/README.md
• http://slider.incubator.apache.org/docs/getting_started.html
• http://docs.hortonworks.com/HDPDocuments/Ambari-2.0.1.0/bk_Installing_HDP_AMB/content/ch_Installing_Ambari.html