appConfig.json

• site.global.app_user (default - yarn): This is the user which will be launching the YARN application for Presto. So all the Slider commands (using bin/slider script) will be run as this user. Make sure that you have a HDFS home directory created for the app_user. Eg: for user yarn create /user/yarn with yarn user as an owner.

hdfs dfs -mkdir -p /user/yarn
hdfs dfs -chown yarn:yarn /user/yarn

Note: For operations involving Hive connector in Presto, especially INSERT, ALTER TABLE etc, it may require that the user running Presto has access to HDFS directories like Hive warehouse directories. So make sure that the app_user you set has appropriate access permissions to those HDFS directories. For eg: /apps/hive/warehouse is usually where Presto user will need access for various DML operations involving Hive connector and is owned by hdfs in most cases. In that case, one way to fix the permission issue is to set site.global.app_user to user hdfs and also create /user/hdfs directory in HDFS if not already there (as above). You will also need to run any slider scripts(bin/slider) as user hdfs in this case.

• site.global.user_group (default - hadoop): The group owning the application.
• site.global.data_dir (default - /var/lib/presto/data): The data directory configured should be pre-created on all nodes and must be owned by user yarn, otherwise slider will fail to start Presto with permission errors.
• site.global.singlenode (default - true): If set to true, the node used act as both coordinator and worker (singlenode mode). For multi-node set up, this should be set to false.
• site.global.presto_query_max_memory (default - 5GB): This will be used as query.max-memory in Presto’s config.properties file.
• site.global.presto_query_max_memory_per_node (default - 600MB): This will be used as query.max-memory-per-node in Presto’s config.properties file.
• site.global.presto_server_port (default - 8080): Presto server’s http port.
• site.global.catalog (optional) (default - configures tpch connector): It should be of the format (note the single quotes around each value) - {‘connector1’ : [‘key1=value1’, ‘key2=value2’..], ‘connector2’ : [‘key1=value1’, ‘key2=value2’..]..}. This will create files connector1.properties, connector2.properties for Presto with entries key1=value1 etc.

"site.global.catalog": "{'hive': ['connector.name=hive-cdh5', 'hive.metastore.uri=thrift://${NN_HOST}:9083'], 'tpch': ['connector.name=tpch']}"

Note: The NN_HOST used in hive.metastore.uri is a variable for your HDFS Namenode and this expects that your hive metastore is up and running on your HDFS Namenode host. You do not have to replace that with your actual Namenode hostname. This variable will be substituted with your Namenode hostname during runtime. If you have hive metastore running elsewhere make sure you update NN_HOST with the appropriate hostname.

• site.global.jvm_args (default - as in example below): This configures Presto jvm.config file and default heapsize is 1GB. Since Presto needs the jvm.config format to be a list of options, one per line, this property must be a String representation of list of strings. Each entry of this list will be a new line in your jvm.config. For example the configuration should look like:

"site.global.jvm_args": "['-server', '-Xmx1024M', '-XX:+UseG1GC', '-XX:G1HeapRegionSize=32M', '-XX:+UseGCOverheadLimit', '-XX:+ExplicitGCInvokesConcurrent', '-XX:+HeapDumpOnOutOfMemoryError', '-XX:OnOutOfMemoryError=kill -9 %p']",

• site.global.additional_node_properties and site.global.additional_config_properties (optional) (default - None): Presto launched via Slider will use config.properties and node.properties created from templates presto-yarn-package/package/templates/config.properties*.j2 and presto-yarn-package/package/target/node.properties.j2 respectively. If you want to add any additional properties to these configuration files, add site.global.additional_config_properties and site.global.additional_node_properties to your appConfig.json. The value of these has to be a string representation of an array of entries (key=value) that has to go to the .properties file. Eg:

"site.global.additional_config_properties": "['task.max-worker-threads=50', 'distributed-joins-enabled=true']"

• site.global.plugin (optional) (default - None): This allows you to add any additional jars you want to copy to plugin presto-server-<version>/plugin/<connector> directory in addition to what is already available there. It should be of the format {‘connector1’ : [‘jar1’, ‘jar2’..], ‘connector2’ : [‘jar3’, ‘jar4’..]..}. This will copy jar1, jar2 to Presto plugin directory at plugin/connector1 directory and jar3, jar4 at plugin/connector2 directory. Make sure you have the plugin jars you want to add to Presto available at presto-yarn-package/src/main/slider/package/plugins/ prior to building the presto-yarn app package and thus the app package built presto-yarn-package-<version>-<presto-version>.zip will have the jars under package/plugins directory.

"site.global.plugin": "{'ml': ['presto-ml-${presto.version}.jar']}",

• java_home (default - /usr/lib/jvm/java): Presto requires Java 1.8. So make jdk8 the default java or add it to java_home here
• Variables in appConfig.json like ${COORDINATOR_HOST}, ${AGENT_WORK_ROOT} etc. do not need any substitution and will be appropriately configured during runtime.

resources.json

The configuration here can be added either globally (for COORDINATOR and WORKER) or for each component. Refer configuration section for further details.

• yarn.vcores (default - 1): By default this is set globally.
• yarn.component.instances (default - 1 for COORDINATOR and 3 for WORKER): The multinode presto-yarn-package/src/main/resources/resources-multinode.json sample file is now configured for a 4 node cluster where there will be 1 coordinator and 3 workers with strict placement policy, meaning, there will be one component instance running on every node irrespective of failure history. If there are insufficient number of nodemanager nodes in your cluster to accomodate the number of workers requested, the application launch will fail. The number of workers could be number of nodemanagers in your cluster - 1, with 1 node reserved for the coordinator, if you want Presto to be on all YARN nodes. If you want to deploy Presto on a single node (site.global.singlenode set to true), make sure you set 1 for the COORDINATOR and just not add the WORKER component section (Refer presto-yarn-package/src/main/resources/resources-singlenode.json). You can also just set yarn.component.instances to 0 for WORKER in this case.
• yarn.memory (default - 1500MB): The heapsize defined as -Xmx of site.global.jvm_args in appConfig.json, is used by the Presto JVM itself. Slider suggests that the value of yarn.memory must be bigger than this heapsize. The value of yarn.memory MUST be bigger than the heap size allocated to any JVM and Slider suggests using atleast 50% more appears to work, though some experimentation will be needed.
• yarn.label.expression (optional) (default - coordinator for COORDINATOR and worker for WORKER``): See label section for details.

Now you are ready to deploy Presto on YARN either manually or via Ambari.

Additional Slider commands

Some additional slider commands to manage your existing Presto application.

bin/slider status presto1 --out status_file

bin/slider destroy presto1
bin/slider create presto1 --template appConfig.json --resources resources.json

bin/slider flex presto1 --component WORKER 2

Reconfiguration in Slider View

Once the application is launched if you want to update the configuration of Presto (eg: add a new connector), first go to Actions on the Slider View instance screen and stop the running application.

Once the running YARN application is stopped, under Actions you will have an option to Destroy the existing Presto instance running via Slider. Destroy the existing one and re-create a new app (Create App button) with whatever updates you want to make to the configuration.

Configuring memory and CPU

Memory and CPU related configuration properties must be modified as per your cluster configuration and requirements.

Memory

yarn.memory in resources.json declares the amount of memory to ask for in YARN containers. It should be defined for each component, COORDINATOR and WORKER based on the expected memory consumption, measured in MB. A YARN cluster is usually configured with a minimum container allocation, set in yarn-site.xml by the configuration parameter yarn.scheduler.minimum-allocation-mb. It will also have a maximum size set in yarn.scheduler.maximum-allocation-mb. Asking for more than this will result in the request being rejected.

The heapsize defined as -Xmx of site.global.jvm_args in appConfig.json, is used by the Presto JVM itself. Slider suggests that the value of yarn.memory must be bigger than this heapsize. The value of yarn.memory MUST be bigger than the heap size allocated to any JVM and Slider suggests using atleast 50% more appears to work, though some experimentation will be needed.

In addition, set other memory specific properties presto_query_max_memory and presto_query_max_memory_per_node in appConfig.json as you would set the properties query.max-memory and query.max-memory-per-node in Presto’s config.properties.

CPU

Slider also supports configuring the YARN virtual cores to use for the process which can be defined per component. yarn.vcores declares the number of “virtual cores” to request. Ask for more vcores if your process needs more CPU time.

See http://slider.incubator.apache.org/docs/configuration/resources.html#core for more details.

CGroups in YARN

If you are using CPU scheduling (using the DominantResourceCalculator), you should also use CGroups to constrain and manage CPU processes. CGroups compliments CPU scheduling by providing CPU resource isolation. With CGroups strict enforcement turned on, each CPU process gets only the resources it asks for. This way, we can guarantee that containers hosting Presto services is assigned with a percentage of CPU. If you have another process that needs to run on a node that also requires CPU resources, you can lower the percentage of CPU allocated to YARN to free up resources for the other process.

See Hadoop documentation on how to configure CGroups in YARN: https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html. Once you have CGroups configured, Presto on YARN containers will be configured in the CGroups hierarchy like any other YARN application containers.

Slider can also define YARN queues to submit the application creation request to, which can set the priority, resource limits and other values of the application. But this configuration is global to Slider and defined in conf/slider-client.xml. You can define the queue name and also the priority within the queue. All containers created in the Slider cluster will share this same queue.

<property>
  <name>slider.yarn.queue</name>
  <value>default</value>
</property>

<property>
  <name>slider.yarn.queue.priority</name>
  <value>1</value>
</property>

Failure policy

Follow this section if you want to change the default Slider failure policy. Yarn containers hosting Presto may fail due to some misconfiguration in Presto or some other conflicts. The number of times the component may fail within a failure window is defined in resources.json.

The related properties are:

• The duration of a failure window, a time period in which failures are counted. The related properties are yarn.container.failure.window.days, yarn.container.failure.window.hours, yarn.container.failure.window.minutes and should be set in the global section as it relates just to slider. The default value is yarn.container.failure.window.hours=6. The initial window is measured from the start of the slider application master —once the duration of that window is exceeded, all failure counts are reset, and the window begins again.
• The maximum number of failures of any component in this time period. yarn.container.failure.threshold is the property for this and in most cases, should be set proportional to the the number of instances of the component. For Presto clusters, where there will be one coordinator and some number of workers it is reasonable to have a failure threshold for workers more than that of coordinator. This is because a higher failure rate of worker nodes is to be expected if the cause of the failure is due to the underlying hardware. At the same time the threshold should be low enough to detect any Presto configuration issues causing the workers to fail rapidly and breach the threshold sooner.

These failure thresholds are all heuristics. When initially configuring an application instance, low thresholds reduce the disruption caused by components which are frequently failing due to configuration problems. In a production application, large failure thresholds and/or shorter windows ensures that the application is resilient to transient failures of the underlying YARN cluster and hardware.

Based on the placement policy there are two more failure related properties you can set.

• The configuration property yarn.node.failure.threshold defines how “unreliable” a node must be before it is skipped for placement requests. This is only used for the default yarn.component.placement.policy where unreliable nodes are avoided.
• yarn.placement.escalate.seconds is the timeout after which slider will escalate the request of pending containers to be launched on other nodes. For strict placement policy where the requested components are deployed on all nodes, this property is irrelevant. For other placement policies this property is relevant and the higher the cost of migrating a component instance from one host to another, the longer value of escalation timeout is recommended. Thus slider will wait longer before the component instance is escalated to be started on other nodes. During restart, for cases where redeploying the component instances on the same node as before is beneficial (due to locality of data or similar reasons), a higher escalation timeout is recommended.

Take a look here: http://slider.incubator.apache.org/docs/configuration/resources.html#failurepolicy for more details on failure policy.

Using YARN label

This is an optional feature and is not required to run Presto in YARN. To guarantee that a certain set of nodes are reserved for deploying Presto or to configure a particular node for a component type we can make use of YARN label expressions. To create and assign node to labels refer the below link. http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_yarn_resource_mgt/content/ch_node_labels.html

Let’s say you have an 8 node cluster with

• 3 master nodes - m1, m2, m3
• 3 data nodes - d1, d2, d3
• 2 edge nodes - e1, e2 (edge nodes are basically nodes with various client like Hbase, Hive, Oozie, etc)

Note: YARN labels can only be assigned to nodes which have YARN NodeManager service running on them. Generally data nodes are the nodes which have NodeManager service running. Without yarn labels, when we deploy presto-on-yarn the presto-coordinator/worker can be on any of the nodes d1, d2, d3. In case, we want coordinator to be always on d1 and workers on d2 and d3, we can create 2 yarn labels namely “Coordinator” and “Worker”

Assign Coordinator to d1 and Worker to d2 and d3. Now Presto coordinator will be always be running on d1. For assigning YARN labels to nodes, you can either use the Ambari Slider View( Create App –> Allocate Resource –> YARN Labels) or add the label expression to resources.json if installing using Slider cli.

• First assign the nodes/subset of nodes with appropriate labels.
• Then set the components in resource.json with yarn.label.expression to have labels to be used when allocating containers for Presto.
• Create the application using bin/slider create .. --queue <queuename>. queuename will be the queue defined in step one for the appropriate label.

If a label expression is specified for the slider-appmaster component then it also becomes the default label expression for all component. Sample resources.json may look like:

"COORDINATOR": {
  "yarn.role.priority": "1",
  "yarn.component.instances": "1",
  "yarn.component.placement.policy": "1",
  "yarn.label.expression":"coordinator"
},
"WORKER": {
  "yarn.role.priority": "2",
  "yarn.component.instances": "2",
  "yarn.component.placement.policy": "1",
  "yarn.label.expression":"worker"
}

where coordinator and worker are the node labels created and configured with a scheduler queue in YARN