10.6. Debugging and Logging for a YARN-Based Cluster

  • Once the YARN application is launched, you can monitor the status at YARN ResourceManager WebUI.
  • A successfully launched application will be in RUNNING state. The YARN ApplicationMaster UI (eg: http://master:8088/cluster/app/application_<id>) will show slider-appmaster, COORDINATOR and WORKER components and the associated containers running based on your configuration. You can also use Slider cli script to Check the Status.
  • Slider retries to launch Presto in case of failure in the same YARN application. The YARN application will be still in RUNNING state during this retry phase. It ultimately kills the job after 5 unsuccessful retrials.
  • If you have used Using YARN label your COORDINATOR and WORKER components will be running on nodes which were ‘labeled’.
  • If you have not used labels, then you can check the status either at the YARN ResourceManager (eg: http://master:8088/cluster/app/application_<id>) or you can use Check the Status to get the “live” containers, and thus get the node hosting the Presto components.
  • If Presto is up and running, then a pgrep of PrestoServer on your NodeManager nodes will give you the process details. This should also give the directory Presto is installed and the configuration files used by Presto.
  • If the YARN application has failed to launch Presto, then you may want to take a look at the slider logs created under YARN log directory for the corresponding application. It is recommended that log aggregation of YARN application log files be enabled in YARN, using yarn.log-aggregation-enable property in your yarn-site.xml. Then slider logs created during the launch of Presto-YARN will be available locally on your nodemanager nodes (where slider-appmaster and Presto components-COORDINATOR/WORKER are deployed) under contanier logs directory eg: /var/log/hadoop-yarn/application_<id>/container_<id>/. For any retries attempted by Slider to launch Presto a new container will be launched and hence you will find a new container_<id> directory. You can look for any errors under errors_*.txt there, and also there is a slider-agent.log file which will give you Slider application lifetime details. Subsequently every Slider application owner has the flexibility to set the include and exclude patterns of file names that they intend to aggregate, by adding the following properties in their resources.json. For example, using

See http://slider.incubator.apache.org/docs/configuration/resources.html#logagg for details.

  • If there are no errors in slider.log then you may want to look at Presto logs for any errors. Presto logs will be available under the standard Presto data directory location. By default it is /var/lib/presto/data/var/log directory where /var/lib/presto/data is the default data directory site.global.data_dir configured in Slider appConfig.json. You can find both server.log and http-request.log files here. Please note that log rotation of these Presto log files will have to be manually enabled (for eg: using http://linuxcommand.org/man_pages/logrotate8.html)
  • Presto configuration files will be at /var/lib/presto/etc directory if you are using the default appConfig.json property site.global.config_dir. The configuration files here will be generated by Slider and overwritten for every application restart. These files should NOT be modified manually.