10.6. Debugging and Logging for a YARN-Based Cluster
- Once the YARN application is launched, you can monitor the status at
YARN ResourceManager WebUI.
- A successfully launched application will be in
RUNNING state. The
YARN ApplicationMaster UI (eg:
http://master:8088/cluster/app/application_<id>) will show
slider-appmaster, COORDINATOR and WORKER components and the
associated containers running based on your configuration. You can
also use Slider cli script to Check the Status.
- Slider retries to launch Presto in case of failure in the same YARN application.
The YARN application will be still in
RUNNING state during this retry phase.
It ultimately kills the job after 5 unsuccessful retrials.
- If you have used Using YARN label your COORDINATOR and WORKER
components will be running on nodes which were ‘labeled’.
- If you have not used labels, then you can check the status either at
the YARN ResourceManager (eg:
http://master:8088/cluster/app/application_<id>) or you can use
Check the Status to get the “live” containers, and thus get the
node hosting the Presto components.
- If Presto is up and running, then a
pgrep of PrestoServer on your
NodeManager nodes will give you the process details. This should also
give the directory Presto is installed and the configuration files
used by Presto.
- If the YARN application has failed to launch Presto, then you may want to
take a look at the slider logs created under YARN log directory for the
corresponding application. It is recommended that log aggregation of YARN application log
files be enabled in YARN, using
yarn.log-aggregation-enable property in your
Then slider logs created during the launch of Presto-YARN will be
available locally on your nodemanager nodes (where slider-appmaster
and Presto components-COORDINATOR/WORKER are deployed) under
contanier logs directory eg:
/var/log/hadoop-yarn/application_<id>/container_<id>/. For any
retries attempted by Slider to launch Presto a new container will be
launched and hence you will find a new
You can look for any errors under
errors_*.txt there, and also
there is a
slider-agent.log file which will give you Slider
application lifetime details.
Subsequently every Slider application owner has the flexibility to
set the include and exclude patterns of file names that they intend
to aggregate, by adding the following properties in their
resources.json. For example, using
- If there are no errors in
slider.log then you may want to look at Presto
logs for any errors. Presto logs will be available under the standard Presto data
directory location. By default it is
/var/lib/presto/data is the default data
site.global.data_dir configured in Slider
You can find both
http-request.log files here. Please note that
log rotation of these Presto log files will have to be manually
enabled (for eg: using
- Presto configuration files will be at
directory if you are using the default
site.global.config_dir. The configuration files here will be
generated by Slider and overwritten for every application restart.
These files should NOT be modified manually.