==============================================
Debugging and Logging for a YARN-Based Cluster
==============================================

-  Once the YARN application is launched, you can monitor the status at
   YARN ResourceManager WebUI.

-  A successfully launched application will be in ``RUNNING`` state. The
   YARN ApplicationMaster UI (eg:
   ``http://master:8088/cluster/app/application_<id>``) will show
   slider-appmaster, COORDINATOR and WORKER components and the
   associated containers running based on your configuration. You can
   also use Slider cli script to :ref:`check-status-label`.

-  Slider retries to launch Presto in case of failure in the same YARN application.
   The YARN application will be still in ``RUNNING`` state during this retry phase.
   It ultimately kills the job after 5 unsuccessful retrials. 

-  If you have used :ref:`using-yarn-label` your COORDINATOR and WORKER
   components will be running on nodes which were 'labeled'.

-  If you have not used labels, then you can check the status either at
   the YARN ResourceManager (eg:
   ``http://master:8088/cluster/app/application_<id>``) or you can use
   :ref:`check-status-label` to get the "live" containers, and thus get the
   node hosting the Presto components.

-  If Presto is up and running, then a ``pgrep`` of PrestoServer on your
   NodeManager nodes will give you the process details. This should also
   give the directory Presto is installed and the configuration files
   used by Presto.

-  If the YARN application has failed to launch Presto, then you may want to 
   take a look at the slider logs created under YARN log directory for the
   corresponding application. It is recommended that log aggregation of YARN application log
   files be enabled in YARN, using
   ``yarn.log-aggregation-enable property`` in your ``yarn-site.xml``.
   Then slider logs created during the launch of Presto-YARN will be
   available locally on your nodemanager nodes (where slider-appmaster
   and Presto components-COORDINATOR/WORKER are deployed) under
   contanier logs directory eg:
   ``/var/log/hadoop-yarn/application_<id>/container_<id>/``. For any
   retries attempted by Slider to launch Presto a new container will be
   launched and hence you will find a new ``container_<id>`` directory.
   You can look for any errors under ``errors_*.txt`` there, and also
   there is a ``slider-agent.log`` file which will give you Slider
   application lifetime details.
   Subsequently every Slider application owner has the flexibility to
   set the include and exclude patterns of file names that they intend
   to aggregate, by adding the following properties in their
   ``resources.json``. For example, using

.. code-block: none

     "global": {
        "yarn.log.include.patterns": "*",
        "yarn.log.exclude.patterns": "*.*out"
      }

See
http://slider.incubator.apache.org/docs/configuration/resources.html#logagg
for details.

-  If there are no errors in ``slider.log`` then you may want to look at Presto
   logs for any errors. Presto logs will be available under the standard Presto data
   directory location. By default it is ``/var/lib/presto/data/var/log``
   directory where ``/var/lib/presto/data`` is the default data
   directory ``site.global.data_dir`` configured in Slider ``appConfig.json``.
   You can find both ``server.log`` and ``http-request.log`` files here. Please note that
   log rotation of these Presto log files will have to be manually
   enabled (for eg: using
   http://linuxcommand.org/man_pages/logrotate8.html)

-  Presto configuration files will be at ``/var/lib/presto/etc``
   directory if you are using the default ``appConfig.json`` property
   ``site.global.config_dir``. The configuration files here will be
   generated by Slider and overwritten for every application restart.
   These files should NOT be modified manually.