— Anna Dowling 2018/03/08 07:55

As this process relies on RabbitMQ to pass the messages back and forth between the Ticketer and the Footprints Component, the rabbitmq server connection should always be checked to ensure that it is operating correctly and is actually running. RabbitMQ server connection details can be found in the FootprintsConfig.groovy file.

• Check you can connect to rabbit mq in the browser and can view all relevant queues, exchanges and bindings defined in the Ticketer and Footprints config.groovy files.
  • http://atclb1.atc:15672/

• Check rabbitmq is running on the server. RabbitMQ runs on the distributor server currently set as atclb1.

Check rabbitmq is running:

ps -edf | grep rabbit

Restart rabbitmq if necessary:

rabbitmq-server start

If any email sending issues are encountered either with sending the emails out or polling for updates coming back into the Footprints Component and the Ticketer the first thing to do would be checking the email credentials for the configured mailbox:

Login credentials for this mailbox can be found in password safe under the heading: ticketing@atcwatchdog.com

The url to login to this mailbox is as follows: https://atcwatchdog.awsapps.com/mail

Email Configuration parameters should also be checked in the footprints_component database:

+----+---------+---------------------------+---------------------------------+----------------+------------------+---------------------------+-------------------------------+------------------+
| id | version | email_from                | name                            | password       | activesmtpconfig | user_name                 | email_to                      | activeimapconfig |
+----+---------+---------------------------+---------------------------------+----------------+------------------+---------------------------+-------------------------------+------------------+
|  1 |       0 | ticketing@atcwatchdog.com | Footprints Active Configuration | #ATCemail321!# |                 | ticketing@atcwatchdog.com | atc_noc_das@americantower.com |                  |
|  2 |       0 | n/a                       | Footprints IMAP Configuration   | #ATCemail321!# |                  | ticketing@atcwatchdog.com | atc_noc_das@americantower.com |                 |
+----+---------+---------------------------+---------------------------------+----------------+------------------+---------------------------+-------------------------------+------------------+

The logs should also be checked for any exceptions:

~/logs/grails/footprints.log

If an update is not received back from American Towers mailbox to the Errigal Inbox, the next step to take is verify that the update was sent by American Tower as this could potentially be an issue on their side.

The POC for American Tower currently for this process is Aaron Carey: aaron.carey@americantower.com

Grails App process monitoring is configured for the footprints_component microservice via prometheus.

We will receive Appdown alerts if the application process is unavailable on either handlers (atcapps1 & atcapps2) for more than 120 seconds as per the current threshold.