Chicago Department of Transport - Errigal Digger Support in Watchdog/Dig Portal

Author: David McGee

• CDOT operates a web service which allows us to retrieve unacknowledged dig tickets via SOAP. From here, the destination is the Crown Castle Dig Portal to enable dig tech operators to locate or clear these tickets.

• Technical questions regarding the Chicago Department of Transport (CDOT) API and service should be directed to the official web services team at DiggerTechQuestions@cityofchicago.org
  • All non-technical requests, continue to email these to DiggerProject@cityofchicago.org

• Watchdog performs this check every 15 minutes on digportal1a using a “SOAP Get Request” command

• All dig tickets returned in the get request response are converted to emails and sent to the specified Dig Portal mailbox using the configured SMTP mailbox

• Once each dig ticket is parsed and sent as an email: the relevant ID is tracked

• For all successfully emailed tickets, Errigal sends an acknowledgement back to CDOT to let them know we have received and processed these tickets. This removes the relevant dig tickets from the unacknowledged pending dig tickets queue.

• This brings the tickets into the normal flow for the Dig Portal Ticketer to parse them as emails, which results in tickets in the Dig Portal.

• All parameters used by the operation are configured in the ‘ChicagoSmtpMailbox’ and ‘ChicagoDepartmentOfTransportDigWebService’ resources. For the most part, every part of the operation can be externally configured in ~/watchdog/resources/ResourceConfig.groovy

1. ChicagoDepartmentOfTransportDigWebService
   1. Service End-point URL (Web Service URL)
   2. SOAP Action URL
   3. Connect and read timeouts
   4. Associated SMTP Mailbox (Used to send dig tickets found in SOAP messages to Dig Portal mailbox)
   5. XML Request (The message to send to the web service to retrieve all pending dig tickets for our customer)
   6. XML Response Node (The node in the XML response from CDOT that contains info about dig tickets)
   7. Result Delimiter (How we split up separate dig tickets in the response received)
   8. XML Acknowledgement Token (The token to add to the XML Acknowledgement for every dig ticket that has been successfully emailed to the Dig Portal)
   9. XML Acknowledgement Request (The main body of the XML Acknowledgement request for letting CDOT know we’ve received these tickets)
   10. XML Acknowledgement Response Node (The node in the XML response from CDOT that contains the status of our acknowledgement)

1. ChicagoSmtpMailbox
   1. This is the mailbox used to send the converted SOAP dig tickets to the Dig Portal application mailbox for parsing into the Dig Portal
   2. Most parameters here are obvious but the important ones are:
      1. username (the mailbox account to use for sending the emails)
      2. emailTo (the mailbox account that the emails are sent to - should really be a Dig Portal application mailbox)

The current production setup for Crown Castle involves all mails being sent to digportal@nextgnoc.net from fiberdig@errigal.com

• Is the Dig Portal mailbox receiving emails from the Watchdog account?
  • Check ~/watchdog/logs for today’s log on digportal1a. Are there error messages relating to emails being sent?
  • Is the fiberdig@errigal.com showing any Dig Ticket mails in it’s sent folder? (See subject format below)
  • Does the digportal@nextgnoc.net account have any mails from fiberdig@errigal.com in it’s “DigTickets” folder?

• These mails appear in the “DigTickets” folder of the mailbox as follows
• It is important to note that as of Monday, 14th December 2015, the rules in the Dig Portal email parsing rule for Chicago don’t create tickets for Renew, Remark, NoShow or Cancel tickets.

• If the above all checks out, and there is still issues with dig tickets appearing in the Dig Portal. Check out the following. This will likely assist with the diagnosis of any issue through to resolution.

• Are there any errors in the Ticketer logs for email parsing?

• Log into the MySQL RDS instance for the ticketer on the cloud
  • Use ticketer

• Make use of the dig_ticket_email_summary table. This tracks all emails that the Ticketer attempts to parse. This helps for auditing purposes!
  • Make sure to key off of the “fiberdig@errigal.com” from_address

• It is wise to use a small range for received_date to ensure you don’t get 1000s of results.

• Utilise the “parsed” flag. If you receive 100 emails from Chicago, we should be parsing 100 of them. Otherwise, we have a email parsing rule error.

• Utilise the “is_ticketable_email” flag. This will help us determining if the emails we’re receiving should even be creating tickets (For example: Renews/Remarks/NoShows won’t create tickets. Initial tickets will create tickets).
  • We would hope that is_ticketable_email=true entries would have a corresponding ticket_id entry.

• Alternatively, there could be issues with the web service itself or the associated mailbox
  • This should result in watchdog “UnableToRunCommand” failures to developers@errigal.com

• This is basically the watchdog going from being not able to hit the Chicago SOAP gateway to it coming back. It could be down to network, unsolicited issues on gateway, maintenance, etc. Impact is low provided that it returns as nothing is pulled from the queue until Errigal acknowledge it. Sometimes this will auto-clear within a matter of minutes.

• In the event of needing tech support from CDOT, contact Digger Support diggertechquestions@gmail.com