Internal Errigal Collaboration Wiki

User Tools

Site Tools

Trace: • extenet • support_request_sco • kla_ticket_escalation_process • alarm_sync_failures • introduction_to_ticketing_from_a_noc_perspective
support:kla_ticket_escalation_process

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
support:kla_ticket_escalation_process [2022/06/14 10:12] – created 10.91.110.100support:kla_ticket_escalation_process [2022/06/15 11:41] (current) – [Overview Of Process] 10.91.110.100
Line 1: Line 1:
 ====== KLA Ticket Escalation Process ====== ====== KLA Ticket Escalation Process ======
- 
-===== THIS IS HEAVILY WORK IN PROGRESS ===== 
 ===== Introduction ===== ===== Introduction =====
-ATC's technical support staff have a standard operational time of ** 7 AM to 8 PM 5 days a week, Monday to Friday**. ATC need to ensure that service affecting outages and issues are still responded to appropriately and efficiently out of hours. ATC provide this cover by having a TSE(Technical Service Engineer) on-call. The TSE on call is rotated amongst the technical support staff in ATC each week.+KLA's technical support staff have an operation time of 24x7 at Ford and need a system that will automatically escalte untouched tickets to a primary and a secondary on-call staff member. 
 + 
 +The escalation process is to be able to send a text and email to the primary on-call after 30 mins of a ticket's inactivity. 
 +After an additional 30 minutes, 
  
-Since beginning using the Errigal platform, ATC requested for a process/mechanism to be put place to escalate/alert on-call technicians to service affecting issues out of hours. Due to this request, a Ticketer Groovlet was created and put in place to escalate tickets to on-call technicians based on ATC's defined criteria. This wiki entry serves as an overview on how the process was implemented, how it works. It will help you in the instance you are tasked with making any adjustments or changes to the process in the future. 
  
 ===== Overview Of Process ===== ===== Overview Of Process =====
-Every time a ticket is created in the Ticketer, in the 'ATC DAS Workflow' with the start state 'Alarm Received'. The 'service level agreement' Groovlet executes to calculate various SLA metrics for that ticket. In this Groovlet is where the Ticket Escalation process is also scripted.+Every time a ticket is created in the Ticketer, in the 'SNMP Trap' with the start state 'Alarm Received'.  
 +The 'IDMS Ticket Grouping' Groovlet executes to calculate various SLA metrics for that ticket.  
 +In this Groovlet is where the Ticket Escalation process is also scripted.
  
-As soon as non-child ticket enter's the 'Alarm Received' state, the ticket escalation process checks if this ticket was created during out of hours(Escalation Time Range) and if so it creates a number of 'Persisted Tasks' to track this ticket. Each Persisted Task contains a closure that executes after a specified time interval, and when executed after the defined duration of time, it checks if the tracked ticket is still in a certain state and if it is still in the escalation time, if so it sends escalations(Email & SMS) to ATC's tech support staff.+As soon as non-child ticket enter's the 'Alarm Received' state, the ticket escalation process creates a number of 'Persisted Tasks' to track this ticket.  
 +Each Persisted Task contains a closure that executes after a specified time interval, and when executed after the defined duration of time, it checks if the tracked ticket is still in a certain state and if it is still in the escalation time, if so it sends escalations(Email & SMS) to KLA's various tech support tiers.
  
-Each Persisted Task created for a ticket tracked for escalation, represents a different level of ticket escalation priority. There are 4 Levels of escalation priority defined for each ticket escalation - **Level 1, 2, 3** and **4**. ATC has the ability to change the contact details associated with each level of priority.+KLA doesn't change the contact details with each level of priority.
  
 Escalation process also considers priority changing by more (child) ticket in same network identifier, by removing the current tasks on the priority updated parent ticket and assign them a new task. Escalation process also considers priority changing by more (child) ticket in same network identifier, by removing the current tasks on the priority updated parent ticket and assign them a new task.
  
 ===== Escalation Levels ===== ===== Escalation Levels =====
-  * **Level 1 Escalation**: A ticket in alarm received 1 Hour -> Email and SMS to On Call Tech and every 15 minutes after this where it remains in that state.+  * **Level 1 Escalation**: A ticket in alarm received 30 mins -> Email and SMS to all primary users
  
-  * **Level 2 Escalation**: A ticket in alarm received 1 Hour and 30 Min -> Email and SMS to Michael Johnson +  * **Level 2 Escalation**: A ticket in alarm received 1 Hour -> Email and SMS to all secondary users 
- +  
-  * **Level 3 Escalation**: A ticket in alarm received 2 Hours -> Email to Chad Dagenhardt & CC's Michael Johnson +
- +
-  * **Level 4 Escalation**: A ticket in alarm received 1 Hour -> Email to GNOC & CC's Chad Dagenhardt+
 ===== Escalation Criteria ===== ===== Escalation Criteria =====
-  * **P1 Tickets** are escalated 24 hours a day, 7 Days a week+  * **All Tickets** are escalated 24 hours a day, 7 Days a week
  
-  * **P2 Tickets** are escalated between the hours 8pm and 7am during weekday, and all day weekend +===== How KLA Changes The Escalation Contacts Details =====
- +
-  * **P3 Tickets** are escalated between all-day weekend and late Friday(20:00--24:00) +
- +
-  * **P4 Tickets** are not considered for escalation +
-   +
-  * **P1 to P3** Tickets are escalated on ATC holiday  +
-===== How ATC Changes The Escalation Contacts Details =====+
  
-When the escalation process was created, there needed to be a mechanism to allow ATC simply change to contact details of the On-Call TSE, as the TSE On-Call is a different person each week, the email and phone number for this escalation level will vary. In order to support this and not have hard-coded contact details in the script, a number of place-holder users where created in the ticketer(see below).+When the escalation process was created, there needed to be a mechanism to allow KLA simply change to contact details of the On-Call primary and secondary Tech Support Engineers. This escalation level will vary.  
 +In order to support this and not have hard-coded contact details in the script, a number of place-holder users where created in the ticketer(see below).
  
 {{:support:escalation_contacts_ticketer.png?800|}} {{:support:escalation_contacts_ticketer.png?800|}}
  
-The Ticket Escalation process keys of off each of these placeholder users in the Ticketer. It uses the contact details in each of these users to send Email and SMS. The key fields used in each user are highlighted below.+The Ticket Escalation process keys of off each of these placeholder users in the Ticketer.  
 +It uses the contact details in each of these users to send Email and SMS.  
 +The key fields used in each user are highlighted below.
  
 {{:support:escalation_user_example.png?800|}} {{:support:escalation_user_example.png?800|}}
  
-Email templates are also used when sending Email escalation, the Ticket escalation process looks up the appropriate email template for a given escalation level e.g Level 1, 2, 3 or 4. The email templates used are shown in the below screenshot.+Email templates are also used when sending Email escalation, the Ticket escalation process looks up the appropriate email template for a given escalation level e.g Level 1, 2, 3 or 4.  
 +The email templates used are shown in the below screenshot.
  
 {{:support:escalation_template_example.png?800|}} {{:support:escalation_template_example.png?800|}}
Line 51: Line 48:
  
 **Ticket reported by as not having escalated**  **Ticket reported by as not having escalated** 
-First thing to check is that the ticket was created in the escalation time-frame, a ticket won't escalate even if it is created one second outside of the escalation timeframe. If the ticket was created in the escalation timeframe, check the priority of the ticket at the time of creation and see if it is eligible for escalation. For example, a customer reports a ticket was created at 02:00 am on Wednesday morning which did not escalate. If the ticket was a priority 3 ticket, these tickets are not escalated outside of the weekend.+First thing to check is that the ticket was created in the escalation time-frame, a ticket won't escalate even if it is created one second outside of the escalation timeframe.  
 +If the ticket was created in the escalation timeframe, check the priority of the ticket at the time of creation and see if it is eligible for escalation.  
 +For example, a customer reports a ticket was created at 02:00 am on Wednesday morning which did not escalate.  
 +If the ticket was a priority 3 ticket, these tickets are not escalated outside of the weekend.
  
 **Query To Check If A Ticket Should Have Escalated** **Query To Check If A Ticket Should Have Escalated**
Line 58: Line 58:
 SELECT SELECT
  t.*,  t.*,
-        ts.status,+    ts.status,
  ts.status_duration_in_seconds  ts.status_duration_in_seconds
 FROM FROM
support/kla_ticket_escalation_process.1655197956.txt.gz · Last modified: 2022/06/14 10:12 by 10.91.110.100

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki