Level: Major

Purpose: Notify operations that an ATC Das ticket was not created in Salesforce

Scenario: More than one ATC DAS ticket was not created in Salesforce for the previous 7 days.

Resolution: After the 3.21 release a API will be available to recreate the missing link.

Ticketer/api/v1/integration/salesforce/link/${ticketId}

This can be completed in a Postman POST request or using the below cURL command:

curl --request POST -i -u '<USERNAME>':'<PASSWORD>' <DOMAIN>/Ticketer/api/v1/integration/salesforce/link/<TICKET-ID>

For ATC

curl --request POST -i -u '<USERNAME>':'<PASSWORD>' http://atcwatchdog.com/Ticketer/api/v1/integration/salesforce/link/<TICKET-ID>

The username and password is any username or password that has ticketer access on that environment. Before this is implemented the only way an alert will be cleared is if ATC DAS ticket is auto cleared.

Manual Action Steps: Check for ATC Tickets with no Salesforce link

  select
      t.*
  from
      ticketer.ticket t
          left join
      ticketer.external_source es ON es.ticket_id = t.id
      	left join
      snmp_manager.network_element child on child.id = t.network_element_id
      	left join
      snmp_manager.network_element parent on parent.id = child.parent_id
      	left join
      snmp_manager.network_element grandparent on grandparent.id = parent.parent_id
  where
      es.ticket_id is null
      and t.creator_id = 4 -- IDMS Monitor
      and t.workflow_id = 909 -- ATC DAS Workflow
      and t.create_date <= now() - interval 1 MINUTE
      and (t.create_date >= now() - Interval 7 day)
      and t.parent_id is null
      and (parent.name != 'ORPHANS' and grandparent.name != 'ORPHANS');

To fix the problem you will have to requeue the dropped messages Here is a template for this request.

  {
      "event_type": "PostInsert",
      "domain": "f_snmp_trap",
      "CN_Priority__c": "2",
      "Due_Date__c": null,
      "SLA_Date__c": null,
      "Originating_System_ID__c": "ERRIGALTICKET"
      "Description": " ",
      "Alarm_Start_Date__c": "",
      "Alarm_Type__c": "errigalMonitoredCarrierDeviceMissingAlarm",
      "trap_name": "errigalMonitoredCarrierDeviceMissingAlarm",
      "Alarm_System_Link__c": "https://atcwatchdog.com/SnmpManager/errigalEMS#parent/223393",
      "Alarm_IP_Source__c": "10.32.30.57",
      "INT_Integration_ID__c": "223393",
      "Optima_Network_Element__c": "209694 MBTA - 209694 MBTA - Solid DMS#2"
  }

You will need to replace the IP, NE ID, Description, Errigal Ticket, Alarm Start Date, Network Element and others.

1. Once you have edited the json with the appropriate values from the query, then you can try requeue it.
2. Go to http://atclb1.atc:15672
3. Click on the exchanges tab
4. Find `ticketer_audit_event_exchange_topic`
5. Under publish, use the routing key as f_snmp_trap and paste the json into the payload section
6. Publish the message. If there is an error it will tell you
7. Rerun the query to double check that the Salesforce has a case for the Ticket.

For more details, please check this. https://docs.google.com/document/d/1_zwi02B8WoysEUEzfIK2A046xsmFfnrYqiaO4kA7QHk/edit#heading=h.z4rx22cg5uw8

If the curl request is performed but the alaert doesn't clear, check the ticketer-integration logs. Example error:

2022-09-29 03:08:20.563 ERROR 25776 --- [Camel (camel-1) thread #27 - rabbitmq://ticketer_audit_event_exchange_topic] utbound$$EnhancerBySpringCGLIB$$bb0998e1 : The HTTP response: {"error":"duplicate value found: Originating_System_ID__c duplicates value on record with id: 5003g00000N6V2w","success":"false"}

This is likely caused by a RabbitMQ issue. The message will need to be requeued.

FIX: Requeue the message in the rabbit MQ queue ticketer_inbound_salesforce_queue

{    "caseId": "5003g00000N6Z0TAAV",    "ticketId": 764517,    "caseNumber": "02475388"}

The case Id and caseNumber fields can be found in the original ticketer-integration logs.

zgrep  "Originating_System_ID__c=764919" ~/logs/spring-boot/ticketer-integration.log* | grep CaseNumber

2022-09-28 10:11:16.149  INFO 25776 --- [SalesforceHttpClient@44be69aa-42] org.apache.camel.Tracing                 :      [route11     ] [rabbitmq://{{rabbitmq.queues.sale] Exchange[Id: 7B60355912D6F25-000000000000EC78, BodyType: byte[], Body: {"Status":"New","Carrier_Update__c":null,"Force_Majeure__c":false,"CaseNumber":"02475388","CN_Priority__c":"4","Id":"5003g00000N6Z0TAAV","Originating_System_ID__c":"764517","correlationId":"bf2d5435-abed-4f0e-aa40-01fb8aa30cff"}]

Auto Clear: Yes, Alert will clear when the count of ATC DAS ticket was not created in Salesforce is 0