Level: Critical

Purpose: Notify ATC Team that SalesForce has not sent a Carrier Update for 7 days.

Scenario: Due to the new customer-focused methodology that was introduced to SalesForce in 2023, more customer interaction is required by SalesForce. This means that if we do not receive CUs (Carrier Update) or FMs (Force Majeure) from them, it might become a problem for SalesForce as they are not receiving anything from their customers.

Resolution: First check if when the alarm fires, is this still an issue with SalesForce that hasn't been resolved. If it is, talk to the SalesForce representative. If not, silence alert.

Recent Example No carrier updates for 7 days. No force majeure. They both stopped on 14th October. IN the logs, the below connection error occurred:

2023-10-14 01:03:12.689  WARN 26593 --- [SalesforceHttpClient@cb39552-3403] o.a.c.c.s.i.s.SubscriptionHelper         : Connect failure: {failure={exception=java.io.EOFException: HttpConnectionOverHTTP@30421889::DecryptedEndPoint@3a06d35e{l=/10.91.202.137:43534,r=americantower.my.salesforce.com/13.110.35.77:443,OPEN,fill=-,flush=-,to=157926/340000}, message={clientId=3sq1vat48bk0zyvq1c7l7ik0jndnl, channel=/meta/connect, id=111037, connectionType=long-polling}, connectionType=long-polling}, channel=/meta/connect, id=111037, successful=false}
2023-10-14 01:03:12.719  INFO 26593 --- [SalesforceHttpClient@cb39552-3467] o.a.c.c.s.i.s.SubscriptionHelper         : Restarting on unexpected disconnect from Salesforce...
2023-10-14 01:03:12.787  WARN 26593 --- [SalesforceHttpClient@cb39552-3334] o.a.c.c.s.i.s.SubscriptionHelper         : Handshake failure: {ext={sfdc={failureReason=401::Authentication invalid}, replay=true, payload.format=true}, advice={reconnect=none}, channel=/meta/handshake, id=111039, error=403::Handshake denied, successful=false}

Resolution Restart service

sudo systemctl restart ticketer-integration

Manual Action Steps: N/A

Auto Clear: Will clear when we receive a CU.