User Tools

Site Tools


resolution_area:prometheus_resolutions:res-p1116

AppDown: CAS Out Of Memory

Level:

  • CASOutOfMemory - Critical

Purpose: When CAS reports in the logs that is running out of memory or already has run out.

Scenario: The CAS logs printed:

java.lang.OutOfMemoryError: unable to create new native thread 

Resolution: CAS doesn't print an out of memory line for another 60 mins.

Manual Action Steps: CAS is on the handlers and sometimes one. Use this page to see where CAS is deployed for your investigation.

If there are two CAS's then you may be able to perform a restart of CAS for one handler without affecting the other.

  1. Open a terminal and cd into your local deployment-playbooks repo.
  2. Stop the CAS application:
    ansible-playbook -i ../env-configuration/server/hosts.ini --vault-id @prompt cas.yml --extra-vars="actions='stop'" --limit="apps1" 
  3. Do please note the –limit apps1 appended to the end of that command. You may want to change it for apps2 or remove it entirely.
  4. Once CAS has stopped, have a quick check on the server if its still running just in case.
    1. ssh scotty@serverappsX.err "ps aux | grep cas"
    2. ssh scotty@atcappsX.atc "ps aux | grep cas"
    3. ssh scotty@extappsX.ext "ps aux | grep cas"
    4. ssh scotty@scoapps1.err "ps aux | grep cas"
    5. ssh scotty@scolb1.err "ps aux | grep cas"
  5. Start the CAS application:
    ansible-playbook -i ../env-configuration/server/hosts.ini --vault-id @prompt cas.yml --extra-vars="actions='start'" --limit="apps1" 
  6. After the playbook has complete, ensure that the applicaiton is running.

Auto Clear: Not sure, it might auto clear… Restart just in case.

resolution_area/prometheus_resolutions/res-p1116.txt · Last modified: 2021/12/20 16:06 by wflaherty