User Tools

Site Tools


resolution_area:prometheus_resolutions:res-p1116

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

resolution_area:prometheus_resolutions:res-p1116 [2021/12/20 14:44] – created wflahertyresolution_area:prometheus_resolutions:res-p1116 [2021/12/20 16:06] (current) wflaherty
Line 1: Line 1:
-CAS is out of memory+=====AppDown: CAS Out Of Memory===== 
 + 
 +**Level:**    
 +  * CASOutOfMemory - __Critical__  
 + 
 +**Purpose:** When CAS reports in the logs that is running out of memory or already has run out.  
 + 
 +**Scenario:** The CAS logs printed: <code java>java.lang.OutOfMemoryError: unable to create new native thread </code> 
 + 
 +**Resolution:** CAS doesn't print an out of memory line for another 60 mins.  
 + 
 +**Manual Action Steps:**  
 +CAS is on the handlers and sometimes one.  
 +Use [[https://docs.google.com/spreadsheets/d/1Ebj0kWPl63Q4L3f_oo_kvWoh5ZRq2vFI8FmrWbjoPlo/edit#gid=1762968851|this page]] 
 +to see where CAS is deployed for your investigation. 
 + 
 +If there are two CAS's then you may be able to perform a restart of CAS for one handler without affecting the other. 
 + 
 +  - Open a terminal and cd into your local deployment-playbooks repo. 
 +  - Stop the CAS application: <code bash>ansible-playbook -i ../env-configuration/server/hosts.ini --vault-id @prompt cas.yml --extra-vars="actions='stop'" --limit="apps1" </code> 
 +  - Do please note the --limit apps1 appended to the end of that command. You may want to change it for apps2 or remove it entirely. 
 +  - Once CAS has stopped, have a quick check on the server if its still running just in case.  
 +    - <code bash>ssh scotty@serverappsX.err "ps aux | grep cas"</code> 
 +    - <code bash>ssh scotty@atcappsX.atc "ps aux | grep cas"</code> 
 +    - <code bash>ssh scotty@extappsX.ext "ps aux | grep cas"</code> 
 +    - <code bash>ssh scotty@scoapps1.err "ps aux | grep cas"</code> 
 +    - <code bash>ssh scotty@scolb1.err "ps aux | grep cas"</code> 
 +  - Start the CAS application: <code bash>ansible-playbook -i ../env-configuration/server/hosts.ini --vault-id @prompt cas.yml --extra-vars="actions='start'" --limit="apps1" </code> 
 +  - After the playbook has complete, ensure that the applicaiton is running.  
 +     
 + 
 + 
 +**Auto Clear:**  
 +Not sure, it might auto clear...  
 +Restart just in case.  
resolution_area/prometheus_resolutions/res-p1116.1640011483.txt.gz · Last modified: 2021/12/20 14:44 by wflaherty