User Tools

Site Tools


resolution_area:prometheus_resolutions:res-p1109

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
resolution_area:prometheus_resolutions:res-p1109 [2021/12/17 13:09] wflahertyresolution_area:prometheus_resolutions:res-p1109 [2021/12/17 13:09] (current) wflaherty
Line 9: Line 9:
  
 **Resolution:** **Resolution:**
-    The service is running in a stable state+The service is running in a stable state
  
 **Manual Action Steps:** **Manual Action Steps:**
- - ssh onto the affected server. +  - ssh onto the affected server. 
- - Use `ps aux | grep <app>` to see if the application is still running. +  - Use `ps aux | grep <app>` to see if the application is still running. 
- - use `systemctl status <app>` to check the status of the application. +  - use `systemctl status <app>` to check the status of the application. 
-   - If the application is trying to restart over and over open /etc/systemd/system/<app>.service +    - If the application is trying to restart over and over open /etc/systemd/system/<app>.service 
-   - Edit the `Restart=` line to be off rather than on-failure or always. +    - Edit the `Restart=` line to be off rather than on-failure or always. 
- - Use `sudo journalctl -ex` to see the logs of the server after attempting to restart the application. +  - Use `sudo journalctl -ex` to see the logs of the server after attempting to restart the application. 
- - A problem for some things in the past that weren't written by Errigal was users required for applications. +  - A problem for some things in the past that weren't written by Errigal was users required for applications. 
-   - ELK stack and MySQL all require an elasticsearch, logstash, kibana and mysql user respectively. +    - ELK stack and MySQL all require an elasticsearch, logstash, kibana and mysql user respectively. 
- - Sometimes just fully shutting down the service with `sudo systemctl stop <app>` for a minute before trying to start it again with `sudo systemctl start <app>` can help the application recover. +  - Sometimes just fully shutting down the service with `sudo systemctl stop <app>` for a minute before trying to start it again with `sudo systemctl start <app>` can help the application recover. 
- - If the bash prompt is behaving strangely, the server is likely running out of RAM for some reason. +  - If the bash prompt is behaving strangely, the server is likely running out of RAM for some reason. 
- - Another thing worth checking is the disk space. `df -h`  +  - Another thing worth checking is the disk space. `df -h`  
- - If this is an Errigal app, you can check the logs at moros.err:5601/app/kibana to see the application logs before the service died.+  - If this is an Errigal app, you can check the logs at moros.err:5601/app/kibana to see the application logs before the service died.
  
 **Auto Clear:** **Auto Clear:**
-  Its entirely possible the service will automatically recover+Its entirely possible the service will automatically recover
resolution_area/prometheus_resolutions/res-p1109.1639746554.txt.gz · Last modified: 2021/12/17 13:09 by wflaherty