User Tools

Site Tools


watchdogs:creating_a_new_alert

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
watchdogs:creating_a_new_alert [2023/03/03 16:47] – created 10.91.129.5watchdogs:creating_a_new_alert [2023/03/06 16:31] (current) 10.91.110.100
Line 1: Line 1:
-===== Creating a New Alert =====+===== Creating a New Alert on Prometheus =====
  
-Author: Máté Domonics+Author: Máté Domonics on 06/03/2023 
 + 
 +**If you haven't created a new metric yet, please refer to [[http://wiki.err/doku.php?id=watchdogs:creating_a_new_metric|this wiki page]].** 
 + 
 +==== Creating a Wiki entry for your alarm ==== 
 +When creating a new alarm, it is invaluable to create a new wiki page entry, where you detail what action should be taken if this alarm is firing. 
 +Follow these steps to create a new wiki page: 
 +  - Go to [[http://wiki.err/doku.php?id=resolution_area:prometheus_resolutions|this]] page, and edit the page. 
 +  - Find the relevant subsection where you want to add your new entry, and following the naming conventions, create a new page. 
 +  - Take a look at the other pages for reference on how to fill out the page. 
 +  - Once your page is made, continue to the next step. 
 + 
 + 
 +==== Changing the Rules file ==== 
 +  - Navigate to **prometheus-monitoring-config/files/rules** and find the relevant .rules file to make changes to. 
 +  - Create a new alert, matching the layout from the alerts that are already present. 
 +  - Match the **expr** field to the name of the metric that you created. 
 +  - Set the label to the desired severity level. 
 +    - //Warning, Minor, Major or Critical// 
 +  - Under **annotations**, create a short summary of the alert.  
 +    - //It is important to remain clear and concise, as this is the message that is shown when the alert fires!// 
 +  - In the resolution field, you should include the link to the wiki page that you created for this alarm above. 
 +  - Save your changes, and push both your new alarm and metric(s) to the repo. 
 +  - Congratulations! You just made your first alarm and metric(s)!
watchdogs/creating_a_new_alert.1677862023.txt.gz · Last modified: 2023/03/03 16:47 by 10.91.129.5