===== Creating a New Alert on Prometheus =====

Author: Máté Domonics on 06/03/2023

**If you haven't created a new metric yet, please refer to [[http://wiki.err/doku.php?id=watchdogs:creating_a_new_metric|this wiki page]].**

==== Creating a Wiki entry for your alarm ====
When creating a new alarm, it is invaluable to create a new wiki page entry, where you detail what action should be taken if this alarm is firing.
Follow these steps to create a new wiki page:
  - Go to [[http://wiki.err/doku.php?id=resolution_area:prometheus_resolutions|this]] page, and edit the page.
  - Find the relevant subsection where you want to add your new entry, and following the naming conventions, create a new page.
  - Take a look at the other pages for reference on how to fill out the page.
  - Once your page is made, continue to the next step.


==== Changing the Rules file ====
  - Navigate to **prometheus-monitoring-config/files/rules** and find the relevant .rules file to make changes to.
  - Create a new alert, matching the layout from the alerts that are already present.
  - Match the **expr** field to the name of the metric that you created.
  - Set the label to the desired severity level.
    - //Warning, Minor, Major or Critical//
  - Under **annotations**, create a short summary of the alert. 
    - //It is important to remain clear and concise, as this is the message that is shown when the alert fires!//
  - In the resolution field, you should include the link to the wiki page that you created for this alarm above.
  - Save your changes, and push both your new alarm and metric(s) to the repo.
  - Congratulations! You just made your first alarm and metric(s)!