User Tools

Site Tools


watchdogs:creating_a_new_alert

Creating a New Alert on Prometheus

Author: Máté Domonics on 06/03/2023

If you haven't created a new metric yet, please refer to this wiki page.

Creating a Wiki entry for your alarm

When creating a new alarm, it is invaluable to create a new wiki page entry, where you detail what action should be taken if this alarm is firing. Follow these steps to create a new wiki page:

  1. Go to this page, and edit the page.
  2. Find the relevant subsection where you want to add your new entry, and following the naming conventions, create a new page.
  3. Take a look at the other pages for reference on how to fill out the page.
  4. Once your page is made, continue to the next step.

Changing the Rules file

  1. Navigate to prometheus-monitoring-config/files/rules and find the relevant .rules file to make changes to.
  2. Create a new alert, matching the layout from the alerts that are already present.
  3. Match the expr field to the name of the metric that you created.
  4. Set the label to the desired severity level.
    1. Warning, Minor, Major or Critical
  5. Under annotations, create a short summary of the alert.
    1. It is important to remain clear and concise, as this is the message that is shown when the alert fires!
  6. In the resolution field, you should include the link to the wiki page that you created for this alarm above.
  7. Save your changes, and push both your new alarm and metric(s) to the repo.
  8. Congratulations! You just made your first alarm and metric(s)!
watchdogs/creating_a_new_alert.txt · Last modified: 2023/03/06 16:31 by 10.91.110.100