{"id":8584,"date":"2021-04-10T13:02:38","date_gmt":"2021-04-10T10:02:38","guid":{"rendered":"https:\/\/kifarunix.com\/?p=8584"},"modified":"2024-03-19T08:23:52","modified_gmt":"2024-03-19T05:23:52","slug":"configure-grafana-email-alerting","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/configure-grafana-email-alerting\/","title":{"rendered":"Configure Grafana Email Alerting"},"content":{"rendered":"\n

In this tutorial, you will learn how to configure Grafana Email alerting. Grafana alerting allows you to attach rules to your dashboard panels.<\/em> When you save the dashboard, Grafana extracts the alert rules into a separate alert rule storage and schedules them for evaluation.<\/em> When an alert changes state, it sends out notifications<\/em> through various channels including Email, Slack, Google Hangout Chat or several other integrations<\/a>.<\/p>\n\n\n\n

Note that Alerting is only available in Grafana v4.0 and above and is only supported on graph panels, at least as at now.<\/p>\n\n\n\n

Configuring Grafana Email Alerting<\/h2>\n\n\n\n

There are several notification channels that you can configure your Grafana to sent alerts through. However, this tutorial focuses on using Email alerting.<\/p>\n\n\n\n

To demonstrate Grafana Email alerting, we have created dashboards for monitoring system metrics using Prometheus and Grafana. See the dashboard below for visualizing the load average for a system.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

To easily demonstrate Grafana email alerting, we will setup Grafana to sent alerts when the load for the last 1, 5, 15 mins is 3.00,2.00,1.00.<\/p>\n\n\n\n

Configure Grafana SMTP settings<\/h3>\n\n\n\n

Before Grafana can be able to sent email alerts, you need to configure how alerts will be relayed.<\/p>\n\n\n\n

In this demo, we will configure Grafana to Gmail relay to sent out emails. You can use any of your preferred mail servers.<\/p>\n\n\n\n

Hence, open the Grafana configuration file and navigate to smtp<\/strong> section.<\/p>\n\n\n\n

Check the values for the highlighted lines and replace them accordingly.<\/p>\n\n\n\n

vim \/etc\/grafana\/grafana.ini<\/code><\/pre>\n\n\n\n
[smtp]\nenabled = true\nhost = smtp.gmail.com:587 \nuser = username@gmail.com<\/strong>\n# If the password contains # or ; you have to wrap it with triple quotes. Ex \"\"\"#password;\"\"\"\npassword = \"\"\"pass123#\"\"\"\n<\/strong>;cert_file =\n;key_file =\nskip_verify = true\nfrom_address = admin@grafana.kifarunix-demo.com\nfrom_name = Grafana<\/strong><\/code><\/pre>\n\n\n\n
Pay attention to the line, # If the password contains # or ; you have to wrap it with triple quotes. Ex “””#password;”””<\/strong>.<\/p>\n\n\n\n
Save and exit the file.<\/p>\n\n\n\n
Restart Grafana service;<\/p>\n\n\n\n
systemctl restart grafana-server<\/code><\/pre>\n\n\n\n
Note: If you are using Gmail relay, ensure that you turn on less secure apps<\/a> to sent mails.<\/p>\n\n\n\n
Add Grafana Notification Channel<\/h3>\n\n\n\n
Click Alerting icon (bell like icon) on the Grafana side bar and then click Notification channel<\/strong>. This should open up the Alerts and Notifications page;<\/p>\n\n\n\n
\"Configure<\/figure>\n\n\n\n
Click Add Channel<\/strong> to create a new Email notification channel<\/p>\n\n\n\n
    \n
  1. Set the name of the channel<\/li>\n\n\n\n
  2. Choose the channel notification method, Email<\/strong> is used in this case.<\/li>\n\n\n\n
  3. Enter addresses of the recipients. Multiple addresses are separated using semi-colon.<\/li>\n\n\n\n
  4. Select your notification settings;<\/li>\n<\/ol>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    Once you have configured your channel settings, click Test<\/strong> to verify email deliverability.<\/p>\n\n\n\n
    If all is fine, you should get Test notification sent<\/strong>.<\/p>\n\n\n\n
    Also, you should receive Test notification email on your recipient inbox.<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    If not, check Grafana logs;<\/p>\n\n\n\n
    tail -f \/var\/log\/grafana\/grafana.log<\/code><\/pre>\n\n\n\n
    Save the Channel, when done testing.<\/p>\n\n\n\n
    Click on the title of the graph panel that you want to configure alerts for and click Edit<\/strong>. For example in our case, we have a graph for CPU load average.<\/p>\n\n\n
    \n
    \"\"<\/figure><\/div>\n\n\n
    When the panel is opened for editing, click on the Alert<\/strong> tab.<\/p>\n\n\n\n
    Next, click Create Alert<\/strong> to setup your alerting;<\/p>\n\n\n\n