{"id":934,"date":"2018-10-14T16:07:14","date_gmt":"2018-10-14T13:07:14","guid":{"rendered":"http:\/\/kifarunix.com\/?p=934"},"modified":"2024-03-11T20:05:56","modified_gmt":"2024-03-11T17:05:56","slug":"how-to-monitor-hosts-and-services-with-icinga-2-on-ubuntu-18-04","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/how-to-monitor-hosts-and-services-with-icinga-2-on-ubuntu-18-04\/","title":{"rendered":"How to Monitor Remote Linux Hosts and Services with Icinga 2 on Ubuntu 18.04"},"content":{"rendered":"\n
In this very tutorial, you are going learn how to monitor hosts and services with Icinga 2 on Ubuntu 18.04 LTS. Well, Icinga 2 can be used to monitor the availability of hosts and services. Hosts and services can be virtually anything; Network services (HTTP, SMTP, SNMP, SSH, etc.),Printers, Switches or routers, Temperature sensors, or any other local or network-accessible services.<\/p>\n\n\n\n
In our last tutorial, we learnt how to install Icinga 2 and Icinga Web 2 on Ubuntu 18.04 LTS<\/a>.<\/p>\n\n\n\n Hosts and services can be monitored in two ways;<\/p>\n\n\n\n For simple host monitoring, we are going to learn how to monitor state of the host and services like HTTP and SSH.<\/p>\n\n\n\n Hosts can be in any one of the following states:<\/p>\n\n\n\n Services can be in any one of the following states:<\/p>\n\n\n\n To begin with, you can create a configuration file for your hosts, if you like, and add services to be monitored as shown below.<\/p>\n\n\n\n The You can also define the services as shown below.<\/p>\n\n\n\n Once you are done with host definition save the configuration file and run the following command to check for syntactical errors.<\/p>\n\n\n\n If all is well, restart icinga2 service<\/p>\n\n\n\n Now login to the Icinga Web 2 dashboard and should be able to see your services.<\/p>\n\n\n\n Agent based monitoring<\/a> becomes handy when remote services are not directly accessible through the network. While there are certain mechanisms to ensure a secure communication between all nodes (firewalls, policies, software hardening, etc.), Icinga 2 imposes mandatory SSL communication between nodes. The communication between the master and client nodes happens on port 5665 and thus, if firewall is running, this port has to be opened.<\/p>\n\n\n\n Our environment comprises of;<\/p>\n\n\n\n Run the command below to setup a Icinga 2 master node.<\/p>\n\n\n\n This will prompt you if it is a Satellite or Client setup. Choose n <\/strong>to select a master setup.<\/p>\n\n\n\n Press Enter <\/strong>to accept the defaults.<\/p>\n\n\n\n Once that is done, restart Icinga 2 daemon to finalize setup.<\/p>\n\n\n\n If firewall is running, open port 5665.<\/p>\n\n\n\n A client which sends a certificate signing request (CSR) must authenticate itself in a trusted way. The master generates a client ticket which is included in this request. That way the master can verify that the request matches the previously trusted ticket and sign the request.<\/p>\n\n\n\n Run the commands below on the master node to generate the tickets for your client nodes;<\/p>\n\n\n\n For client node, server01.example.com<\/strong>;<\/p>\n\n\n\n For client node, mydevapp.example.com<\/strong>;<\/p>\n\n\n\n Save these keys as you will require them later while setting up the client nodes.<\/p>\n\n\n\n Note that before you can set up the client nodes, Icinga 2 on the master node must be running and accepting connections on port Login in to the client nodes and Install the Icinga 2 package and setup the required plugins as follows;<\/p>\n\n\n\n For CentOS 7, server01.example.com<\/p>\n\n\n\n Setup EPEL Repository<\/p>\n\n\n\n For Ubuntu 18.04 LTS, mydevapp..example.com<\/p>\n\n\n\n Install Icinga2 and monitoring plugins<\/p>\n\n\n\n On CentOS 7;<\/p>\n\n\n\n On Ubuntu 18.04 LTS;<\/p>\n\n\n\n Once the installation is done, start and enable icinga2 daemon to run on boot.<\/p>\n\n\n\n On CentOS 7:<\/p>\n\n\n\n On Ubuntu 18.04, icinga 2 is started and enabled upon installation. You can check this by running;<\/p>\n\n\n\n Next, Set up the client nodes as shown below.<\/p>\n\n\n\n For CentOS 7 Client Node;<\/p>\n\n\n\n Repeat the same for the other client node. Be sure to replace the ticket with the respective one created above.<\/p>\n\n\n\n The setup wizard fetches the parent node\u2019s certificate and ask you to verify that information. This is to prevent MITM attacks or any kind of untrusted parent relationship.<\/p>\n\n\n\n Once done, restart Icinga2 daemon as requested.<\/p>\n\n\n\n If firewall is running, open port 5665.<\/p>\n\n\n\n On Ubuntu;<\/p>\n\n\n\n On CentOS 7<\/p>\n\n\n\n You can verify that the master and the client node are now connected.<\/p>\n\n\n\n For client node, server01.example.com<\/p>\n\n\n\n For client node, mydevapp.example.com<\/p>\n\n\n\n Now that the master nodes and the client nodes are connected, it is time to configure the master so it can execute a remote check on the client using the command endpoint.<\/p>\n\n\n\n Login to master node and create a zone directory where you can add the configuration files for the host and service objects you want to monitor.<\/p>\n\n\n\n Create the hosts configuration file for the clients.<\/p>\n\n\n\n Create a services configuration file for your clients.<\/p>\n\n\n\n Save the changes and validate the configuration on the master node:<\/p>\n\n\n\n Restart the Icinga 2 daemon<\/p>\n\n\n\n Login to Icinga Web 2 and confirm the checks.<\/p>\n\n\n\n You have successfully configured Icinga 2 agent-based monitoring on Ubuntu 18.04. Happy monitoring!!.<\/p>\n\n\n\n Related Tutorials;<\/p>\n\n\n\n Install Icinga Web 2 on Debian 10 Buster<\/a><\/p>\n\n\n\n Install Icinga 2 on Debian 10 Buster<\/a><\/p>\n\n\n\n Install Nagios Core on Debian 10 Buster<\/a><\/p>\n\n\n\n Install Elastic Stack 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n\n
This involves the use of services that are accessible across the local network or the Internet for example include HTTP, SMTP, SNMP, SSH, and ICMP etc. This method is fit for monitoring hosts on which monitoring agents cannot be installed.<\/li>\n\n\n\n
In this method, an agent is required in order to monitor host and services like CPU load, disk space, memory and the running process. In this configuration, Icinga 2 acts as the master node and the remote host acts as the client node.<\/li>\n<\/ul>\n\n\n\nSimple Host Monitoring<\/h3>\n\n\n\n
\n
\n
vim \/etc\/icinga2\/conf.d\/myhosts.conf<\/pre>\n\n\n\n
object Host \"mydevapp.example.com\" {\n import \"generic-host\"\n address = \"192.168.43.214\"\n vars.os = \"Linux\"\n\n vars.http_vhosts[\"http\"] = {\n http_uri = \"\/\"\n }\n }<\/pre>\n\n\n\n
import<\/code> keyword is used to import the
generic-host<\/code> template which takes care of setting up the host check command to
hostalive<\/code>. If you require a different check command, you can override it in the object definition.<\/p>\n\n\n\n
vars<\/code> attribute can be used to define custom attributes which are available for checks for example checking for ssh service as defined in the services.conf<\/strong> file for all systems whose OS is set to Linux.<\/p>\n\n\n\n
vars.http_vhosts<\/code> defines http vhost attributes for the
http<\/code> service apply rule defined in services.conf.<\/p>\n\n\n\n
object Host \"mydevapp.example.com\" {\naddress = \"192.168.43.214\"\ncheck_command = \"hostalive\"\n}\n\nobject Service \"http\" {\nhost_name = \"mydevapp.example.com\"\ncheck_command = \"http\"\n}\nobject Service \"ssh\" {\nhost_name = \"mydevapp.example.com\"\ncheck_command = \"ssh\"\n}<\/pre>\n\n\n\n
icinga2 daemon -C<\/pre>\n\n\n\n
systemctl restart icinga2<\/pre>\n\n\n\n
<\/a>
Agent-based Monitoring<\/h3>\n\n\n\n
\n
Set up Icinga 2 Master Node<\/h3>\n\n\n\n
icinga2 node wizard<\/pre>\n\n\n\n
<\/a>
systemctl restart icinga2<\/pre>\n\n\n\n
ufw allow 5665<\/pre>\n\n\n\n
Generate a client ticket for client nodes<\/h4>\n\n\n\n
icinga2 pki ticket --cn server01.example.com\n970a747db8d676fd2057e3ede2d0e9b5e6ae0dfd<\/pre>\n\n\n\n
icinga2 pki ticket --cn mydevapp.example.com\n99030d0af2c9cb27696cbe3ca3c40a6b043bd487<\/pre>\n\n\n\n
Configure Client Nodes<\/h2>\n\n\n\n
5665<\/code>.<\/p>\n\n\n\n
ss -altnp | grep 5665\nLISTEN 0 128 0.0.0.0:5665 0.0.0.0:* users:((\"icinga2\",pid=18872,fd=14))<\/pre>\n\n\n\n
Setting up Client Nodes<\/h4>\n\n\n\n
Add the Icinga repository to your package management configuration<\/h5>\n\n\n\n
yum install https:\/\/packages.icinga.com\/epel\/icinga-rpm-release-7-latest.noarch.rpm -y\n<\/pre>\n\n\n\n
yum install epel-release -y<\/pre>\n\n\n\n
wget -O - https:\/\/packages.icinga.com\/icinga.key | apt-key add -\necho \"deb http:\/\/packages.icinga.com\/ubuntu icinga-bionic main\" > \/etc\/apt\/sources.list.d\/icinga2.list\necho \"deb-src http:\/\/packages.icinga.com\/ubuntu icinga-bionic main\" >> \/etc\/apt\/sources.list.d\/icinga2.list\n
apt update<\/code><\/pre>\n\n\n\n
yum install icinga2 nagios-plugins-all -y<\/pre>\n\n\n\n
apt install icinga2 monitoring-plugins -y<\/pre>\n\n\n\n
systemctl start icinga2\nsystemctl enable icinga2<\/pre>\n\n\n\n
systemctl status icinga2\nsystemctl is-enabled icinga2<\/pre>\n\n\n\n
icinga2 node wizard\nWelcome to the Icinga 2 Setup Wizard!\n\nWe will guide you through all required configuration details.\n\nPlease specify if this is a satellite\/client setup ('n' installs a master setup) [Y\/n]: y <- Enter y to setup client<\/strong>\n\nStarting the Client\/Satellite setup routine...\n\nPlease specify the common name (CN) [server01.example.com]: ENTER<\/strong>\n\nPlease specify the parent endpoint(s) (master or satellite) where this node should connect to:\nMaster\/Satellite Common Name (CN from your master\/satellite node): icinga.example.com\n\nDo you want to establish a connection to the parent node from this node? [Y\/n]: y<\/strong>\nPlease specify the master\/satellite connection information:\nMaster\/Satellite endpoint host (IP address or FQDN): 192.168.43.138 < IP of the master node<\/strong>\nMaster\/Satellite endpoint port [5665]: ENTER<\/strong>\n\nAdd more master\/satellite endpoints? [y\/N]: n<\/strong>\nParent certificate information:\n\nSubject: CN = icinga.example.com\nIssuer: CN = Icinga CA\nValid From: Oct 13 08:47:58 2018 GMT\nValid Until: Oct 9 08:47:58 2033 GMT\nFingerprint: 64 A7 AA F8 C0 F4 A8 A1 FE 95 9D 1E DC 86 07 3B E8 1F 97 41\n\nIs this information correct? [y\/N]: y<\/strong>\n\nPlease specify the request ticket generated on your Icinga 2 master (optional).\n(Hint: # icinga2 pki ticket --cn 'server01.example.com'): 970a747db8d676fd2057e3ede2d0e9b5e6ae0dfd < ticket generated before<\/strong>\nPlease specify the API bind host\/port (optional):\nBind Host []: ENTER<\/strong>\nBind Port []: ENTER<\/strong>\n\nAccept config from parent node? [y\/N]: y<\/strong>\nAccept commands from parent node? [y\/N]: y<\/strong>\n\nReconfiguring Icinga...\n\nLocal zone name [server01.example.com]: ENTER<\/strong>\nParent zone name [master]: ENTER<\/strong>\n\nDefault global zones: global-templates director-global\nDo you want to specify additional global zones? [y\/N]: n<\/strong>\n\nDo you want to disable the inclusion of the conf.d directory [Y\/n]: y<\/strong>\nDisabling the inclusion of the conf.d directory...\n\nDone.\n\nNow restart your Icinga 2 daemon to finish the installation!<\/pre>\n\n\n\n
systemctl restart icinga2<\/pre>\n\n\n\n
ufw allow 5665\nufw reload<\/pre>\n\n\n\n
firewall-cmd --add-port=5665\/tcp --permanent\nfirewall-cmd --reload<\/pre>\n\n\n\n
ss | grep :5665<\/pre>\n\n\n\n
tcp ESTAB 0 0 192.168.43.63:33970 192.168.43.138:5665<\/pre>\n\n\n\n
ss | grep :5665<\/pre>\n\n\n\n
tcp ESTAB 0 0 192.168.43.214:43528 192.168.43.138:5665<\/pre>\n\n\n\n
Configure the Agent Monitoring on Icinga 2 Master<\/h3>\n\n\n\n
mkdir \/etc\/icinga2\/zones.d\/master<\/pre>\n\n\n\n
vim \/etc\/icinga2\/zones.d\/master\/hosts.conf<\/pre>\n\n\n\n
\/\/ for server01.example.com\nobject Endpoint \"server01.example.com\" {\n}\n\nobject Zone \"server01.example.com\" {\nendpoints = [ \"server01.example.com\" ]\nparent = \"master\"\n}\nobject Host \"server01.example.com\" {\ncheck_command = \"hostalive\" \/\/check is executed on the master\naddress = \"192.168.43.63\"\n\nvars.client_endpoint = name \/\/follows the convention that host name == endpoint name\n}\n\/\/ for mydevapp.example.com\nobject Endpoint \"mydevapp.example.com\" {\n}\n\nobject Zone \"mydevapp.example.com\" {\nendpoints = [ \"mydevapp.example.com\" ]\nparent = \"master\"\n}\nobject Host \"mydevapp.example.com\" {\ncheck_command = \"hostalive\" \naddress = \"192.168.43.214\"\n\nvars.client_endpoint = name \n}<\/pre>\n\n\n\n
vim \/etc\/icinga2\/zones.d\/master\/services.conf<\/pre>\n\n\n\n
\/\/ Check System Load\napply Service \"System Load\" {\n check_command = \"load\"\n command_endpoint = host.vars.client_endpoint \/\/ Check executed on client node\n assign where host.vars.client_endpoint\n}\n\n\/\/ Check number of running system Processes\napply Service \"Process\" {\n check_command = \"procs\"\n command_endpoint = host.vars.client_endpoint\n assign where host.vars.client_endpoint\n}\n\n\/\/ Check number of Logged in Users\napply Service \"Users\" {\n check_command = \"users\"\n command_endpoint = host.vars.client_endpoint\n assign where host.vars.client_endpoint\n}\n\n\/\/ Check System Disk Usage\napply Service \"Disk\" {\n check_command = \"disk\"\n command_endpoint = host.vars.client_endpoint\n assign where host.vars.client_endpoint\n}\n\n\/\/ Check for SWAP memory Usage\napply Service \"SWAP\" {\n check_command = \"swap\"\n command_endpoint = host.vars.client_endpoint\n assign where host.vars.client_endpoint\n}\n\n\/\/ SSH Service Check\napply Service \"SSH Service\" {\n check_command = \"ssh\"\n command_endpoint = host.vars.client_endpoint\n assign where host.vars.client_endpoint\n\n}\napply Service \"Ping\" {\n check_command = \"ping4\"\n assign where host.address\n}<\/pre>\n\n\n\n
icinga2 daemon -C\n<\/pre>\n\n\n\n
systemctl restart icinga2<\/pre>\n\n\n\n
<\/a>
<\/a>