{"id":3454,"date":"2019-07-02T15:53:17","date_gmt":"2019-07-02T12:53:17","guid":{"rendered":"https:\/\/kifarunix.com\/?p=3454"},"modified":"2023-06-25T19:50:43","modified_gmt":"2023-06-25T16:50:43","slug":"install-filebeat-on-fedora-30-fedora-29-centos-7","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/install-filebeat-on-fedora-30-fedora-29-centos-7\/","title":{"rendered":"Install Filebeat on Fedora 30\/Fedora 29\/CentOS 7"},"content":{"rendered":"\n<p>In this guide, we are going to learn how to install Filebeat on Fedora 30\/Fedora 29\/CentOS 7. <a href=\"https:\/\/www.elastic.co\/beats\/filebeat\" target=\"_blank\" rel=\"noreferrer noopener\">Filebeat<\/a> is a lightweight shipper for collecting, forwarding and centralizing event log data. It is installed as an agent on the servers you are collecting logs from. It can forward the logs it is collecting to either Elasticsearch or Logstash for indexing.<\/p>\n\n\n\n<div class=\"wp-block-rank-math-toc-block\" id=\"rank-math-toc\"><h2>Table of Contents<\/h2><nav><ul><li><a href=\"#install-filebeat-on-fedora-30-fedora-29-cent-os-7\">Install Filebeat on Fedora 30\/Fedora 29\/CentOS 7<\/a><ul><li><a href=\"#setup-elk-stack-server\">Setup ELK Stack Server<\/a><\/li><li><a href=\"#install-filebeat-on-fedora-30-fedora-29-cent-os-7-1\">Install Filebeat on Fedora 30\/Fedora 29\/CentOS 7<\/a><ul><li><a href=\"#install-filebeat-7-using-rpm-repository\">Install Filebeat 7 using RPM Repository<\/a><\/li><li><a href=\"#install-filebeat-using-rpm-binary\">Install Filebeat Using RPM Binary<\/a><\/li><\/ul><\/li><li><a href=\"#configure-filebeat-7-on-fedora-30-fedora-29-cent-os-7\">Configure Filebeat 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a><ul><li><a href=\"#configure-filebeat-output\">Configure Filebeat Output<\/a><\/li><li><a href=\"#enable-filebeat-system-module\">Enable Filebeat System Module<\/a><\/li><li><a href=\"#load-the-index-template-in-elasticsearch\">Load the index template in Elasticsearch<\/a><\/li><\/ul><\/li><li><a href=\"#verify-elasticsearch-index-data-reception\">Verify Elasticsearch Index Data Reception<\/a><\/li><li><a href=\"#other-related-guides\">Other Related Guides:<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"install-filebeat-on-fedora-30-fedora-29-cent-os-7\">Install Filebeat on Fedora 30\/Fedora 29\/CentOS 7<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"setup-elk-stack-server\">Setup ELK Stack Server<\/h3>\n\n\n\n<p>To setup Elastic Stack, follow the link below.<\/p>\n\n\n\n<p><a aria-label=\" (opens in a new tab)\" href=\"https:\/\/kifarunix.com\/install-elastic-stack-7-on-fedora-30-fedora-29-centos-7\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install Elastic Stack 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"install-filebeat-on-fedora-30-fedora-29-cent-os-7-1\">Install Filebeat on Fedora 30\/Fedora 29\/CentOS 7<\/h3>\n\n\n\n<p>Assuming you have already setup Elastic Stack, proceed to install Filebeat to collect your system logs for processing. In this guide, we are going to configure Filebeat to collect system authentication logs for processing.<\/p>\n\n\n\n<p>Update your system packages.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>yum update\nyum upgrade<\/code><\/pre>\n\n\n\n<p>Next, install Filebeat on Fedora 30\/Fedora 29\/CentOS 7. Installation can be done using RPM binary or using YUM repos.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"install-filebeat-7-using-rpm-repository\">Install Filebeat 7 using RPM Repository<\/h4>\n\n\n\n<p>Import the repository signing GPG key.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>sudo rpm --import https:\/\/packages.elastic.co\/GPG-KEY-elasticsearch<\/code><\/pre>\n\n\n\n<p>Next, install YUM Elastic repo.<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\ncat > \/etc\/yum.repos.d\/elastic-7.x.repo << EOF\n[elasticsearch-7.x]\nname=Elastic repository for 7.x packages\nbaseurl=https:\/\/artifacts.elastic.co\/packages\/7.x\/yum\ngpgcheck=1\ngpgkey=https:\/\/artifacts.elastic.co\/GPG-KEY-elasticsearch\nenabled=1\nautorefresh=1\ntype=rpm-md\nEOF\n<\/code><\/pre>\n\n\n\n<p>Install Filebeat.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>yum install filebeat<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"install-filebeat-using-rpm-binary\">Install Filebeat Using RPM Binary<\/h4>\n\n\n\n<p>Download the binary by executing the command below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -L -O https:\/\/artifacts.elastic.co\/downloads\/beats\/filebeat\/filebeat-7.2.0-x86_64.rpm<\/code><\/pre>\n\n\n\n<p>Install Filebeat<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>yum localinstall filebeat-7.2.0-x86_64.rpm<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"configure-filebeat-7-on-fedora-30-fedora-29-cent-os-7\"><a href=\"#configurefilebeattocollectlogs\">Configure Filebeat 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/h3>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"configure-filebeat-output\"><a href=\"#configurefilebeatoutput\">Configure Filebeat Output<\/a><\/h4>\n\n\n\n<p>Next, configure Filebeat to sent event data to Elastic stack. Filebeat can ship logs directly to Elasticsearch or to Logstash or other<a href=\"https:\/\/www.elastic.co\/guide\/en\/beats\/filebeat\/current\/configuring-output.html\" target=\"_blank\" rel=\"noreferrer noopener\"> outputs<\/a>. The Filebeat output is defined on the Filebeat configuration file, <strong><code>\/etc\/filebeat\/filebeat.yml<\/code><\/strong>.<\/p>\n\n\n\n<p>To send event data or event logs directly to Elasticsearch, open the configuration file and define Elasticsearch output as follows;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>vim \/etc\/filebeat\/filebeat.yml<\/code><\/pre>\n\n\n\n<p>Elasticsearch is the default output. All you need to do is update the IP address, Elasticsearch, which is set to localhost by default;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n...\n#================================ Outputs =====================================\n \n# Configure what output to use when sending the data collected by the beat.\n \n#-------------------------- Elasticsearch output ------------------------------\noutput.elasticsearch:\n  # Array of hosts to connect to.\n  #hosts: [\"localhost:9200\"]\n  <strong>hosts: [\"192.168.43.75:9200\"]<\/strong>\n...\n<\/code><\/pre>\n\n\n\n<p>If you are instead pushing event data to Logstash, comment out the Elasticsearch output and define Logstash output as shown below;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n#================================ Outputs =====================================\n \n# Configure what output to use when sending the data collected by the beat.\n \n#-------------------------- Elasticsearch output ------------------------------\n<strong>#output.elasticsearch:\n  # Array of hosts to connect to.\n  #hosts: [\"localhost:9200\"]<\/strong>\n \n  # Protocol - either `http` (default) or `https`.\n  #protocol: \"https\"\n \n  # Authentication credentials - either API key or username\/password.\n  #api_key: \"id:api_key\"\n  #username: \"elastic\"\n  #password: \"changeme\"\n \n#----------------------------- Logstash output --------------------------------\n<strong>output.logstash:<\/strong>\n  # The Logstash hosts\n  #hosts: [\"localhost:5044\"]\n  <strong>hosts: [\"192.168.43.75:5044\"]<\/strong>\n<\/code><\/pre>\n\n\n\n<p>For each output chosen, ensure that the ports are reachable. For example you can verify connection to Logstash;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>telnet 192.168.43.75 5044<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>Trying 192.168.43.75...\nConnected to 192.168.43.75.\nEscape character is '^]'.<\/code><\/pre>\n\n\n\n<p>Similarly, if you are using Elasticsearch directly, ensure that you can reach port 9200\/tcp.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"enable-filebeat-system-module\">Enable Filebeat System Module<\/h4>\n\n\n\n<p>In this setup, our Logstash was configured to process system authentication events. Hence, enable the System module which collects and parses logs created by the system logging service of common Unix\/Linux based distributions. This module is disabled by default.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>filebeat modules enable system<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>Enabled system<\/code><\/pre>\n\n\n\n<p>Configure system module to read authentication logs only. Simply set the value of syslog to <strong>false<\/strong>.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>vim \/etc\/filebeat\/modules.d\/system.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n...\n- module: system\n  # Syslog\n  syslog:\n  enabled: false\n\n    # Set custom paths for the log files. If left empty,\n    # Filebeat will choose the paths depending on your OS.\n    #var.paths:\n\n    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.\n    #var.convert_timezone: false\n\n  # Authorization logs\n  auth:\n    enabled: true\n\n    # Set custom paths for the log files. If left empty,\n    # Filebeat will choose the paths depending on your OS.\n    var.paths: [\"\/var\/log\/secure\"]\n...\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"load-the-index-template-in-elasticsearch\">Load the index template in Elasticsearch<\/h4>\n\n\n\n<p>If you are sending data directly to Elasticsearch, Filebeat will load the template automatically after successfully connecting to Elasticsearch.<\/p>\n\n\n\n<p>However, if you are using Logstash as the event data process engine, you need to manually load the index template into Elasticsearch. Hence, ensure that there a connection to Elasticsearch before you can load the index template.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>telnet 192.168.43.75 9200<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>Trying 192.168.43.75...\nConnected to 192.168.43.75.\nEscape character is '^]'.<\/code><\/pre>\n\n\n\n<p>If all is well., load the template.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>filebeat setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=[\"192.168.43.75:9200\"]'<\/code><\/pre>\n\n\n\n<p>If you see the output, <strong>Index setup finished<\/strong>, template load was successful.<\/p>\n\n\n\n<p>If the host doesn\u2019t have direct connectivity to Elasticsearch, you can generate the index template, copy it to Elastic Stack Server and install it locally.<\/p>\n\n\n\n<p>To generate the template;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>filebeat export template &gt; filebeat.template.json<\/code><\/pre>\n\n\n\n<p>To install the template on Elastic Stack server, copy it and run locally on Elastic Stack server.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -XPUT -H 'Content-Type: application\/json' http:\/\/192.168.43.75:9200\/_template\/filebeat-7.0.2 -d@filebeat.template.json<\/code><\/pre>\n\n\n\n<p>Once you are done with that, start and enable Filebeat to run on system boot.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl enable --now filebeat<\/code><\/pre>\n\n\n\n<p>You can run Filebeat in debug mode using the command below;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl stop filebeat<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>filebeat -e -c filebeatconfig.yml<\/code><\/pre>\n\n\n\n<p>By default, <strong><code>\/etc\/filebeat\/filebeat.yml<\/code><\/strong> is used. Hence, you can just run;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>filebeat -e<\/code><\/pre>\n\n\n\n<p>Press Ctrl+C to cancel and then start it;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl start filebeat<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"verify-elasticsearch-index-data-reception\">Verify Elasticsearch Index Data Reception<\/h3>\n\n\n\n<p>After the configuration above, simulate a failed and successful SSH authentication to the server on which Filebeat is running. Once that is done, login to Elastic stack server and verify data reception.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>curl -X GET 192.168.43.75:9200\/_cat\/indices?v<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nhealth status index                            uuid                   pri rep docs.count docs.deleted store.size pri.store.size\ngreen  open   .kibana_task_manager             xQztoO5CRoygONVw6ujEVg   1   0          2           18     27.8kb         27.8kb\nyellow open   ssh_auth-2019.07                 f6lBK5osQemJEb1lUtwGEQ   1   1         41            0    118.9kb        118.9kb\ngreen  open   .kibana_1                        1iR0TWklToSzoEBeZiE1Dg   1   0          3            1     43.2kb         43.2kb\nyellow open   filebeat-7.2.0-2019.07.02-000001 nelIPqlOSfKzGidOOk5C4g   1   1          0            0       283b           283b\n<\/code><\/pre>\n\n\n\n<p>After that, proceed to the Kibana and <strong>Create Index Pattern<\/strong>. See our guide on setting up <a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/install-elastic-stack-7-on-fedora-30-fedora-29-centos-7\/\" target=\"_blank\">Elastic Stack 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a>.<\/p>\n\n\n\n<p>You should now be able to see your SSH authentication events.<\/p>\n\n\n\n<p>SSH successful Logins<\/p>\n\n\n\n<figure class=\"wp-block-image\"><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-successful-ssh-logins.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1347\" height=\"637\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-successful-ssh-logins.png\" alt=\"Kibana SSH successful login events\" class=\"wp-image-3499\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-successful-ssh-logins.png 1347w, https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-successful-ssh-logins-768x363.png 768w\" sizes=\"(max-width: 1347px) 100vw, 1347px\" \/><\/a><\/figure>\n\n\n\n<p>SSH failed logins<\/p>\n\n\n\n<figure class=\"wp-block-image\"><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-failed-ssh-logins.png\"><img loading=\"lazy\" decoding=\"async\" width=\"1344\" height=\"640\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-failed-ssh-logins.png\" alt=\"Kibana SSH failed login events\" class=\"wp-image-3500\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-failed-ssh-logins.png 1344w, https:\/\/kifarunix.com\/wp-content\/uploads\/2019\/07\/kibana-failed-ssh-logins-768x366.png 768w\" sizes=\"(max-width: 1344px) 100vw, 1344px\" \/><\/a><\/figure>\n\n\n\n<p>Congratulations. That is all on how to install Filebeat on Fedora 30\/Fedora 29\/CentOS 7. Enjoy.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"other-related-guides\">Other Related Guides:<\/h3>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-configure-logstash-7-on-ubuntu-18-debian-9-8\/\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">Install and Configure Logstash 7 on Ubuntu 18\/Debian 9.8<\/a><\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/install-and-configure-filebeat-7-on-ubuntu-18-04-debian-9-8\/\" target=\"_blank\">Install and Configure Filebeat 7 on Ubuntu 18.04\/Debian 9.8<\/a><\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/install-elastic-stack-7-on-ubuntu-18-04-debian-9-8\/\" target=\"_blank\">Install Elastic Stack 7 on Ubuntu 18.04\/Debian 9.8<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-elasticsearch-7-x-on-ubuntu-18-04-debian-9-8\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install Elasticsearch 7.x on Ubuntu 18.04\/Debian 9.8<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this guide, we are going to learn how to install Filebeat on Fedora 30\/Fedora 29\/CentOS 7. Filebeat is a lightweight shipper for collecting, forwarding<\/p>\n","protected":false},"author":2,"featured_media":11333,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[72,910,121],"tags":[923,88,1654,912,913,289,924,922,1653,1652],"class_list":["post-3454","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-monitoring","category-elastic-stack","category-howtos","tag-beats","tag-centos-7","tag-configure-filebeat","tag-elastic-stack","tag-elk","tag-fedora-29","tag-fedora-30","tag-filebeat","tag-install-filebeat-on-centos-7","tag-install-filebeat-on-fedora-30-29","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/3454"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=3454"}],"version-history":[{"count":14,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/3454\/revisions"}],"predecessor-version":[{"id":17533,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/3454\/revisions\/17533"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/11333"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=3454"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=3454"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=3454"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}