{"id":15538,"date":"2023-03-06T01:09:46","date_gmt":"2023-03-05T22:09:46","guid":{"rendered":"https:\/\/kifarunix.com\/?p=15538"},"modified":"2024-03-10T08:30:56","modified_gmt":"2024-03-10T05:30:56","slug":"how-to-deploy-filebeat-using-ansible","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/how-to-deploy-filebeat-using-ansible\/","title":{"rendered":"How to Deploy Filebeat using Ansible"},"content":{"rendered":"\n<p>In this tutorial, you will learn how to deploy Filebeat using Ansible. <a href=\"https:\/\/www.ansible.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Ansible<\/a> is an open-source automation tool used for configuration management, application deployment, and task automation. It is designed to simplify IT automation by providing a way to automate tasks across a large number of computers.<\/p>\n\n\n\n<div class=\"wp-block-rank-math-toc-block\" id=\"rank-math-toc\"><h2>Table of Contents<\/h2><nav><ul><li><a href=\"#deploying-filebeat-using-ansible\">Deploying Filebeat using Ansible<\/a><ul><li><a href=\"#example-environment\">Example Environment<\/a><\/li><li><a href=\"#install-ansible-on-linux\">Install Ansible on Linux<\/a><\/li><li><a href=\"#create-ansible-user-on-managed-nodes\">Create Ansible User on Managed Nodes<\/a><\/li><li><a href=\"#setup-ansible-ssh-keys\">Setup Ansible SSH Keys<\/a><\/li><li><a href=\"#copy-ansible-ssh-public-key-to-managed-nodes\">Copy Ansible SSH Public Key to Managed Nodes<\/a><\/li><li><a href=\"#create-ansible-configuration-directory\">Create Ansible Configuration Directory<\/a><\/li><li><a href=\"#create-ansible-inventory-file\">Create Ansible Inventory File<\/a><\/li><li><a href=\"#run-ansible-passphrase-protected-ssh-key-without-prompting-for-passphrase\">Run Ansible Passphrase Protected SSH Key Without Prompting for Passphrase<\/a><\/li><li><a href=\"#test-connection-to-ansible-managed-nodes\">Test Connection to Ansible Managed Nodes<\/a><\/li><li><a href=\"#create-filebeat-playbook-roles-and-tasks\">Create Filebeat Playbook Roles and Tasks<\/a><ul><li><a href=\"#create-ansible-task-to-install-filebeat-on-debian-systems\">Create Ansible task to install Filebeat on Debian systems<\/a><\/li><li><a href=\"#create-ansible-task-to-install-filebeat-on-rhel-systems\">Create Ansible task to install Filebeat on RHEL systems<\/a><\/li><li><a href=\"#create-main-ansible-tasks-to-install-filebeat\">Create Main Ansible tasks to install Filebeat<\/a><\/li><li><a href=\"#create-main-ansible-playbook\">Create Main Ansible Playbook<\/a><\/li><\/ul><\/li><li><a href=\"#use-ansible-to-deploy-filebeat\">Use Ansible to Deploy Filebeat<\/a><\/li><li><a href=\"#more-tutorials\">More Tutorials<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"deploying-filebeat-using-ansible\">Deploying Filebeat using Ansible<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"example-environment\">Example Environment<\/h3>\n\n\n\n<p>To demonstrate how Ansible can be used to deploy Filebeat on various Linux systems in an environment, we have four systems (<strong><code>managed nodes<\/code><\/strong>) we will deploy Filebeat on using Ansible;<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table><tbody><tr><td><strong>Linux Distribution<\/strong><\/td><td><strong>IP Address<\/strong><\/td><\/tr><tr><td>Oracle Linux 9<\/td><td>192.168.56.152<\/td><\/tr><tr><td>Rocky Linux 9<\/td><td>192.168.56.144<\/td><\/tr><tr><td>Debian 11<\/td><td>192.168.56.153<\/td><\/tr><tr><td>Ubuntu 22.04<\/td><td>192.168.56.142<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>We also have;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>ELK (8) server running on Ubuntu 22.04, IP address 192.168.56.124<\/li>\n\n\n\n<li>Ansible Control Node running on Ubuntu 22.04, IP address 192.168.56.10<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"install-ansible-on-linux\">Install Ansible on Linux<\/h3>\n\n\n\n<p>To begin with, you need to Ansible on your <strong><code>control node<\/code><\/strong>. A control node, in context of Ansible IT automation, is the machine where Ansible is installed and from which all the automation tasks are executed.<\/p>\n\n\n\n<p>In this guide, we use an Ubuntu 22.04 node as our Ansible control node.<\/p>\n\n\n\n<p>Thus, to install Ansible on Ubuntu 22.04, proceed as follows;<\/p>\n\n\n\n<p>Run system update;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt install python3-pip -y<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>python3 -m pip install --upgrade pip<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo python3 -m pip install ansible<\/code><\/pre>\n\n\n\n<p>Enable Ansible auto-completion;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo python3 -m pip install argcomplete<\/code><\/pre>\n\n\n\n<p>Confirm the Ansible version;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible --version<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nansible [core 2.14.3]\n  config file = None\n  configured module search path = ['\/home\/kifarunix\/.ansible\/plugins\/modules', '\/usr\/share\/ansible\/plugins\/modules']\n  ansible python module location = \/usr\/local\/lib\/python3.10\/dist-packages\/ansible\n  ansible collection location = \/home\/kifarunix\/.ansible\/collections:\/usr\/share\/ansible\/collections\n  executable location = \/usr\/local\/bin\/ansible\n  python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] (\/usr\/bin\/python3)\n  jinja version = 3.0.3\n  libyaml = True\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-ansible-user-on-managed-nodes\">Create Ansible User on Managed Nodes<\/h3>\n\n\n\n<p>Ansible uses the user you are logged in on control node as to connect to all remote devices.<\/p>\n\n\n\n<p>If the user doesn&#8217;t exists on the remote hosts, you need to create the account.<\/p>\n\n\n\n<p>So, on all Ansible remotely managed hosts, create an account with sudo rights. This is the only manual task you might need to do at the beggining, -:).<\/p>\n\n\n\n<p>On Debian based systems;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo useradd -G sudo -s \/bin\/bash -m kifadmin<\/code><\/pre>\n\n\n\n<p>On RHEL based systems;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo useradd -G wheel -s \/bin\/bash -m kifadmin<\/code><\/pre>\n\n\n\n<p>Set the account password;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo passwd kifadmin<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"setup-ansible-ssh-keys\">Setup Ansible SSH Keys<\/h3>\n\n\n\n<p>When using Ansible, you have two options for authenticating with remote systems:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>SSH keys or<\/li>\n\n\n\n<li>password authentication<\/li>\n<\/ul>\n\n\n\n<p>SSH keys authentication is more preferred as a method of connection and is considered more secure and convenient compared to password authentication.<\/p>\n\n\n\n<p>Therefore, on the Ansible control node, you need to generate a public and private key pair and then copy the public key to the remote system&#8217;s authorized keys file.<\/p>\n\n\n\n<p>Thus, as a non root user, on the control node, run the command below to generate SSH key pair. It is recommended to use SSH keys with a passphrase for increased security.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ssh-keygen<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\nGenerating public\/private rsa key pair.\nEnter file in which to save the key (\/home\/kifarunix\/.ssh\/id_rsa): \/home\/kifarunix\/.ssh\/id_ansible_rsa\nEnter passphrase (empty for no passphrase): \nEnter same passphrase again: \nYour identification has been saved in \/home\/kifarunix\/.ssh\/id_ansible_rsa\nYour public key has been saved in \/home\/kifarunix\/.ssh\/id_ansible_rsa.pub\nThe key fingerprint is:\nSHA256:lXmQC0UVM1bIpMeg1t0L0tofaLRPSEW65RnO7hVGcsQ kifarunix@control-node\nThe key's randomart image is:\n+---[RSA 3072]----+\n|         o==B==. |\n|        .o.@o*.E |\n|        o.B.@.+o |\n|       . ..O @++ |\n|        S . B Oo |\n|           . =...|\n|              + .|\n|             . . |\n|              .  |\n+----[SHA256]-----+\n<\/code><\/pre>\n\n\n\n<p>Note that we already had another SSH key pair stored on the default file, <strong><code>\/home\/kifarunix\/.ssh\/id_rsa<\/code><\/strong> used for other services. Hence, why we had to create the key pair on a different file, <strong><code>\/home\/kifarunix\/.ssh\/id_ansible_rsa<\/code><\/strong>.<\/p>\n\n\n\n<p>In Ansible configuration file, you can define custom path to SSH key using the <strong><code>private_key_file<\/code><\/strong> option.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"copy-ansible-ssh-public-key-to-managed-nodes\">Copy Ansible SSH Public Key to Managed Nodes<\/h3>\n\n\n\n<p>Once you have the SSH key pair generated, proceed to copy the key to the user accounts you will use on managed nodes for Ansible deployment tasks;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in 142 144 152 153; do ssh-copy-id kifadmin@192.168.56.$i; done<\/code><\/pre>\n\n\n\n<p>If you are using non default SSH key file, specify the path to the key using the <code>-i<\/code> option;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in 142 144 152 153; do ssh-copy-id -i \/home\/kifarunix\/.ssh\/id_ansible_rsa kifadmin@192.168.56.$i; done<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-ansible-configuration-directory\">Create Ansible Configuration Directory<\/h3>\n\n\n\n<p>By default, <strong><code>\/etc\/ansible<\/code><\/strong> is a default Ansible directory.<\/p>\n\n\n\n<p>If you install Ansible via PIP, chances are, default configuration for Ansible is not created.<\/p>\n\n\n\n<p>Thus, to create you can create your custom configuration directory to easily control Ansible.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir $HOME\/ansible<\/code><\/pre>\n\n\n\n<p>Similarly, create your custom Ansible configuration file, <code>ansible.cfg<\/code>.<\/p>\n\n\n\n<p>This is how a default Ansible configuration looks like by default;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/etc\/ansible\/ansible.cfg<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n# config file for ansible -- https:\/\/ansible.com\/\n# ===============================================\n\n# nearly all parameters can be overridden in ansible-playbook\n# or with command line flags. ansible will read ANSIBLE_CONFIG,\n# ansible.cfg in the current working directory, .ansible.cfg in\n# the home directory or \/etc\/ansible\/ansible.cfg, whichever it\n# finds first\n\n[defaults]\n\n# some basic default values...\n\n#inventory      = \/etc\/ansible\/hosts\n#library        = \/usr\/share\/my_modules\/\n#module_utils   = \/usr\/share\/my_module_utils\/\n#remote_tmp     = ~\/.ansible\/tmp\n#local_tmp      = ~\/.ansible\/tmp\n#plugin_filters_cfg = \/etc\/ansible\/plugin_filters.yml\n#forks          = 5\n#poll_interval  = 15\n#sudo_user      = root\n#ask_sudo_pass = True\n#ask_pass      = True\n#transport      = smart\n#remote_port    = 22\n#module_lang    = C\n#module_set_locale = False\n\n# plays will gather facts by default, which contain information about\n# the remote system.\n#\n# smart - gather by default, but don't regather if already gathered\n# implicit - gather by default, turn off with gather_facts: False\n# explicit - do not gather by default, must say gather_facts: True\n#gathering = implicit\n\n# This only affects the gathering done by a play's gather_facts directive,\n# by default gathering retrieves all facts subsets\n# all - gather all subsets\n# network - gather min and network facts\n# hardware - gather hardware facts (longest facts to retrieve)\n# virtual - gather min and virtual facts\n# facter - import facts from facter\n# ohai - import facts from ohai\n# You can combine them using comma (ex: network,virtual)\n# You can negate them using ! (ex: !hardware,!facter,!ohai)\n# A minimal set of facts is always gathered.\n#gather_subset = all\n\n# some hardware related facts are collected\n# with a maximum timeout of 10 seconds. This\n# option lets you increase or decrease that\n# timeout to something more suitable for the\n# environment.\n# gather_timeout = 10\n\n# Ansible facts are available inside the ansible_facts.* dictionary\n# namespace. This setting maintains the behaviour which was the default prior\n# to 2.5, duplicating these variables into the main namespace, each with a\n# prefix of 'ansible_'.\n# This variable is set to True by default for backwards compatibility. It\n# will be changed to a default of 'False' in a future release.\n# ansible_facts.\n# inject_facts_as_vars = True\n\n# additional paths to search for roles in, colon separated\n#roles_path    = \/etc\/ansible\/roles\n\n# uncomment this to disable SSH key host checking\n#host_key_checking = False\n\n# change the default callback, you can only have one 'stdout' type  enabled at a time.\n#stdout_callback = skippy\n\n\n## Ansible ships with some plugins that require whitelisting,\n## this is done to avoid running all of a type by default.\n## These setting lists those that you want enabled for your system.\n## Custom plugins should not need this unless plugin author specifies it.\n\n# enable callback plugins, they can output to stdout but cannot be 'stdout' type.\n#callback_whitelist = timer, mail\n\n# Determine whether includes in tasks and handlers are \"static\" by\n# default. As of 2.0, includes are dynamic by default. Setting these\n# values to True will make includes behave more like they did in the\n# 1.x versions.\n#task_includes_static = False\n#handler_includes_static = False\n\n# Controls if a missing handler for a notification event is an error or a warning\n#error_on_missing_handler = True\n\n# change this for alternative sudo implementations\n#sudo_exe = sudo\n\n# What flags to pass to sudo\n# WARNING: leaving out the defaults might create unexpected behaviours\n#sudo_flags = -H -S -n\n\n# SSH timeout\n#timeout = 10\n\n# default user to use for playbooks if user is not specified\n# (\/usr\/bin\/ansible will use current user as default)\n#remote_user = root\n\n# logging is off by default unless this path is defined\n# if so defined, consider logrotate\n#log_path = \/var\/log\/ansible.log\n\n# default module name for \/usr\/bin\/ansible\n#module_name = command\n\n# use this shell for commands executed under sudo\n# you may need to change this to bin\/bash in rare instances\n# if sudo is constrained\n#executable = \/bin\/sh\n\n# if inventory variables overlap, does the higher precedence one win\n# or are hash values merged together?  The default is 'replace' but\n# this can also be set to 'merge'.\n#hash_behaviour = replace\n\n# by default, variables from roles will be visible in the global variable\n# scope. To prevent this, the following option can be enabled, and only\n# tasks and handlers within the role will see the variables there\n#private_role_vars = yes\n\n# list any Jinja2 extensions to enable here:\n#jinja2_extensions = jinja2.ext.do,jinja2.ext.i18n\n\n# if set, always use this private key file for authentication, same as\n# if passing --private-key to ansible or ansible-playbook\n#private_key_file = \/path\/to\/file\n\n# If set, configures the path to the Vault password file as an alternative to\n# specifying --vault-password-file on the command line.\n#vault_password_file = \/path\/to\/vault_password_file\n\n# format of string {{ ansible_managed }} available within Jinja2\n# templates indicates to users editing templates files will be replaced.\n# replacing {file}, {host} and {uid} and strftime codes with proper values.\n#ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}\n# {file}, {host}, {uid}, and the timestamp can all interfere with idempotence\n# in some situations so the default is a static string:\n#ansible_managed = Ansible managed\n\n# by default, ansible-playbook will display \"Skipping [host]\" if it determines a task\n# should not be run on a host.  Set this to \"False\" if you don't want to see these \"Skipping\"\n# messages. NOTE: the task header will still be shown regardless of whether or not the\n# task is skipped.\n#display_skipped_hosts = True\n\n# by default, if a task in a playbook does not include a name: field then\n# ansible-playbook will construct a header that includes the task's action but\n# not the task's args.  This is a security feature because ansible cannot know\n# if the *module* considers an argument to be no_log at the time that the\n# header is printed.  If your environment doesn't have a problem securing\n# stdout from ansible-playbook (or you have manually specified no_log in your\n# playbook on all of the tasks where you have secret information) then you can\n# safely set this to True to get more informative messages.\n#display_args_to_stdout = False\n\n# by default (as of 1.3), Ansible will raise errors when attempting to dereference\n# Jinja2 variables that are not set in templates or action lines. Uncomment this line\n# to revert the behavior to pre-1.3.\n#error_on_undefined_vars = False\n\n# by default (as of 1.6), Ansible may display warnings based on the configuration of the\n# system running ansible itself. This may include warnings about 3rd party packages or\n# other conditions that should be resolved if possible.\n# to disable these warnings, set the following value to False:\n#system_warnings = True\n\n# by default (as of 1.4), Ansible may display deprecation warnings for language\n# features that should no longer be used and will be removed in future versions.\n# to disable these warnings, set the following value to False:\n#deprecation_warnings = True\n\n# (as of 1.8), Ansible can optionally warn when usage of the shell and\n# command module appear to be simplified by using a default Ansible module\n# instead.  These warnings can be silenced by adjusting the following\n# setting or adding warn=yes or warn=no to the end of the command line\n# parameter string.  This will for example suggest using the git module\n# instead of shelling out to the git command.\n# command_warnings = False\n\n\n# set plugin path directories here, separate with colons\n#action_plugins     = \/usr\/share\/ansible\/plugins\/action\n#become_plugins     = \/usr\/share\/ansible\/plugins\/become\n#cache_plugins      = \/usr\/share\/ansible\/plugins\/cache\n#callback_plugins   = \/usr\/share\/ansible\/plugins\/callback\n#connection_plugins = \/usr\/share\/ansible\/plugins\/connection\n#lookup_plugins     = \/usr\/share\/ansible\/plugins\/lookup\n#inventory_plugins  = \/usr\/share\/ansible\/plugins\/inventory\n#vars_plugins       = \/usr\/share\/ansible\/plugins\/vars\n#filter_plugins     = \/usr\/share\/ansible\/plugins\/filter\n#test_plugins       = \/usr\/share\/ansible\/plugins\/test\n#terminal_plugins   = \/usr\/share\/ansible\/plugins\/terminal\n#strategy_plugins   = \/usr\/share\/ansible\/plugins\/strategy\n\n\n# by default, ansible will use the 'linear' strategy but you may want to try\n# another one\n#strategy = free\n\n# by default callbacks are not loaded for \/bin\/ansible, enable this if you\n# want, for example, a notification or logging callback to also apply to\n# \/bin\/ansible runs\n#bin_ansible_callbacks = False\n\n\n# don't like cows?  that's unfortunate.\n# set to 1 if you don't want cowsay support or export ANSIBLE_NOCOWS=1\n#nocows = 1\n\n# set which cowsay stencil you'd like to use by default. When set to 'random',\n# a random stencil will be selected for each task. The selection will be filtered\n# against the `cow_whitelist` option below.\n#cow_selection = default\n#cow_selection = random\n\n# when using the 'random' option for cowsay, stencils will be restricted to this list.\n# it should be formatted as a comma-separated list with no spaces between names.\n# NOTE: line continuations here are for formatting purposes only, as the INI parser\n#       in python does not support them.\n#cow_whitelist=bud-frogs,bunny,cheese,daemon,default,dragon,elephant-in-snake,elephant,eyes,\\\n#              hellokitty,kitty,luke-koala,meow,milk,moofasa,moose,ren,sheep,small,stegosaurus,\\\n#              stimpy,supermilker,three-eyes,turkey,turtle,tux,udder,vader-koala,vader,www\n\n# don't like colors either?\n# set to 1 if you don't want colors, or export ANSIBLE_NOCOLOR=1\n#nocolor = 1\n\n# if set to a persistent type (not 'memory', for example 'redis') fact values\n# from previous runs in Ansible will be stored.  This may be useful when\n# wanting to use, for example, IP information from one group of servers\n# without having to talk to them in the same playbook run to get their\n# current IP information.\n#fact_caching = memory\n\n#This option tells Ansible where to cache facts. The value is plugin dependent.\n#For the jsonfile plugin, it should be a path to a local directory.\n#For the redis plugin, the value is a host:port:database triplet: fact_caching_connection = localhost:6379:0\n\n#fact_caching_connection=\/tmp\n\n\n\n# retry files\n# When a playbook fails a .retry file can be created that will be placed in ~\/\n# You can enable this feature by setting retry_files_enabled to True\n# and you can change the location of the files by setting retry_files_save_path\n\n#retry_files_enabled = False\n#retry_files_save_path = ~\/.ansible-retry\n\n# squash actions\n# Ansible can optimise actions that call modules with list parameters\n# when looping. Instead of calling the module once per with_ item, the\n# module is called once with all items at once. Currently this only works\n# under limited circumstances, and only with parameters named 'name'.\n#squash_actions = apk,apt,dnf,homebrew,pacman,pkgng,yum,zypper\n\n# prevents logging of task data, off by default\n#no_log = False\n\n# prevents logging of tasks, but only on the targets, data is still logged on the master\/controller\n#no_target_syslog = False\n\n# controls whether Ansible will raise an error or warning if a task has no\n# choice but to create world readable temporary files to execute a module on\n# the remote machine.  This option is False by default for security.  Users may\n# turn this on to have behaviour more like Ansible prior to 2.1.x.  See\n# https:\/\/docs.ansible.com\/ansible\/become.html#becoming-an-unprivileged-user\n# for more secure ways to fix this than enabling this option.\n#allow_world_readable_tmpfiles = False\n\n# controls the compression level of variables sent to\n# worker processes. At the default of 0, no compression\n# is used. This value must be an integer from 0 to 9.\n#var_compression_level = 9\n\n# controls what compression method is used for new-style ansible modules when\n# they are sent to the remote system.  The compression types depend on having\n# support compiled into both the controller's python and the client's python.\n# The names should match with the python Zipfile compression types:\n# * ZIP_STORED (no compression. available everywhere)\n# * ZIP_DEFLATED (uses zlib, the default)\n# These values may be set per host via the ansible_module_compression inventory\n# variable\n#module_compression = 'ZIP_DEFLATED'\n\n# This controls the cutoff point (in bytes) on --diff for files\n# set to 0 for unlimited (RAM may suffer!).\n#max_diff_size = 1048576\n\n# This controls how ansible handles multiple --tags and --skip-tags arguments\n# on the CLI.  If this is True then multiple arguments are merged together.  If\n# it is False, then the last specified argument is used and the others are ignored.\n# This option will be removed in 2.8.\n#merge_multiple_cli_flags = True\n\n# Controls showing custom stats at the end, off by default\n#show_custom_stats = True\n\n# Controls which files to ignore when using a directory as inventory with\n# possibly multiple sources (both static and dynamic)\n#inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo\n\n# This family of modules use an alternative execution path optimized for network appliances\n# only update this setting if you know how this works, otherwise it can break module execution\n#network_group_modules=eos, nxos, ios, iosxr, junos, vyos\n\n# When enabled, this option allows lookups (via variables like {{lookup('foo')}} or when used as\n# a loop with `with_foo`) to return data that is not marked \"unsafe\". This means the data may contain\n# jinja2 templating language which will be run through the templating engine.\n# ENABLING THIS COULD BE A SECURITY RISK\n#allow_unsafe_lookups = False\n\n# set default errors for all plays\n#any_errors_fatal = False\n\n[inventory]\n# enable inventory plugins, default: 'host_list', 'script', 'auto', 'yaml', 'ini', 'toml'\n#enable_plugins = host_list, virtualbox, yaml, constructed\n\n# ignore these extensions when parsing a directory as inventory source\n#ignore_extensions = .pyc, .pyo, .swp, .bak, ~, .rpm, .md, .txt, ~, .orig, .ini, .cfg, .retry\n\n# ignore files matching these patterns when parsing a directory as inventory source\n#ignore_patterns=\n\n# If 'true' unparsed inventory sources become fatal errors, they are warnings otherwise.\n#unparsed_is_failed=False\n\n[privilege_escalation]\n#become=True\n#become_method=sudo\n#become_user=root\n#become_ask_pass=False\n\n[paramiko_connection]\n\n# uncomment this line to cause the paramiko connection plugin to not record new host\n# keys encountered.  Increases performance on new host additions.  Setting works independently of the\n# host key checking setting above.\n#record_host_keys=False\n\n# by default, Ansible requests a pseudo-terminal for commands executed under sudo. Uncomment this\n# line to disable this behaviour.\n#pty=False\n\n# paramiko will default to looking for SSH keys initially when trying to\n# authenticate to remote devices.  This is a problem for some network devices\n# that close the connection after a key failure.  Uncomment this line to\n# disable the Paramiko look for keys function\n#look_for_keys = False\n\n# When using persistent connections with Paramiko, the connection runs in a\n# background process.  If the host doesn't already have a valid SSH key, by\n# default Ansible will prompt to add the host key.  This will cause connections\n# running in background processes to fail.  Uncomment this line to have\n# Paramiko automatically add host keys.\n#host_key_auto_add = True\n\n[ssh_connection]\n\n# ssh arguments to use\n# Leaving off ControlPersist will result in poor performance, so use\n# paramiko on older platforms rather than removing it, -C controls compression use\n#ssh_args = -C -o ControlMaster=auto -o ControlPersist=60s\n\n# The base directory for the ControlPath sockets.\n# This is the \"%(directory)s\" in the control_path option\n#\n# Example:\n# control_path_dir = \/tmp\/.ansible\/cp\n#control_path_dir = ~\/.ansible\/cp\n\n# The path to use for the ControlPath sockets. This defaults to a hashed string of the hostname,\n# port and username (empty string in the config). The hash mitigates a common problem users\n# found with long hostnames and the conventional %(directory)s\/ansible-ssh-%%h-%%p-%%r format.\n# In those cases, a \"too long for Unix domain socket\" ssh error would occur.\n#\n# Example:\n# control_path = %(directory)s\/%%h-%%r\n#control_path =\n\n# Enabling pipelining reduces the number of SSH operations required to\n# execute a module on the remote server. This can result in a significant\n# performance improvement when enabled, however when using \"sudo:\" you must\n# first disable 'requiretty' in \/etc\/sudoers\n#\n# By default, this option is disabled to preserve compatibility with\n# sudoers configurations that have requiretty (the default on many distros).\n#\n#pipelining = False\n\n# Control the mechanism for transferring files (old)\n#   * smart = try sftp and then try scp [default]\n#   * True = use scp only\n#   * False = use sftp only\n#scp_if_ssh = smart\n\n# Control the mechanism for transferring files (new)\n# If set, this will override the scp_if_ssh option\n#   * sftp  = use sftp to transfer files\n#   * scp   = use scp to transfer files\n#   * piped = use 'dd' over SSH to transfer files\n#   * smart = try sftp, scp, and piped, in that order [default]\n#transfer_method = smart\n\n# if False, sftp will not use batch mode to transfer files. This may cause some\n# types of file transfer failures impossible to catch however, and should\n# only be disabled if your sftp version has problems with batch mode\n#sftp_batch_mode = False\n\n# The -tt argument is passed to ssh when pipelining is not enabled because sudo \n# requires a tty by default. \n#usetty = True\n\n# Number of times to retry an SSH connection to a host, in case of UNREACHABLE.\n# For each retry attempt, there is an exponential backoff,\n# so after the first attempt there is 1s wait, then 2s, 4s etc. up to 30s (max).\n#retries = 3\n\n[persistent_connection]\n\n# Configures the persistent connection timeout value in seconds.  This value is\n# how long the persistent connection will remain idle before it is destroyed.\n# If the connection doesn't receive a request before the timeout value\n# expires, the connection is shutdown. The default value is 30 seconds.\n#connect_timeout = 30\n\n# The command timeout value defines the amount of time to wait for a command\n# or RPC call before timing out. The value for the command timeout must\n# be less than the value of the persistent connection idle timeout (connect_timeout)\n# The default value is 30 second.\n#command_timeout = 30\n\n[accelerate]\n#accelerate_port = 5099\n#accelerate_timeout = 30\n#accelerate_connect_timeout = 5.0\n\n# The daemon timeout is measured in minutes. This time is measured\n# from the last activity to the accelerate daemon.\n#accelerate_daemon_timeout = 30\n\n# If set to yes, accelerate_multi_key will allow multiple\n# private keys to be uploaded to it, though each user must\n# have access to the system via SSH to add a new key. The default\n# is \"no\".\n#accelerate_multi_key = yes\n\n[selinux]\n# file systems that require special treatment when dealing with security context\n# the default behaviour that copies the existing context or uses the user default\n# needs to be changed to use the file system dependent context.\n#special_context_filesystems=nfs,vboxsf,fuse,ramfs,9p,vfat\n\n# Set this to yes to allow libvirt_lxc connections to work without SELinux.\n#libvirt_lxc_noseclabel = yes\n\n[colors]\n#highlight = white\n#verbose = blue\n#warn = bright purple\n#error = red\n#debug = dark gray\n#deprecate = purple\n#skip = cyan\n#unreachable = red\n#ok = green\n#changed = yellow\n#diff_add = green\n#diff_remove = red\n#diff_lines = cyan\n\n\n[diff]\n# Always print diff when running ( same as always running with -D\/--diff )\n# always = no\n\n# Set how many context lines to show in diff\n# context = 3\n<\/code><\/pre>\n\n\n\n<p>Based on the default config, we will create our own configuration file;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>vim $HOME\/ansible\/ansible.cfg<\/code><\/pre>\n\n\n\n<p>In the configuration file, we will define some basic values;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n[defaults]\ninventory       = \/home\/kifarunix\/ansible\/hosts\nroles_path      = \/home\/kifarunix\/ansible\/roles\nprivate_key_file        = \/home\/kifarunix\/.ssh\/id_ansible_rsa\ninterpreter_python      = \/usr\/bin\/python3\n...\n<\/code><\/pre>\n\n\n\n<p>Whenever you use a custom Ansible configuration directory, you can specify it using <code><strong>ANSIBLE_CONFIG<\/strong><\/code> environment variable or <strong><code>-c<\/code><\/strong> command-line option;<\/p>\n\n\n\n<p>Let&#8217;s set the environment variable to make it easy;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>echo \"export ANSIBLE_CONFIG=$HOME\/ansible\/ansible.cfg\" &gt;&gt; $HOME\/.bashrc<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>source $HOME\/.bashrc<\/code><\/pre>\n\n\n\n<p>With this, you wont have to specify the path to Ansible configuration or path to any path that is defined on the custom configuration.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-ansible-inventory-file\">Create Ansible Inventory File<\/h3>\n\n\n\n<p>The inventory file contains a list of hosts and groups of hosts that are managed by Ansible. By default, inventory file is set to <strong><code>\/etc\/ansible\/hosts<\/code><\/strong>. You can specify a different inventory file using the -i option or using the <strong><code>inventory<\/code><\/strong> option in the configuration file. We set out inventory file to <code><strong>\/home\/kifarunix\/ansible\/hosts<\/strong><\/code> in the configuration above;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>vim \/home\/kifarunix\/ansible\/hosts<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>&#91;debian_nodes]\n192.168.56.142\n192.168.56.153\n\n&#91;rhel_nodes]\n192.168.56.144\n192.168.56.152<\/code><\/pre>\n\n\n\n<p>Save and exit the file.<\/p>\n\n\n\n<p>There are different formats of the hosts file configuration. That is just an example.<\/p>\n\n\n\n<p>See below a sample hosts file!<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/etc\/ansible\/hosts<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n# This is the default ansible 'hosts' file.\n#\n# It should live in \/etc\/ansible\/hosts\n#\n#   - Comments begin with the '#' character\n#   - Blank lines are ignored\n#   - Groups of hosts are delimited by [header] elements\n#   - You can enter hostnames or ip addresses\n#   - A hostname\/ip can be a member of multiple groups\n\n# Ex 1: Ungrouped hosts, specify before any group headers.\n\n#green.example.com\n#blue.example.com\n#192.168.100.1\n#192.168.100.10\n\n# Ex 2: A collection of hosts belonging to the 'webservers' group\n\n#[webservers]\n#alpha.example.org\n#beta.example.org\n#192.168.1.100\n#192.168.1.110\n\n# If you have multiple hosts following a pattern you can specify\n# them like this:\n\n#www[001:006].example.com\n\n# Ex 3: A collection of database servers in the 'dbservers' group\n\n#[dbservers]\n#\n#db01.intranet.mydomain.net\n#db02.intranet.mydomain.net\n#10.25.1.56\n#10.25.1.57\n\n# Here's another example of host ranges, this time there are no\n# leading 0s:\n\n#db-[99:101]-node.example.com\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"run-ansible-passphrase-protected-ssh-key-without-prompting-for-passphrase\">Run Ansible Passphrase Protected SSH Key Without Prompting for Passphrase<\/h3>\n\n\n\n<p>We are trying to automate tasks here. However, if you setup passphrase protected SSH key, you will be prompted to enter the phrase for every single command to be ran against a managed node.<\/p>\n\n\n\n<p>As a work around for this, you can use <code>ssh-agent<\/code> to cache your passphrase, so that you only need to enter it once per session as follows;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>eval \"$(ssh-agent -s)\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>ssh-add \/home\/kifarunix\/.ssh\/id_ansible_rsa<\/code><\/pre>\n\n\n\n<p>You will be prompted to enter your passphrase.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"test-connection-to-ansible-managed-nodes\">Test Connection to Ansible Managed Nodes<\/h3>\n\n\n\n<p>Now that you have setup SSH key authentication and an inventory of the managed hosts, run Ansible ping module to check hosts connection and aliveness;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible all -m ping -u kifadmin<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n192.168.56.153 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n192.168.56.144 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n192.168.56.152 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n192.168.56.142 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n<\/code><\/pre>\n\n\n\n<p>All seems good so far!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-filebeat-playbook-roles-and-tasks\">Create Filebeat Playbook Roles and Tasks<\/h3>\n\n\n\n<p>Ansible Playbooks define a set of tasks that needs to be executed on each managed host. Ansible roles consists of a collection of tasks.<\/p>\n\n\n\n<p>This is our Ansible directory structure;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tree ~\/ansible\/<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n\/home\/kifarunix\/ansible\/\n\u251c\u2500\u2500 ansible.cfg\n\u251c\u2500\u2500 filebeat-configs\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 elastic-ca.crt\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 filebeat.yml\n\u251c\u2500\u2500 hosts\n\u251c\u2500\u2500 main.yml\n\u2514\u2500\u2500 roles\n    \u2514\u2500\u2500 filebeat\n        \u2514\u2500\u2500 tasks\n            \u251c\u2500\u2500 Debian.yml\n            \u251c\u2500\u2500 main.yml\n            \u2514\u2500\u2500 Rhel.yml\n\n4 directories, 8 files\n<\/code><\/pre>\n\n\n\n<p>In this example setup, we will create a role by the name <strong>filebeat<\/strong>. As mentioned, a roles is made of various tasks.<\/p>\n\n\n\n<p>Thus, let&#8217;s create a directory to work from.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir -p ~\/ansible\/roles\/filebeat\/tasks<\/code><\/pre>\n\n\n\n<p>Also, create a directory to store custom filebeat configurations.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir ~\/ansible\/filebeat-configs\/<\/code><\/pre>\n\n\n\n<p>So what tasks do we have in our Filebeat deployment?<\/p>\n\n\n\n<p>We have tasks to;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>install Filebeat on Debian\/Ubuntu systems<\/li>\n\n\n\n<li>install Filebeat on RHEL based systems (Rocky Linux\/Oracle Linux)<\/li>\n<\/ul>\n\n\n\n<p>Note that we are installing Filbeat 8.x since we are running ELK v8.5.2.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-ansible-task-to-install-filebeat-on-debian-systems\">Create Ansible task to install Filebeat on Debian systems<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code>vim ~\/ansible\/roles\/filebeat\/tasks\/Debian.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n---\n- name: Install gnupg\n  apt:\n    name: gnupg2\n    state: present\n    update_cache: yes\n\n- name: Install ELK APT Repo Signing Key\n  apt_key:\n    url: \"{{ ELK_APT_REPO_KEY }}\"\n    state: present\n\n- name: Install ELK 8.x APT Repo \n  apt_repository:\n    repo: \"{{ ELK_8_APT_REPO }}\"\n    state: present\n\n- name: Update Repo cache and install Filebeat\n  apt:\n    name: filebeat=8.5.2\n    update_cache: yes\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-ansible-task-to-install-filebeat-on-rhel-systems\">Create Ansible task to install Filebeat on RHEL systems<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code>vim ~\/ansible\/roles\/filebeat\/tasks\/Rhel.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n---\n- name: Install ELK YUM Repo Signing Key\n  rpm_key:\n    key: \"{{ ELK_YUM_REPO_KEY }}\"\n    state: present\n\n- name: Install ELK 8.x YUM Repo\n  yum_repository:\n    name: elastic-8.x\n    description: Elastic repository for 8.x packages\n    baseurl: \"{{ ELK_8_YUM_REPO }}\"\n    gpgcheck: yes\n    enabled: yes\n\n- name: Update Repo cache and install Filebeat\n  yum:\n    name: filebeat-8.5.2\n    update_cache: yes\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-main-ansible-tasks-to-install-filebeat\">Create Main Ansible tasks to install Filebeat<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code>vim ~\/ansible\/roles\/filebeat\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n---\n- name: Debian\/Ubuntu Filebeat Installation Task\n  include_tasks: \"Debian.yml\"\n  when: ansible_distribution in ['Debian', 'Ubuntu']\n\n- name: RedHat\/Rocky\/CentOS Agent Installation Task\n  include_tasks: \"Rhel.yml\"\n  when: ansible_os_family == \"RedHat\"\n\n- name: Enable Filebeat System Module\n    shell: |\n    filebeat modules enable system\n    sed -i 's\/false\/true\/g' \/etc\/filebeat\/modules.d\/system.yml\n  when: ansible_distribution in ['Debian', 'Ubuntu'] or ansible_os_family == \"RedHat\"\n\n- name: Backup existing Filebeat configuration file\n  copy:\n    src: \/etc\/filebeat\/filebeat.yml\n    dest: \/etc\/filebeat\/filebeat.yml.bak\n    remote_src: yes\n    backup: yes\n  when: ansible_distribution in ['Debian', 'Ubuntu'] or ansible_os_family == \"RedHat\"\n\n- name: Copy custom Filebeat configuration file\n  copy:\n    src: \/home\/kifarunix\/ansible\/filebeat-configs\/filebeat.yml \n    dest: \/etc\/filebeat\/filebeat.yml\n  when: ansible_distribution in ['Debian', 'Ubuntu'] or ansible_os_family == \"RedHat\"\n\n- name: Copy ELK CA Cert file\n  copy:\n    src: \/home\/kifarunix\/ansible\/filebeat-configs\/elastic-ca.crt\n    dest: \/etc\/ssl\/certs\/elastic-ca.crt\n  when: ansible_distribution in ['Debian', 'Ubuntu'] or ansible_os_family == \"RedHat\"\n\n- name: Update hosts file with ELK server address\n  lineinfile:\n    path: \/etc\/hosts\n    line: \"{{ elk_ip }} {{ elk_hostname }} {{ elk_alias }}\"\n    state: present\n  when: ansible_distribution in ['Debian', 'Ubuntu'] or ansible_os_family == \"RedHat\"\n\n- name: Start and enable Filebeat Service\n  systemd:\n    name: filebeat\n    state: started\n    enabled: yes\n  when: ansible_distribution in ['Debian', 'Ubuntu'] or ansible_os_family == \"RedHat\"\n<\/code><\/pre>\n\n\n\n<p>The above is pretty much self-explanatory.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-main-ansible-playbook\">Create Main Ansible Playbook<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code>vim ~\/ansible\/main.yml<\/code><\/pre>\n\n\n\n<p>Here, we define group of hosts to run the installation against, variables applied to the play, roles required to execute, user to run the roles with&#8230;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\n---\n- hosts: all\n  gather_facts: True\n  vars:\n    elk_ip: \"192.168.56.124\"\n    elk_hostname: \"elk.kifarunix.com\"\n    elk_alias: \"elk\"\n  remote_user: kifadmin\n  become: yes\n  roles:\n    - filebeat\n<\/code><\/pre>\n\n\n\n<p>We also updated hosts file with respective OS family variables;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat ~\/ansible\/hosts<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\n[debian_nodes]\n192.168.56.142\n192.168.56.153\n\n[rhel_nodes]\n192.168.56.144\n192.168.56.152\n\n[debian_nodes:vars]\nELK_APT_REPO_KEY=https:\/\/artifacts.elastic.co\/GPG-KEY-elasticsearch\nELK_8_APT_REPO=deb https:\/\/artifacts.elastic.co\/packages\/8.x\/apt stable main\n\n[rhel_nodes:vars]\nELK_YUM_REPO_KEY=https:\/\/packages.elastic.co\/GPG-KEY-elasticsearch\nELK_8_YUM_REPO=https:\/\/artifacts.elastic.co\/packages\/8.x\/yum\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"use-ansible-to-deploy-filebeat\">Use Ansible to Deploy Filebeat<\/h3>\n\n\n\n<p>First of all, let&#8217;s ensure we are not prompted for SSH key passphrase<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>eval \"$(ssh-agent -s)\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>ssh-add \/home\/kifarunix\/.ssh\/id_ansible_rsa<\/code><\/pre>\n\n\n\n<p>Next, run a dry run of your Ansible playbooks to check what would happen and fix and would be issue;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible-playbook ~\/ansible\/main.yml -C --ask-become-pass<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\nBECOME password: \n\nPLAY [all] *****************************************************************************************************************************************************************\n\nTASK [Gathering Facts] *****************************************************************************************************************************************************\nok: [192.168.56.144]\nok: [192.168.56.152]\nok: [192.168.56.142]\nok: [192.168.56.153]\n\nTASK [filebeat : Debian\/Ubuntu Filebeat Installation Task] *****************************************************************************************************************\nskipping: [192.168.56.144]\nskipping: [192.168.56.152]\nincluded: \/home\/kifarunix\/ansible\/roles\/filebeat\/tasks\/Debian.yml for 192.168.56.142, 192.168.56.153\n\nTASK [filebeat : Install gnupg] ********************************************************************************************************************************************\nchanged: [192.168.56.142]\nchanged: [192.168.56.153]\n\nTASK [filebeat : Install ELK APT Repo Signing Key] *************************************************************************************************************************\nchanged: [192.168.56.142]\nchanged: [192.168.56.153]\n\nTASK [filebeat : Install ELK 8.x APT Repo] *********************************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.142]\n\nTASK [filebeat : Update Repo cache and install Filebeat] *******************************************************************************************************************\nfatal: [192.168.56.153]: FAILED! => {\"changed\": false, \"msg\": \"No package matching 'filebeat' is available\"}\nfatal: [192.168.56.142]: FAILED! => {\"changed\": false, \"msg\": \"No package matching 'filebeat' is available\"}\n\nTASK [filebeat : RedHat\/Rocky\/CentOS Agent Installation Task] **************************************************************************************************************\nincluded: \/home\/kifarunix\/ansible\/roles\/filebeat\/tasks\/Rhel.yml for 192.168.56.144, 192.168.56.152\n\nTASK [filebeat : update-crypto-policies to import the key] *****************************************************************************************************************\nskipping: [192.168.56.144]\nskipping: [192.168.56.152]\n\nTASK [filebeat : Install ELK YUM Repo Signing Key] *************************************************************************************************************************\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Install ELK 8.x YUM Repo] *********************************************************************************************************************************\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Update Repo cache and install Filebeat] *******************************************************************************************************************\nfatal: [192.168.56.152]: FAILED! => {\"changed\": false, \"failures\": [\"No package filebeat-8.5.2 available.\"], \"msg\": \"Failed to install some of the specified packages\", \"rc\": 1, \"results\": []}\nfatal: [192.168.56.144]: FAILED! => {\"changed\": false, \"failures\": [\"No package filebeat-8.5.2 available.\"], \"msg\": \"Failed to install some of the specified packages\", \"rc\": 1, \"results\": []}\n\nPLAY RECAP *****************************************************************************************************************************************************************\n192.168.56.142             : ok=5    changed=3    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   \n192.168.56.144             : ok=4    changed=2    unreachable=0    failed=1    skipped=2    rescued=0    ignored=0   \n192.168.56.152             : ok=4    changed=2    unreachable=0    failed=1    skipped=2    rescued=0    ignored=0   \n192.168.56.153             : ok=5    changed=3    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0\n<\/code><\/pre>\n\n\n\n<p>Seems good. errors encountered is due to files that are not installed yet.<\/p>\n\n\n\n<p>Let&#8217;s run the Ansible playbook to deploy Filebeat by omitting option <strong><code>-C<\/code><\/strong> in the command above.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible-playbook ~\/ansible\/main.yml --ask-become-pass<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nBECOME password: \n\nPLAY [all] *****************************************************************************************************************************************************************\n\nTASK [Gathering Facts] *****************************************************************************************************************************************************\nok: [192.168.56.153]\nok: [192.168.56.144]\nok: [192.168.56.152]\nok: [192.168.56.142]\n\nTASK [filebeat : Debian\/Ubuntu Filebeat Installation Task] *****************************************************************************************************************\nskipping: [192.168.56.144]\nskipping: [192.168.56.152]\nincluded: \/home\/kifarunix\/ansible\/roles\/filebeat\/tasks\/Debian.yml for 192.168.56.142, 192.168.56.153\n\nTASK [filebeat : Install gnupg] ********************************************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.142]\n\nTASK [filebeat : Install ELK APT Repo Signing Key] *************************************************************************************************************************\nchanged: [192.168.56.142]\nchanged: [192.168.56.153]\n\nTASK [filebeat : Install ELK 8.x APT Repo] *********************************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.142]\n\nTASK [filebeat : Update Repo cache and install Filebeat] *******************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.142]\n\nTASK [filebeat : RedHat\/Rocky\/CentOS Agent Installation Task] **************************************************************************************************************\nskipping: [192.168.56.142]\nskipping: [192.168.56.153]\nincluded: \/home\/kifarunix\/ansible\/roles\/filebeat\/tasks\/Rhel.yml for 192.168.56.144, 192.168.56.152\n\nTASK [filebeat : update-crypto-policies to import the key] *****************************************************************************************************************\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Install ELK YUM Repo Signing Key] *************************************************************************************************************************\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Install ELK 8.x YUM Repo] *********************************************************************************************************************************\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Update Repo cache and install Filebeat] *******************************************************************************************************************\nchanged: [192.168.56.152]\nchanged: [192.168.56.144]\n\nTASK [filebeat : update-crypto-policies to default] ************************************************************************************************************************\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Enable Filebeat System Module] ****************************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.152]\nchanged: [192.168.56.144]\nchanged: [192.168.56.142]\n\nTASK [filebeat : Backup existing Filebeat configuration file] **************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.144]\nchanged: [192.168.56.142]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Copy custom Filebeat configuration file] ******************************************************************************************************************\nchanged: [192.168.56.153]\nchanged: [192.168.56.142]\nchanged: [192.168.56.144]\nchanged: [192.168.56.152]\n\nTASK [filebeat : Copy ELK CA Cert file] ************************************************************************************************************************************\nok: [192.168.56.142]\nok: [192.168.56.153]\nok: [192.168.56.144]\nok: [192.168.56.152]\n\nTASK [filebeat : Update hosts file with ELK server address] ****************************************************************************************************************\nchanged: [192.168.56.142]\nchanged: [192.168.56.153]\nok: [192.168.56.144]\nok: [192.168.56.152]\n\nTASK [filebeat : Start and enable Filebeat Service] ************************************************************************************************************************\nchanged: [192.168.56.153]\nok: [192.168.56.144]\nchanged: [192.168.56.142]\nchanged: [192.168.56.152]\n\nPLAY RECAP *****************************************************************************************************************************************************************\n192.168.56.142             : ok=12   changed=9    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   \n192.168.56.144             : ok=13   changed=8    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   \n192.168.56.152             : ok=13   changed=9    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   \n192.168.56.153             : ok=12   changed=9    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0\n<\/code><\/pre>\n\n\n\n<p>Filebeat should now be installed and running on all the nodes;<\/p>\n\n\n\n<p>Confirm if status of filebeat;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible -m shell -a \"systemctl status filebeat\" --ask-become-pass -u kifadmin all<\/code><\/pre>\n\n\n\n<p>Login to Kibana dashboard and confirm if events are being received from the nodes;<\/p>\n\n\n\n<div><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/03\/deploy-filebeat-using-ansible.png\" class=\"td-modal-image\"><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1897\" height=\"926\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/03\/deploy-filebeat-using-ansible.png\" alt=\"How to Deploy Filebeat using Ansible\" class=\"wp-image-15705\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/03\/deploy-filebeat-using-ansible.png?v=1678053814 1897w, https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/03\/deploy-filebeat-using-ansible-768x375.png?v=1678053814 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2023\/03\/deploy-filebeat-using-ansible-1536x750.png?v=1678053814 1536w\" sizes=\"(max-width: 1897px) 100vw, 1897px\" \/><\/figure><\/a><\/div>\n\n\n\n<p>Feel free to modify the playbooks as you so wish.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"more-tutorials\">More Tutorials<\/h3>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-setup-ansible-on-ubuntu-20-04\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Setup Ansible on Ubuntu 20.04\/Ubuntu 22.04<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/how-to-fix-filebeat-glibc-related-errors\/\" target=\"_blank\" rel=\"noreferrer noopener\">How to Fix Filebeat Glibc Related Errors on Ubuntu 22.04<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, you will learn how to deploy Filebeat using Ansible. Ansible is an open-source automation tool used for configuration management, application deployment, and<\/p>\n","protected":false},"author":10,"featured_media":15512,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[121,1884,992,1722],"tags":[6430,6427,6429,6428],"class_list":["post-15538","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-howtos","category-ansible","category-automation","category-automation-2","tag-ansible-playbooks-for-ansible","tag-deploy-filebeat-with-ansible","tag-filebeat-ansible-playbooks","tag-use-ansible-to-deploy-filebeat","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/15538"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=15538"}],"version-history":[{"count":20,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/15538\/revisions"}],"predecessor-version":[{"id":20707,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/15538\/revisions\/20707"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/15512"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=15538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=15538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=15538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}