{"id":19950,"date":"2024-01-31T00:09:37","date_gmt":"2024-01-30T21:09:37","guid":{"rendered":"https:\/\/kifarunix.com\/?p=19950"},"modified":"2024-03-10T15:32:47","modified_gmt":"2024-03-10T12:32:47","slug":"deploy-elk-stack-8-cluster-on-docker-using-ansible","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/deploy-elk-stack-8-cluster-on-docker-using-ansible\/","title":{"rendered":"Deploy ELK Stack 8 Cluster on Docker using Ansible"},"content":{"rendered":"\n<p>In this tutorial, you will learn how to deploy ELK stack 8 Cluster on Docker using <a href=\"https:\/\/docs.ansible.com\/ansible\/latest\/getting_started\/index.html\" target=\"_blank\" rel=\"noreferrer noopener\">Ansible<\/a>, an an open-source automation tool used for configuration management, application deployment, and task automation.<\/p>\n\n\n\n<div class=\"wp-block-rank-math-toc-block\" id=\"rank-math-toc\"><h2>Table of Contents<\/h2><nav><ul><li><a href=\"#deploying-elk-stack-8-cluster-on-docker-using-ansible\">Deploying ELK stack 8 Cluster on Docker using Ansible<\/a><ul><li><a href=\"#deployment-environment-nodes\">Deployment Environment Nodes<\/a><\/li><li><a href=\"#install-ansible-on-linux\">Install Ansible on Linux<\/a><\/li><li><a href=\"#create-ansible-user-on-managed-nodes\">Create Ansible User on Managed Nodes<\/a><\/li><li><a href=\"#setup-ansible-ssh-keys\">Setup Ansible SSH Keys<\/a><\/li><li><a href=\"#copy-ansible-ssh-public-key-to-managed-nodes\">Copy Ansible SSH Public Key to Managed Nodes<\/a><\/li><li><a href=\"#create-ansible-configuration-directory\">Create Ansible Configuration Directory<\/a><\/li><li><a href=\"#create-ansible-inventory-file\">Create Ansible Inventory File<\/a><\/li><li><a href=\"#run-ansible-passphrase-protected-ssh-key-without-prompting-for-passphrase\">Run Ansible Passphrase Protected SSH Key Without Prompting for Passphrase<\/a><\/li><li><a href=\"#test-connection-to-ansible-managed-nodes\">Test Connection to Ansible Managed Nodes<\/a><\/li><li><a href=\"#create-elk-stack-8-docker-containers-playbook-roles-and-tasks\">Create ELK Stack 8 Docker Containers Playbook Roles and Tasks<\/a><ul><li><a href=\"#ansible-playbook-variables\">Ansible Playbook Variables<\/a><\/li><li><a href=\"#tasks-to-install-docker-and-docker-compose-on-the-nodes\">Tasks to Install Docker and Docker Compose on the Nodes<\/a><\/li><li><a href=\"#important-elasticsearch-settings\">Important Elasticsearch Settings<\/a><\/li><li><a href=\"#create-nfs-share-on-nfs-server\">Create NFS Share on NFS Server<\/a><\/li><li><a href=\"#elk-stack-configs\">ELK Stack Configs<\/a><\/li><li><a href=\"#copy-elk-stack-configs-and-docker-compose-files-to-respective-nodes\">Copy ELK Stack Configs and Docker Compose Files to Respective Nodes<\/a><\/li><li><a href=\"#deploying-elk-stack-8-cluster-on-docker-using-ansible-1\">Deploying ELK Stack 8 Cluster on Docker using Ansible<\/a><\/li><li><a href=\"#remove-cluster-initial-master-nodes-setting\">Remove cluster.initial_master_nodes\u00a0setting<\/a><\/li><li><a href=\"#docker-clean-up-role\">Docker Clean Up Role<\/a><\/li><li><a href=\"#create-main-ansible-playbook\">Create Main Ansible Playbook<\/a><\/li><\/ul><\/li><li><a href=\"#perform-elk-stack-deployment-via-ansible-dry-run\">Perform ELK Stack Deployment via Ansible Dry Run<\/a><\/li><li><a href=\"#deploy-elk-stack-8-cluster-on-docker-containers-using-ansible\">Deploy ELK Stack 8 Cluster on Docker Containers using Ansible<\/a><\/li><li><a href=\"#verify-elk-stack-8-cluster\">Verify ELK Stack 8 Cluster<\/a><\/li><li><a href=\"#accessing-elk-stack-kibana-interface\">Accessing ELK Stack Kibana Interface<\/a><\/li><li><a href=\"#tear-down-the-stack\">Tear Down the Stack<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"deploying-elk-stack-8-cluster-on-docker-using-ansible\">Deploying ELK stack 8 Cluster on Docker using Ansible<\/h2>\n\n\n\n<p>In our previous tutorial, we learnt how to <a href=\"https:\/\/kifarunix.com\/deploy-elk-stack-8-cluster-on-docker-containers\/\" target=\"_blank\" rel=\"noreferrer noopener\">deploy ELK stack 8 cluster using Docker containers<\/a>. In this tutorial, we are looking at how we can automate the whole process to ensure a seamless deployment of ELK stack 8 Cluster on Docker containers from a single point across the cluster nodes without breaking a sweat!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"deployment-environment-nodes\">Deployment Environment Nodes<\/h3>\n\n\n\n<p>In this guide, we have multiple nodes for various purposes. See the table below;<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Linux Distribution<\/strong><\/td><td><strong>Node IP Address<\/strong><\/td><td><strong>Node Role<\/strong><\/td><\/tr><tr><td>Debian 12<\/td><td>192.168.122.229<\/td><td>Ansible Control Plane<\/td><\/tr><tr><td>Debian 12<\/td><td>192.168.122.60<\/td><td>ES node 01<\/td><\/tr><tr><td>Debian 12<\/td><td>192.168.122.152<\/td><td>ES node 02<\/td><\/tr><tr><td>Debian 12<\/td><td>192.168.122.123<\/td><td>ES node 03<\/td><\/tr><tr><td>Debian 12<\/td><td>192.168.122.47<\/td><td>NFS Server<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"install-ansible-on-linux\">Install Ansible on Linux<\/h3>\n\n\n\n<p>Install Ansible on your <strong><code>control node<\/code><\/strong>. A control node is the machine where Ansible is installed and from which all the automation tasks are executed.<\/p>\n\n\n\n<p>Thus, to install Ansible on Debian 12, proceed as follows;<\/p>\n\n\n\n<p>Run system update;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt update<\/code><\/pre>\n\n\n\n<p>Install and upgrade PIP3<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt install python3-pip -y<\/code><\/pre>\n\n\n\n<p>Create a virtual environment to keep your Ansible installation isolated from your system&#8217;s Python environment;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo apt install python3-venv<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>python3 -m venv ansible-env<\/code><\/pre>\n\n\n\n<p>Activate Ansible Python&#8217;s virtual environment and install PIP3;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>source ansible-env\/bin\/activate<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>python3 -m pip install --upgrade pip<\/code><\/pre>\n\n\n\n<p>Install Ansible;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>python3 -m pip install ansible<\/code><\/pre>\n\n\n\n<p>Enable Ansible auto-completion;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>python3 -m pip install argcomplete<\/code><\/pre>\n\n\n\n<p>Confirm the Ansible version;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible --version<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>ansible [core 2.16.2]\n  config file = None\n  configured module search path = ['\/home\/kifarunix\/.ansible\/plugins\/modules', '\/usr\/share\/ansible\/plugins\/modules']\n  ansible python module location = \/home\/kifarunix\/ansible-env\/lib\/python3.11\/site-packages\/ansible\n  ansible collection location = \/home\/kifarunix\/.ansible\/collections:\/usr\/share\/ansible\/collections\n  executable location = \/home\/kifarunix\/ansible-env\/bin\/ansible\n  python version = 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] (\/home\/kifarunix\/ansible-env\/bin\/python3)\n  jinja version = 3.1.3\n  libyaml = True\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-ansible-user-on-managed-nodes\">Create Ansible User on Managed Nodes<\/h3>\n\n\n\n<p>Ansible uses the user you are logged in on control node as to connect to all remote devices.<\/p>\n\n\n\n<p>If the user doesn&#8217;t exists on the remote hosts, you need to create the account.<\/p>\n\n\n\n<p>So, on all Ansible remotely managed hosts, create an account with sudo rights. This is the only manual task you might need to do at the beggining, -:).<\/p>\n\n\n\n<p>Replace the user, <code>kifarunix<\/code>, below with your username.<\/p>\n\n\n\n<p>On Debian based systems;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo useradd -G sudo -s \/bin\/bash -m kifarunix<\/code><\/pre>\n\n\n\n<p>On RHEL based systems;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo useradd -G wheel -s \/bin\/bash -m kifarunix<\/code><\/pre>\n\n\n\n<p>Set the account password;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo passwd kifarunix<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"setup-ansible-ssh-keys\">Setup Ansible SSH Keys<\/h3>\n\n\n\n<p>When using Ansible, you have two options for authenticating with remote systems:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>SSH keys or<\/li>\n\n\n\n<li>Password Authentication<\/li>\n<\/ul>\n\n\n\n<p>SSH keys authentication is more preferred as a method of connection and is considered more secure and convenient compared to password authentication.<\/p>\n\n\n\n<p>Therefore, on the Ansible control node, you need to generate a public and private SSH key pair. Once you have the keys, copy the public key to the remote system&#8217;s authorized keys file.<\/p>\n\n\n\n<p>Thus, as a non root user, on the control node, run the command below to generate SSH key pair. It is recommended to use SSH keys with a passphrase for increased security.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ssh-keygen<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>Generating public\/private rsa key pair.\nEnter file in which to save the key (\/home\/kifarunix\/.ssh\/id_rsa): \nEnter passphrase (empty for no passphrase): \nEnter same passphrase again: \nYour identification has been saved in \/home\/kifarunix\/.ssh\/id_rsa\nYour public key has been saved in \/home\/kifarunix\/.ssh\/id_rsa.pub\nThe key fingerprint is:\nSHA256:mv7b7OynAaft9wnDNniAfxE+dzMJTKP6Ag53DMb4SpE kifarunix@kifarunix-dev\nThe key's randomart image is:\n+---[RSA 3072]----+\n|             o   |\n|      +     + .  |\n|     E +   ..o   |\n|      + o... .. .|\n|     o +S+o + .+.|\n|    . =oo*.+ + .o|\n|     .o...=.O    |\n|     .   =.=o+ . |\n|      ..o+O+ .o  |\n+----[SHA256]-----+\n<\/code><\/pre>\n\n\n\n<p>If you used a different path other than the default one to store SSH key pair, then in Ansible configuration file, you can define custom path to SSH key using the <strong><code>private_key_file<\/code><\/strong> option.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"copy-ansible-ssh-public-key-to-managed-nodes\">Copy Ansible SSH Public Key to Managed Nodes<\/h3>\n\n\n\n<p>Once you have the SSH key pair generated, proceed to copy the key to the user accounts you will use on managed nodes for Ansible deployment tasks;<\/p>\n\n\n\n<p>(We are using the name, <strong><code>kifarunix<\/code><\/strong>, in this guide. Replace the username accordingly)<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in 60 152 123 47; do ssh-copy-id kifarunix@192.168.122.$i; done<\/code><\/pre>\n\n\n\n<p>If you are using non default SSH key file, specify the path to the key using the <code>-i<\/code> option;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for i in 142 144 152 153; do ssh-copy-id -i &lt;custom-ssh-key-path&gt; kifarunix@192.168.56.$i; done<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-ansible-configuration-directory\">Create Ansible Configuration Directory<\/h3>\n\n\n\n<p>By default, <strong><code>\/etc\/ansible<\/code><\/strong> is a default Ansible directory.<\/p>\n\n\n\n<p>If you install Ansible via PIP, chances are, default configuration for Ansible is not created.<\/p>\n\n\n\n<p>Thus, to create you can create your custom configuration directory to easily control Ansible.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir $HOME\/ansible<\/code><\/pre>\n\n\n\n<p>Similarly, create your custom Ansible configuration file, <code>ansible.cfg<\/code>.<\/p>\n\n\n\n<p>This is how a default Ansible configuration looks like by default;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/etc\/ansible\/ansible.cfg<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code># config file for ansible -- https:\/\/ansible.com\/\n# ===============================================\n\n# nearly all parameters can be overridden in ansible-playbook\n# or with command line flags. ansible will read ANSIBLE_CONFIG,\n# ansible.cfg in the current working directory, .ansible.cfg in\n# the home directory or \/etc\/ansible\/ansible.cfg, whichever it\n# finds first\n\n[defaults]\n\n# some basic default values...\n\n#inventory      = \/etc\/ansible\/hosts\n#library        = \/usr\/share\/my_modules\/\n#module_utils   = \/usr\/share\/my_module_utils\/\n#remote_tmp     = ~\/.ansible\/tmp\n#local_tmp      = ~\/.ansible\/tmp\n#plugin_filters_cfg = \/etc\/ansible\/plugin_filters.yml\n#forks          = 5\n#poll_interval  = 15\n#sudo_user      = root\n#ask_sudo_pass = True\n#ask_pass      = True\n#transport      = smart\n#remote_port    = 22\n#module_lang    = C\n#module_set_locale = False\n\n# plays will gather facts by default, which contain information about\n# the remote system.\n#\n# smart - gather by default, but don't regather if already gathered\n# implicit - gather by default, turn off with gather_facts: False\n# explicit - do not gather by default, must say gather_facts: True\n#gathering = implicit\n\n# This only affects the gathering done by a play's gather_facts directive,\n# by default gathering retrieves all facts subsets\n# all - gather all subsets\n# network - gather min and network facts\n# hardware - gather hardware facts (longest facts to retrieve)\n# virtual - gather min and virtual facts\n# facter - import facts from facter\n# ohai - import facts from ohai\n# You can combine them using comma (ex: network,virtual)\n# You can negate them using ! (ex: !hardware,!facter,!ohai)\n# A minimal set of facts is always gathered.\n#gather_subset = all\n\n# some hardware related facts are collected\n# with a maximum timeout of 10 seconds. This\n# option lets you increase or decrease that\n# timeout to something more suitable for the\n# environment.\n# gather_timeout = 10\n\n# Ansible facts are available inside the ansible_facts.* dictionary\n# namespace. This setting maintains the behaviour which was the default prior\n# to 2.5, duplicating these variables into the main namespace, each with a\n# prefix of 'ansible_'.\n# This variable is set to True by default for backwards compatibility. It\n# will be changed to a default of 'False' in a future release.\n# ansible_facts.\n# inject_facts_as_vars = True\n\n# additional paths to search for roles in, colon separated\n#roles_path    = \/etc\/ansible\/roles\n\n# uncomment this to disable SSH key host checking\n#host_key_checking = False\n\n# change the default callback, you can only have one 'stdout' type  enabled at a time.\n#stdout_callback = skippy\n\n\n## Ansible ships with some plugins that require whitelisting,\n## this is done to avoid running all of a type by default.\n## These setting lists those that you want enabled for your system.\n## Custom plugins should not need this unless plugin author specifies it.\n\n# enable callback plugins, they can output to stdout but cannot be 'stdout' type.\n#callback_whitelist = timer, mail\n\n# Determine whether includes in tasks and handlers are \"static\" by\n# default. As of 2.0, includes are dynamic by default. Setting these\n# values to True will make includes behave more like they did in the\n# 1.x versions.\n#task_includes_static = False\n#handler_includes_static = False\n\n# Controls if a missing handler for a notification event is an error or a warning\n#error_on_missing_handler = True\n\n# change this for alternative sudo implementations\n#sudo_exe = sudo\n\n# What flags to pass to sudo\n# WARNING: leaving out the defaults might create unexpected behaviours\n#sudo_flags = -H -S -n\n\n# SSH timeout\n#timeout = 10\n\n# default user to use for playbooks if user is not specified\n# (\/usr\/bin\/ansible will use current user as default)\n#remote_user = root\n\n# logging is off by default unless this path is defined\n# if so defined, consider logrotate\n#log_path = \/var\/log\/ansible.log\n\n# default module name for \/usr\/bin\/ansible\n#module_name = command\n\n# use this shell for commands executed under sudo\n# you may need to change this to bin\/bash in rare instances\n# if sudo is constrained\n#executable = \/bin\/sh\n\n# if inventory variables overlap, does the higher precedence one win\n# or are hash values merged together?  The default is 'replace' but\n# this can also be set to 'merge'.\n#hash_behaviour = replace\n\n# by default, variables from roles will be visible in the global variable\n# scope. To prevent this, the following option can be enabled, and only\n# tasks and handlers within the role will see the variables there\n#private_role_vars = yes\n\n# list any Jinja2 extensions to enable here:\n#jinja2_extensions = jinja2.ext.do,jinja2.ext.i18n\n\n# if set, always use this private key file for authentication, same as\n# if passing --private-key to ansible or ansible-playbook\n#private_key_file = \/path\/to\/file\n\n# If set, configures the path to the Vault password file as an alternative to\n# specifying --vault-password-file on the command line.\n#vault_password_file = \/path\/to\/vault_password_file\n\n# format of string {{ ansible_managed }} available within Jinja2\n# templates indicates to users editing templates files will be replaced.\n# replacing {file}, {host} and {uid} and strftime codes with proper values.\n#ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}\n# {file}, {host}, {uid}, and the timestamp can all interfere with idempotence\n# in some situations so the default is a static string:\n#ansible_managed = Ansible managed\n\n# by default, ansible-playbook will display \"Skipping [host]\" if it determines a task\n# should not be run on a host.  Set this to \"False\" if you don't want to see these \"Skipping\"\n# messages. NOTE: the task header will still be shown regardless of whether or not the\n# task is skipped.\n#display_skipped_hosts = True\n\n# by default, if a task in a playbook does not include a name: field then\n# ansible-playbook will construct a header that includes the task's action but\n# not the task's args.  This is a security feature because ansible cannot know\n# if the *module* considers an argument to be no_log at the time that the\n# header is printed.  If your environment doesn't have a problem securing\n# stdout from ansible-playbook (or you have manually specified no_log in your\n# playbook on all of the tasks where you have secret information) then you can\n# safely set this to True to get more informative messages.\n#display_args_to_stdout = False\n\n# by default (as of 1.3), Ansible will raise errors when attempting to dereference\n# Jinja2 variables that are not set in templates or action lines. Uncomment this line\n# to revert the behavior to pre-1.3.\n#error_on_undefined_vars = False\n\n# by default (as of 1.6), Ansible may display warnings based on the configuration of the\n# system running ansible itself. This may include warnings about 3rd party packages or\n# other conditions that should be resolved if possible.\n# to disable these warnings, set the following value to False:\n#system_warnings = True\n\n# by default (as of 1.4), Ansible may display deprecation warnings for language\n# features that should no longer be used and will be removed in future versions.\n# to disable these warnings, set the following value to False:\n#deprecation_warnings = True\n\n# (as of 1.8), Ansible can optionally warn when usage of the shell and\n# command module appear to be simplified by using a default Ansible module\n# instead.  These warnings can be silenced by adjusting the following\n# setting or adding warn=yes or warn=no to the end of the command line\n# parameter string.  This will for example suggest using the git module\n# instead of shelling out to the git command.\n# command_warnings = False\n\n\n# set plugin path directories here, separate with colons\n#action_plugins     = \/usr\/share\/ansible\/plugins\/action\n#become_plugins     = \/usr\/share\/ansible\/plugins\/become\n#cache_plugins      = \/usr\/share\/ansible\/plugins\/cache\n#callback_plugins   = \/usr\/share\/ansible\/plugins\/callback\n#connection_plugins = \/usr\/share\/ansible\/plugins\/connection\n#lookup_plugins     = \/usr\/share\/ansible\/plugins\/lookup\n#inventory_plugins  = \/usr\/share\/ansible\/plugins\/inventory\n#vars_plugins       = \/usr\/share\/ansible\/plugins\/vars\n#filter_plugins     = \/usr\/share\/ansible\/plugins\/filter\n#test_plugins       = \/usr\/share\/ansible\/plugins\/test\n#terminal_plugins   = \/usr\/share\/ansible\/plugins\/terminal\n#strategy_plugins   = \/usr\/share\/ansible\/plugins\/strategy\n\n\n# by default, ansible will use the 'linear' strategy but you may want to try\n# another one\n#strategy = free\n\n# by default callbacks are not loaded for \/bin\/ansible, enable this if you\n# want, for example, a notification or logging callback to also apply to\n# \/bin\/ansible runs\n#bin_ansible_callbacks = False\n\n\n# don't like cows?  that's unfortunate.\n# set to 1 if you don't want cowsay support or export ANSIBLE_NOCOWS=1\n#nocows = 1\n\n# set which cowsay stencil you'd like to use by default. When set to 'random',\n# a random stencil will be selected for each task. The selection will be filtered\n# against the `cow_whitelist` option below.\n#cow_selection = default\n#cow_selection = random\n\n# when using the 'random' option for cowsay, stencils will be restricted to this list.\n# it should be formatted as a comma-separated list with no spaces between names.\n# NOTE: line continuations here are for formatting purposes only, as the INI parser\n#       in python does not support them.\n#cow_whitelist=bud-frogs,bunny,cheese,daemon,default,dragon,elephant-in-snake,elephant,eyes,\\\n#              hellokitty,kitty,luke-koala,meow,milk,moofasa,moose,ren,sheep,small,stegosaurus,\\\n#              stimpy,supermilker,three-eyes,turkey,turtle,tux,udder,vader-koala,vader,www\n\n# don't like colors either?\n# set to 1 if you don't want colors, or export ANSIBLE_NOCOLOR=1\n#nocolor = 1\n\n# if set to a persistent type (not 'memory', for example 'redis') fact values\n# from previous runs in Ansible will be stored.  This may be useful when\n# wanting to use, for example, IP information from one group of servers\n# without having to talk to them in the same playbook run to get their\n# current IP information.\n#fact_caching = memory\n\n#This option tells Ansible where to cache facts. The value is plugin dependent.\n#For the jsonfile plugin, it should be a path to a local directory.\n#For the redis plugin, the value is a host:port:database triplet: fact_caching_connection = localhost:6379:0\n\n#fact_caching_connection=\/tmp\n\n\n\n# retry files\n# When a playbook fails a .retry file can be created that will be placed in ~\/\n# You can enable this feature by setting retry_files_enabled to True\n# and you can change the location of the files by setting retry_files_save_path\n\n#retry_files_enabled = False\n#retry_files_save_path = ~\/.ansible-retry\n\n# squash actions\n# Ansible can optimise actions that call modules with list parameters\n# when looping. Instead of calling the module once per with_ item, the\n# module is called once with all items at once. Currently this only works\n# under limited circumstances, and only with parameters named 'name'.\n#squash_actions = apk,apt,dnf,homebrew,pacman,pkgng,yum,zypper\n\n# prevents logging of task data, off by default\n#no_log = False\n\n# prevents logging of tasks, but only on the targets, data is still logged on the master\/controller\n#no_target_syslog = False\n\n# controls whether Ansible will raise an error or warning if a task has no\n# choice but to create world readable temporary files to execute a module on\n# the remote machine.  This option is False by default for security.  Users may\n# turn this on to have behaviour more like Ansible prior to 2.1.x.  See\n# https:\/\/docs.ansible.com\/ansible\/become.html#becoming-an-unprivileged-user\n# for more secure ways to fix this than enabling this option.\n#allow_world_readable_tmpfiles = False\n\n# controls the compression level of variables sent to\n# worker processes. At the default of 0, no compression\n# is used. This value must be an integer from 0 to 9.\n#var_compression_level = 9\n\n# controls what compression method is used for new-style ansible modules when\n# they are sent to the remote system.  The compression types depend on having\n# support compiled into both the controller's python and the client's python.\n# The names should match with the python Zipfile compression types:\n# * ZIP_STORED (no compression. available everywhere)\n# * ZIP_DEFLATED (uses zlib, the default)\n# These values may be set per host via the ansible_module_compression inventory\n# variable\n#module_compression = 'ZIP_DEFLATED'\n\n# This controls the cutoff point (in bytes) on --diff for files\n# set to 0 for unlimited (RAM may suffer!).\n#max_diff_size = 1048576\n\n# This controls how ansible handles multiple --tags and --skip-tags arguments\n# on the CLI.  If this is True then multiple arguments are merged together.  If\n# it is False, then the last specified argument is used and the others are ignored.\n# This option will be removed in 2.8.\n#merge_multiple_cli_flags = True\n\n# Controls showing custom stats at the end, off by default\n#show_custom_stats = True\n\n# Controls which files to ignore when using a directory as inventory with\n# possibly multiple sources (both static and dynamic)\n#inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo\n\n# This family of modules use an alternative execution path optimized for network appliances\n# only update this setting if you know how this works, otherwise it can break module execution\n#network_group_modules=eos, nxos, ios, iosxr, junos, vyos\n\n# When enabled, this option allows lookups (via variables like {{lookup('foo')}} or when used as\n# a loop with `with_foo`) to return data that is not marked \"unsafe\". This means the data may contain\n# jinja2 templating language which will be run through the templating engine.\n# ENABLING THIS COULD BE A SECURITY RISK\n#allow_unsafe_lookups = False\n\n# set default errors for all plays\n#any_errors_fatal = False\n\n[inventory]\n# enable inventory plugins, default: 'host_list', 'script', 'auto', 'yaml', 'ini', 'toml'\n#enable_plugins = host_list, virtualbox, yaml, constructed\n\n# ignore these extensions when parsing a directory as inventory source\n#ignore_extensions = .pyc, .pyo, .swp, .bak, ~, .rpm, .md, .txt, ~, .orig, .ini, .cfg, .retry\n\n# ignore files matching these patterns when parsing a directory as inventory source\n#ignore_patterns=\n\n# If 'true' unparsed inventory sources become fatal errors, they are warnings otherwise.\n#unparsed_is_failed=False\n\n[privilege_escalation]\n#become=True\n#become_method=sudo\n#become_user=root\n#become_ask_pass=False\n\n[paramiko_connection]\n\n# uncomment this line to cause the paramiko connection plugin to not record new host\n# keys encountered.  Increases performance on new host additions.  Setting works independently of the\n# host key checking setting above.\n#record_host_keys=False\n\n# by default, Ansible requests a pseudo-terminal for commands executed under sudo. Uncomment this\n# line to disable this behaviour.\n#pty=False\n\n# paramiko will default to looking for SSH keys initially when trying to\n# authenticate to remote devices.  This is a problem for some network devices\n# that close the connection after a key failure.  Uncomment this line to\n# disable the Paramiko look for keys function\n#look_for_keys = False\n\n# When using persistent connections with Paramiko, the connection runs in a\n# background process.  If the host doesn't already have a valid SSH key, by\n# default Ansible will prompt to add the host key.  This will cause connections\n# running in background processes to fail.  Uncomment this line to have\n# Paramiko automatically add host keys.\n#host_key_auto_add = True\n\n[ssh_connection]\n\n# ssh arguments to use\n# Leaving off ControlPersist will result in poor performance, so use\n# paramiko on older platforms rather than removing it, -C controls compression use\n#ssh_args = -C -o ControlMaster=auto -o ControlPersist=60s\n\n# The base directory for the ControlPath sockets.\n# This is the \"%(directory)s\" in the control_path option\n#\n# Example:\n# control_path_dir = \/tmp\/.ansible\/cp\n#control_path_dir = ~\/.ansible\/cp\n\n# The path to use for the ControlPath sockets. This defaults to a hashed string of the hostname,\n# port and username (empty string in the config). The hash mitigates a common problem users\n# found with long hostnames and the conventional %(directory)s\/ansible-ssh-%%h-%%p-%%r format.\n# In those cases, a \"too long for Unix domain socket\" ssh error would occur.\n#\n# Example:\n# control_path = %(directory)s\/%%h-%%r\n#control_path =\n\n# Enabling pipelining reduces the number of SSH operations required to\n# execute a module on the remote server. This can result in a significant\n# performance improvement when enabled, however when using \"sudo:\" you must\n# first disable 'requiretty' in \/etc\/sudoers\n#\n# By default, this option is disabled to preserve compatibility with\n# sudoers configurations that have requiretty (the default on many distros).\n#\n#pipelining = False\n\n# Control the mechanism for transferring files (old)\n#   * smart = try sftp and then try scp [default]\n#   * True = use scp only\n#   * False = use sftp only\n#scp_if_ssh = smart\n\n# Control the mechanism for transferring files (new)\n# If set, this will override the scp_if_ssh option\n#   * sftp  = use sftp to transfer files\n#   * scp   = use scp to transfer files\n#   * piped = use 'dd' over SSH to transfer files\n#   * smart = try sftp, scp, and piped, in that order [default]\n#transfer_method = smart\n\n# if False, sftp will not use batch mode to transfer files. This may cause some\n# types of file transfer failures impossible to catch however, and should\n# only be disabled if your sftp version has problems with batch mode\n#sftp_batch_mode = False\n\n# The -tt argument is passed to ssh when pipelining is not enabled because sudo \n# requires a tty by default. \n#usetty = True\n\n# Number of times to retry an SSH connection to a host, in case of UNREACHABLE.\n# For each retry attempt, there is an exponential backoff,\n# so after the first attempt there is 1s wait, then 2s, 4s etc. up to 30s (max).\n#retries = 3\n\n[persistent_connection]\n\n# Configures the persistent connection timeout value in seconds.  This value is\n# how long the persistent connection will remain idle before it is destroyed.\n# If the connection doesn't receive a request before the timeout value\n# expires, the connection is shutdown. The default value is 30 seconds.\n#connect_timeout = 30\n\n# The command timeout value defines the amount of time to wait for a command\n# or RPC call before timing out. The value for the command timeout must\n# be less than the value of the persistent connection idle timeout (connect_timeout)\n# The default value is 30 second.\n#command_timeout = 30\n\n[accelerate]\n#accelerate_port = 5099\n#accelerate_timeout = 30\n#accelerate_connect_timeout = 5.0\n\n# The daemon timeout is measured in minutes. This time is measured\n# from the last activity to the accelerate daemon.\n#accelerate_daemon_timeout = 30\n\n# If set to yes, accelerate_multi_key will allow multiple\n# private keys to be uploaded to it, though each user must\n# have access to the system via SSH to add a new key. The default\n# is \"no\".\n#accelerate_multi_key = yes\n\n[selinux]\n# file systems that require special treatment when dealing with security context\n# the default behaviour that copies the existing context or uses the user default\n# needs to be changed to use the file system dependent context.\n#special_context_filesystems=nfs,vboxsf,fuse,ramfs,9p,vfat\n\n# Set this to yes to allow libvirt_lxc connections to work without SELinux.\n#libvirt_lxc_noseclabel = yes\n\n[colors]\n#highlight = white\n#verbose = blue\n#warn = bright purple\n#error = red\n#debug = dark gray\n#deprecate = purple\n#skip = cyan\n#unreachable = red\n#ok = green\n#changed = yellow\n#diff_add = green\n#diff_remove = red\n#diff_lines = cyan\n\n\n[diff]\n# Always print diff when running ( same as always running with -D\/--diff )\n# always = no\n\n# Set how many context lines to show in diff\n# context = 3\n<\/code><\/pre>\n\n\n\n<p>Based on the default config, we will create our own configuration file;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>vim $HOME\/ansible\/ansible.cfg<\/code><\/pre>\n\n\n\n<p>In the configuration file, we will define some basic values;<\/p>\n\n\n\n<p><em>(note the value of <\/em>private_key_file<em> option. If using default SSH key path, then ignore it, otherwise, set correct path).<\/em><\/p>\n\n\n\n<pre class=\"scroll-box\"><code>[defaults]\ninventory       = \/home\/kifarunix\/ansible\/inventory\/hosts\nroles_path      = \/home\/kifarunix\/ansible\/roles\nprivate_key_file        = \/home\/kifarunix\/.ssh\/id_ansible_rsa\ninterpreter_python      = \/usr\/bin\/python3\n<\/code><\/pre>\n\n\n\n<p>Whenever you use a custom Ansible configuration directory, you can specify it using <code><strong>ANSIBLE_CONFIG<\/strong><\/code> environment variable or <strong><code>-c<\/code><\/strong> command-line option;<\/p>\n\n\n\n<p>Let&#8217;s set the environment variable to make it easy;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>echo \"export ANSIBLE_CONFIG=$HOME\/ansible\/ansible.cfg\" &gt;&gt; $HOME\/.bashrc<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>source $HOME\/.bashrc<\/code><\/pre>\n\n\n\n<p>With this, you wont have to specify the path to Ansible configuration or path to any path that is defined on the custom configuration.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-ansible-inventory-file\">Create Ansible Inventory File<\/h3>\n\n\n\n<p>The inventory file contains a list of hosts and groups of hosts that are managed by Ansible. By default, inventory file is set to <strong><code>\/etc\/ansible\/hosts<\/code><\/strong>. You can specify a different inventory file using the -i option or using the <strong><code>inventory<\/code><\/strong> option in the configuration file.<\/p>\n\n\n\n<p>We set out inventory file to <code><strong>\/home\/kifarunix\/ansible\/inventory\/hosts<\/strong><\/code> in the configuration above;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir ~\/ansible\/inventory<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>vim \/home\/kifarunix\/ansible\/inventory\/hosts<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>[elk-stack]\n192.168.122.60\n192.168.122.123\n192.168.122.152\n\n[nfs-server]\n192.168.122.47\n<\/code><\/pre>\n\n\n\n<p>Save and exit the file.<\/p>\n\n\n\n<p>There are different formats of the hosts file configuration. That is just an example.<\/p>\n\n\n\n<p>See below a sample hosts file!<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/etc\/ansible\/hosts<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code># This is the default ansible 'hosts' file.\n#\n# It should live in \/etc\/ansible\/hosts\n#\n#   - Comments begin with the '#' character\n#   - Blank lines are ignored\n#   - Groups of hosts are delimited by [header] elements\n#   - You can enter hostnames or ip addresses\n#   - A hostname\/ip can be a member of multiple groups\n\n# Ex 1: Ungrouped hosts, specify before any group headers.\n\n#green.example.com\n#blue.example.com\n#192.168.100.1\n#192.168.100.10\n\n# Ex 2: A collection of hosts belonging to the 'webservers' group\n\n#[webservers]\n#alpha.example.org\n#beta.example.org\n#192.168.1.100\n#192.168.1.110\n\n# If you have multiple hosts following a pattern you can specify\n# them like this:\n\n#www[001:006].example.com\n\n# Ex 3: A collection of database servers in the 'dbservers' group\n\n#[dbservers]\n#\n#db01.intranet.mydomain.net\n#db02.intranet.mydomain.net\n#10.25.1.56\n#10.25.1.57\n\n# Here's another example of host ranges, this time there are no\n# leading 0s:\n\n#db-[99:101]-node.example.com\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"run-ansible-passphrase-protected-ssh-key-without-prompting-for-passphrase\">Run Ansible Passphrase Protected SSH Key Without Prompting for Passphrase<\/h3>\n\n\n\n<p>If you setup passphrase protected SSH key, you will be prompted to enter the phrase for every single command to be ran against a managed node.<\/p>\n\n\n\n<p>As a work around for this, you can use <code>ssh-agent<\/code> to cache your passphrase, so that you only need to enter it once per session as follows;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>eval \"$(ssh-agent -s)\"<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>ssh-add \/home\/kifarunix\/.ssh\/id_ansible_rsa<\/code><\/pre>\n\n\n\n<p>You will be prompted to enter your passphrase.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"test-connection-to-ansible-managed-nodes\">Test Connection to Ansible Managed Nodes<\/h3>\n\n\n\n<p>Once you have setup SSH key authentication and an inventory of the managed hosts, run Ansible ping module to check hosts connection and aliveness. Replace the user accordingly.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible all -m ping -u kifarunix<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>192.168.122.47 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n192.168.122.123 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n192.168.122.60 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n192.168.122.152 | SUCCESS => {\n    \"changed\": false,\n    \"ping\": \"pong\"\n}\n<\/code><\/pre>\n\n\n\n<p>That confirms that you are ready to deploy ELK stack 8 Cluster on Docker.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"create-elk-stack-8-docker-containers-playbook-roles-and-tasks\">Create ELK Stack 8 Docker Containers Playbook Roles and Tasks<\/h3>\n\n\n\n<p>To begin with, here is how our directory setup looks like;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>tree ~\/ansible<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\/home\/kifarunix\/ansible\n\u251c\u2500\u2500 ansible.cfg\n\u251c\u2500\u2500 elkstack-configs\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 docker-compose-v1.yml\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 docker-compose-v2.yml\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 docker-compose.yml\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 logstash\n\u2502\u00a0\u00a0     \u2514\u2500\u2500 modsec.conf\n\u251c\u2500\u2500 inventory\n\u2502\u00a0\u00a0 \u251c\u2500\u2500 group_vars\n\u2502\u00a0\u00a0 \u2502\u00a0\u00a0 \u2514\u2500\u2500 all.yml\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 hosts\n\u251c\u2500\u2500 main.yml\n\u2514\u2500\u2500 roles\n    \u251c\u2500\u2500 clean\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 tasks\n    \u2502\u00a0\u00a0     \u2514\u2500\u2500 main.yml\n    \u251c\u2500\u2500 configure-nfs-share\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 tasks\n    \u2502\u00a0\u00a0     \u2514\u2500\u2500 main.yml\n    \u251c\u2500\u2500 copy-docker-compose\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 tasks\n    \u2502\u00a0\u00a0     \u2514\u2500\u2500 main.yml\n    \u251c\u2500\u2500 deploy-elk-cluster\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 tasks\n    \u2502\u00a0\u00a0     \u2514\u2500\u2500 main.yml\n    \u251c\u2500\u2500 extra-configs\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 tasks\n    \u2502\u00a0\u00a0     \u2514\u2500\u2500 main.yml\n    \u251c\u2500\u2500 install-docker\n    \u2502\u00a0\u00a0 \u2514\u2500\u2500 tasks\n    \u2502\u00a0\u00a0     \u2514\u2500\u2500 main.yml\n    \u2514\u2500\u2500 remove_initial_master_nodes_setting\n        \u2514\u2500\u2500 tasks\n            \u2514\u2500\u2500 main.yml\n\n20 directories, 15 files\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"ansible-playbook-variables\">Ansible Playbook Variables<\/h4>\n\n\n\n<p>We have defined various Ansible playbook environment variables. Those that apply to all hosts under the inventory\/group_vars and those that are specific to each role under respective role directory.<\/p>\n\n\n\n<p>Sample variables use for all hosts;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat inventory\/group_vars\/all.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>target_user: kifarunix\ndocker_compose_ver: 2.24.0\nvm_max_map_count: 262144\ndomain_name: kifarunix-demo.com\nelkstack_base_path: \"\/home\/{{ target_user }}\/elkstack_docker\"\nsrc_elkstack_configs: \/home\/kifarunix\/ansible\/elkstack-configs\nelk_nfs_export: \/mnt\/elkstack\nnfs_clients:\n  - \"192.168.122.152(rw,sync,no_root_squash,no_subtree_check)\"\n  - \"192.168.122.123(rw,sync,no_root_squash,no_subtree_check)\"\n  - \"192.168.122.60(rw,sync,no_root_squash,no_subtree_check)\"\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"tasks-to-install-docker-and-docker-compose-on-the-nodes\">Tasks to Install Docker and Docker Compose on the Nodes<\/h4>\n\n\n\n<p>Under the directory <strong><code>roles\/docker\/tasks<\/code><\/strong>, we define how to install Docker and Docker compose on cluster nodes. We will also define how to adjust<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/install-docker\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Install Docker Engine\n  block:\n    - name: Install Packages Required to install Various Repos and GPG Keys:Ubuntu\/Debian\n      apt:\n        name:\n          - apt-transport-https\n          - ca-certificates\n          - curl\n          - gnupg-agent\n          - software-properties-common\n        state: present\n        update_cache: yes\n      when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname in groups['elk-stack']\n\n    - name: Install Docker Engine Repo GPG Key on Ubuntu\/Debian\n      shell: \"curl -fsSL https:\/\/download.docker.com\/linux\/{{ ansible_distribution | lower }}\/gpg | gpg --dearmor > \/etc\/apt\/trusted.gpg.d\/docker.gpg\"\n      when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname in groups['elk-stack']\n\n    - name: Install Docker Engine Repository on Ubuntu\/Debian\n      apt_repository:\n        repo: \"deb [arch=amd64] https:\/\/download.docker.com\/linux\/{{ ansible_distribution | lower }} {{ ansible_distribution_release }} stable\"\n        state: present\n        update_cache: yes\n      when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname in groups['elk-stack']\n\n    - name: Install Docker Repos on CentOS\/Rocky\/RHEL Derivatives\n      command: \"yum-config-manager --add-repo https:\/\/download.docker.com\/linux\/centos\/docker-ce.repo\"\n      when: ansible_distribution in ['CentOS', 'Rocky'] or ansible_os_family == \"RedHat\" and inventory_hostname in groups['elk-stack']\n\n    - name: Install Docker CE on Ubuntu\/Debian\n      apt:\n        name:\n          - docker-ce\n        state: present\n      when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname in groups['elk-stack']\n\n    - name: Install Docker CE on CentOS\/RHEL\/Rocky\n      yum:\n        name:\n          - docker-ce\n        state: present\n      when: ansible_distribution in ['CentOS', 'Rocky'] or ansible_os_family == \"RedHat\" and inventory_hostname in groups['elk-stack']\n\n    - name: Start and Enable Docker Engine\n      systemd:\n        name: docker\n        state: started\n        enabled: yes\n      when: inventory_hostname in groups['elk-stack']\n\n    - name: Add User to Docker group\n      user:\n        name: \"{{ target_user }}\"\n        groups: docker\n        append: yes\n      when: inventory_hostname in groups['elk-stack']\n\n    - name: Install Docker Compose\n      shell: 'curl -L \"https:\/\/github.com\/docker\/compose\/releases\/download\/v{{ docker_compose_ver }}\/docker-compose-{{ ansible_system | lower }}-{{ ansible_architecture }}\" -o \/usr\/local\/bin\/docker-compose;chmod +x \/usr\/local\/bin\/docker-compose'\n      when: inventory_hostname in groups['elk-stack']\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"important-elasticsearch-settings\">Important Elasticsearch Settings<\/h4>\n\n\n\n<p>There are multiple important settings required for Elasticsearch such as disabling swappiness, setting max number of open files and max user processes, JVM settings, max virtual memory map count&#8230;<\/p>\n\n\n\n<p>Here, we will only create a role for setting up maximum virtual memory map count on the Elastic cluster nodes.<\/p>\n\n\n\n<p>Similarly, let&#8217;s ensure the nodes can access the other nodes via domain addresses. Hence, we will define a static hosts file entry for our ELK stack nodes.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/extra-configs\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Update Virtual Memory Settings on All Cluster Nodes\n  sysctl:\n    name: vm.max_map_count\n    value: '{{ vm_max_map_count }}'\n    sysctl_set: true\n  when: inventory_hostname in groups['elk-stack']\n- name: Update hosts file\n  lineinfile:\n    dest: \/etc\/hosts\n    regexp: \"^{{ item.split()[0] }}\"\n    line: \"{{ item }}\"\n  with_items:\n    - \"192.168.122.60 es01.kifarunix-demo.com es01\"\n    - \"192.168.122.123 es02.kifarunix-demo.com es02\"      \n    - \"192.168.122.152 es03.kifarunix-demo.com es03\"\n  when: inventory_hostname in groups['elk-stack']\n<\/code><\/pre>\n\n\n\n<p>The variable used <strong>vm_max_map_count<\/strong>, is defined in the global variables file, <strong>inventory\/group_vars\/all.yml<\/strong>. See above.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-nfs-share-on-nfs-server\">Create NFS Share on NFS Server<\/h4>\n\n\n\n<p>We will be using shared volumes to store some ELK stack configuration files. Some configuration files will need to be accessed by various containers running on separate ELK stack nodes. As such, we will create an NFS share called <strong>\/mnt\/elkstack<\/strong> as we did in our <a href=\"https:\/\/kifarunix.com\/deploy-elk-stack-8-cluster-on-docker-containers\/#create-a-network-share-for-docker-volumes\" target=\"_blank\" rel=\"noreferrer noopener\">previous guide<\/a>.<\/p>\n\n\n\n<p>Kindly note that <a href=\"https:\/\/kifarunix.com\/?s=install+nfs+server\" target=\"_blank\" rel=\"noreferrer noopener\">NFS server is already installed<\/a> and running.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/configure-nfs-share\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Check if NFS Share is already configured\n  shell: grep -q \"{{ elk_nfs_export }} {{ nfs_clients | join(' ') }}\" \/etc\/exports\n  register: grep_result\n  failed_when: false\n  when: inventory_hostname in groups['nfs-server']\n\n- name: Configure NFS Share\n  lineinfile:\n    path: \/etc\/exports\n    line: \"{{ elk_nfs_export }} {{ nfs_clients | join(' ') }}\"\n    create: yes\n  when: inventory_hostname in groups['nfs-server'] and grep_result.rc != 0\n\n- name: Create ELK Stack Docker Volumes Directories\n  file:\n    path: \"{{ item }}\"\n    state: directory\n    recurse: yes\n  with_items:\n    - \"{{ elk_nfs_export }}\/certs\"\n    - \"{{ elk_nfs_export }}\/data\"\n    - \"{{ elk_nfs_export }}\/configs\/logstash\"\n    - \"{{ elk_nfs_export }}\/data\/elasticsearch\/01\"\n    - \"{{ elk_nfs_export }}\/data\/elasticsearch\/02\"\n    - \"{{ elk_nfs_export }}\/data\/elasticsearch\/03\"\n    - \"{{ elk_nfs_export }}\/data\/kibana\/01\"\n    - \"{{ elk_nfs_export }}\/data\/kibana\/02\"\n    - \"{{ elk_nfs_export }}\/data\/kibana\/03\"\n    - \"{{ elk_nfs_export }}\/data\/logstash\/01\"\n    - \"{{ elk_nfs_export }}\/data\/logstash\/02\"\n    - \"{{ elk_nfs_export }}\/data\/logstash\/03\"\n  when: inventory_hostname in groups['nfs-server']\n\n- name: Reload NFS Exports\n  command: exportfs -arvf\n  when: inventory_hostname in groups['nfs-server']\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"elk-stack-configs\">ELK Stack Configs<\/h4>\n\n\n\n<p>Under the <strong>elkstack-configs<\/strong> directory, we have configuration files related to Elasticsearch, Kibana or Logstash and (File)beats.<\/p>\n\n\n\n<p>To begin with, let&#8217;s create a <a href=\"https:\/\/kifarunix.com\/deploy-a-single-node-elk-stack-cluster-on-docker-containers\/#define-logstash-data-processing-pipeline\" target=\"_blank\" rel=\"noreferrer noopener\">Logstash pipeline filters configuration<\/a> as we used in our previous guide.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat elkstack-configs\/logstash\/modsec.conf<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>input {\n  beats {\n    port => 5044\n  }\n}\nfilter {\n    # Extract event time, log severity level, source of attack (client), and the alert message.\n    grok {\n      match => { \"message\" => \"(?<event_time>%{MONTH}\\s%{MONTHDAY}\\s%{TIME}\\s%{YEAR})\\] \\[\\:%{LOGLEVEL:log_level}.*client\\s%{IPORHOST:src_ip}:\\d+]\\s(?<alert_message>.*)\" }\n    }\n    # Extract Rules File from Alert Message\n    grok {\n      match => { \"alert_message\" => \"(?<rulesfile>\\[file \\\"(\/.+.conf)\\\"\\])\" }\n    }\t\n    grok {\n      match => { \"rulesfile\" => \"(?<rules_file>\/.+.conf)\" }\n    }\t\n    # Extract Attack Type from Rules File\n    grok {\n      match => { \"rulesfile\" => \"(?<attack_type>[A-Z]+-[A-Z][^.]+)\" }\n    }\t\n    # Extract Rule ID from Alert Message\n    grok {\n      match => { \"alert_message\" => \"(?<ruleid>\\[id \\\"(\\d+)\\\"\\])\" }\n    }\t\n    grok {\n      match => { \"ruleid\" => \"(?<rule_id>\\d+)\" }\n    }\n    # Extract Attack Message (msg) from Alert Message \t\n    grok {\n      match => { \"alert_message\" => \"(?<msg>\\[msg \\S(.*?)\\\"\\])\" }\n    }\t\n    grok {\n      match => { \"msg\" => \"(?<alert_msg>\\\"(.*?)\\\")\" }\n    }\n    # Extract the User\/Scanner Agent from Alert Message\t\n    grok {\n      match => { \"alert_message\" => \"(?<scanner>User-Agent' \\SValue: `(.*?)')\" }\n    }\t\n    grok {\n      match => { \"scanner\" => \"(?<user_agent>:(.*?)\\')\" }\n    }\t\n    grok {\n      match => { \"alert_message\" => \"(?<agent>User-Agent: (.*?)\\')\" }\n    }\t\n    grok {\n      match => { \"agent\" => \"(?<user_agent>: (.*?)\\')\" }\n    }\t\n    # Extract the Target Host\n    grok {\n      match => { \"alert_message\" => \"(hostname \\\"%{IPORHOST:dst_host})\" }\n    }\t\n    # Extract the Request URI\n    grok {\n      match => { \"alert_message\" => \"(uri \\\"%{URIPATH:request_uri})\" }\n    }\n    grok {\n      match => { \"alert_message\" => \"(?<ref>referer: (.*))\" }\n    }\t\n    grok {\n      match => { \"ref\" => \"(?<referer> (.*))\" }\n    }\n    mutate {\n      # Remove unnecessary characters from the fields.\n      gsub => [\n        \"alert_msg\", \"[\\\"]\", \"\",\n        \"user_agent\", \"[:\\\"'`]\", \"\",\n        \"user_agent\", \"^\\s*\", \"\",\n        \"referer\", \"^\\s*\", \"\"\n      ]\n      # Remove the Unnecessary fields so we can only remain with\n      # General message, rules_file, attack_type, rule_id, alert_msg, user_agent, hostname (being attacked), Request URI and Referer. \n      remove_field => [ \"alert_message\", \"rulesfile\", \"ruleid\", \"msg\", \"scanner\", \"agent\", \"ref\" ]\n    }\t\n}\noutput {\n   elasticsearch {\n     hosts => [\"https:\/\/${ES_NAME}:9200\"]\n     user => \"${ELASTICSEARCH_USERNAME}\"\n     password => \"${ELASTICSEARCH_PASSWORD}\"\n     ssl => true\n     cacert => \"config\/certs\/ca\/ca.crt\"\n   }\n}\n<\/code><\/pre>\n\n\n\n<p>Basically, we will have three ELK Stack nodes in the cluster and each running a single instance of Elasticsearch, Logstash, Kibana containers.<\/p>\n\n\n\n<p>The variables used in Elastic configs will be defined in the Docker compose environment variables file.<\/p>\n\n\n\n<p>We also have two compose files:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The compose file required to setup and start ELK stack container on the first node. Will generate the required SSL certs and mount on the NFS share volumes for use by other ELK stack on the other two nodes.<\/li>\n\n\n\n<li>The second compose file to start the two other ELK stack containers and join them to the first ELK stack to make a cluster.<\/li>\n<\/ul>\n\n\n\n<p>Create Initial ELK Stack 8 Cluster Node Docker compose file;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat elkstack-configs\/docker-compose-v1.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>version: '3.8'\n\nservices:\n  es_setup:\n    image: docker.elastic.co\/elasticsearch\/elasticsearch:${STACK_VERSION}\n    volumes:\n      - certs:\/usr\/share\/elasticsearch\/${CERTS_PATH}\n    user: \"0\"\n    command: >\n      bash -c '\n        echo \"Creating ES certs directory...\"\n        [[ -d ${CERTS_PATH} ]] || mkdir ${CERTS_PATH}\n        # Check if CA certificate exists\n        if [ ! -f ${CERTS_PATH}\/ca\/ca.crt ]; then\n          echo \"Generating Wildcard SSL certs for ES (in PEM format)...\"\n          bin\/elasticsearch-certutil ca --pem --days 3650 --out ${CERTS_PATH}\/elkstack-ca.zip\n          unzip -d ${CERTS_PATH} ${CERTS_PATH}\/elkstack-ca.zip\n          bin\/elasticsearch-certutil cert \\\n            --name elkstack-certs \\\n            --ca-cert ${CERTS_PATH}\/ca\/ca.crt \\\n            --ca-key ${CERTS_PATH}\/ca\/ca.key \\\n            --pem \\\n            --dns \"*.${DOMAIN_SUFFIX},localhost,${NODE01_NAME},${NODE02_NAME},${NODE03_NAME}\" \\\n            --ip ${NODE01_IP} \\\n            --ip ${NODE02_IP} \\\n            --ip ${NODE03_IP} \\\n            --days ${DAYS} \\\n            --out ${CERTS_PATH}\/elkstack-certs.zip\n          unzip -d ${CERTS_PATH} ${CERTS_PATH}\/elkstack-certs.zip\n        else\n          echo \"CA certificate already exists. Skipping Certificates generation.\"\n        fi\n        # Check if Elasticsearch is ready\n        until curl -s --cacert ${CERTS_PATH}\/ca\/ca.crt -u \"elastic:${ELASTIC_PASSWORD}\" https:\/\/${NODE01_NAME}:9200 | grep -q \"${CLUSTER_NAME}\"; do sleep 10; done\n        # Set kibana_system password\n        if curl -sk -XGET --cacert ${CERTS_PATH}\/ca\/ca.crt \"https:\/\/${NODE01_NAME}:9200\" -u \"kibana_system:${KIBANA_PASSWORD}\" | grep -q \"${CLUSTER_NAME}\"; then\n          echo \"Password for kibana_system is working. Proceeding with Elasticsearch setup for kibana_system.\"\n        else\n          echo \"Failed to authenticate with kibana_system password. Trying to set the password for kibana_system.\"\n          until curl -s -XPOST --cacert ${CERTS_PATH}\/ca\/ca.crt -u \"elastic:${ELASTIC_PASSWORD}\" -H \"Content-Type: application\/json\" https:\/\/${NODE01_NAME}:9200\/_security\/user\/kibana_system\/_password -d \"{\\\"password\\\":\\\"${KIBANA_PASSWORD}\\\"}\" | grep -q \"^{}\"; do sleep 10; done\n        fi\n        echo \"Setup is done!\"\n      '\n    networks:\n      - elastic\n    healthcheck:\n      test: [\"CMD-SHELL\", \"[ -f ${CERTS_PATH}\/elkstack-certs\/elkstack-certs.crt ]\"]\n      interval: 1s\n      timeout: 5s\n      retries: 120\n\n  elasticsearch:\n    depends_on:\n      es_setup:\n        condition: service_healthy\n    container_name: ${NODE01_NAME}\n    image: docker.elastic.co\/elasticsearch\/elasticsearch:${STACK_VERSION}\n    environment:\n      - node.name=${NODE01_NAME}\n      - network.publish_host=${NODE01_IP} \n      - cluster.name=${CLUSTER_NAME}\n      - bootstrap.memory_lock=true\n      - \"ES_JAVA_OPTS=-Xms1g -Xmx1g\"\n      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}\n      - xpack.security.enabled=true\n      - xpack.security.http.ssl.enabled=true\n      - xpack.security.transport.ssl.enabled=true\n      - xpack.security.enrollment.enabled=false\n      - xpack.security.autoconfiguration.enabled=false\n      - xpack.security.http.ssl.key=certs\/elkstack-certs\/elkstack-certs.key\n      - xpack.security.http.ssl.certificate=certs\/elkstack-certs\/elkstack-certs.crt\n      - xpack.security.http.ssl.certificate_authorities=certs\/ca\/ca.crt\n      - xpack.security.transport.ssl.key=certs\/elkstack-certs\/elkstack-certs.key\n      - xpack.security.transport.ssl.certificate=certs\/elkstack-certs\/elkstack-certs.crt\n      - xpack.security.transport.ssl.certificate_authorities=certs\/ca\/ca.crt\n      - cluster.initial_master_nodes=${NODE01_NAME},${NODE02_NAME},${NODE03_NAME}\n      - discovery.seed_hosts=${NODE01_IP},${NODE02_IP},${NODE03_IP}\n      - KIBANA_USERNAME=${KIBANA_USERNAME}\n      - KIBANA_PASSWORD=${KIBANA_PASSWORD}\n    ulimits:\n      memlock:\n        soft: -1\n        hard: -1\n    volumes:\n      - elasticsearch_data:\/usr\/share\/elasticsearch\/data\n      - certs:\/usr\/share\/elasticsearch\/${CERTS_PATH}\n      - \/etc\/hosts:\/etc\/hosts\n    ports:\n      - ${ES_PORT}:9200\n      - ${ES_TS_PORT}:9300\n    networks:\n      - elastic\n    healthcheck:\n      test: [\"CMD-SHELL\", \"curl --fail -k -s -u elastic:${ELASTIC_PASSWORD} --cacert ${CERTS_PATH}\/ca\/ca.crt https:\/\/${NODE01_NAME}:9200\"]\n      interval: 30s\n      timeout: 10s\n      retries: 5\n    restart: unless-stopped\n\n  kibana:\n    image: docker.elastic.co\/kibana\/kibana:${STACK_VERSION}\n    container_name: kibana\n    environment:\n      - SERVER_NAME=${KIBANA_SERVER_HOST}\n      - ELASTICSEARCH_HOSTS=https:\/\/${NODE01_NAME}:9200\n      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=${CERTS_PATH}\/ca\/ca.crt\n      - ELASTICSEARCH_USERNAME=${KIBANA_USERNAME}\n      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}\n      - XPACK_REPORTING_ROLES_ENABLED=false\n      - XPACK_REPORTING_KIBANASERVER_HOSTNAME=localhost\n      - XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=${SAVEDOBJECTS_ENCRYPTIONKEY}\n      - XPACK_SECURITY_ENCRYPTIONKEY=${REPORTING_ENCRYPTIONKEY}\n      - XPACK_REPORTING_ENCRYPTIONKEY=${SECURITY_ENCRYPTIONKEY}\n    volumes:\n      - kibana_data:\/usr\/share\/kibana\/data\n      - certs:\/usr\/share\/kibana\/${CERTS_PATH}\n      - \/etc\/hosts:\/etc\/hosts\n    ports:\n      - ${KIBANA_PORT}:5601\n    networks:\n      - elastic\n    depends_on:\n      elasticsearch:\n        condition: service_healthy\n    restart: unless-stopped\n  \n  logstash:\n    image: docker.elastic.co\/logstash\/logstash:${STACK_VERSION}\n    container_name: logstash\n    environment:\n      - XPACK_MONITORING_ENABLED=false\n      - ELASTICSEARCH_USERNAME=${ES_USER}\n      - ELASTICSEARCH_PASSWORD=${ELASTIC_PASSWORD}\n      - NODE_NAME=${NODE01_NAME}\n      - CERTS_PATH=${CERTS_PATH}\n    ports:\n      - ${BEATS_INPUT_PORT}:5044\n    volumes:\n      - logstash_filters:\/usr\/share\/logstash\/pipeline\/:ro\n      - certs:\/usr\/share\/logstash\/${CERTS_PATH}\n      - logstash_data:\/usr\/share\/logstash\/data\n      - \/etc\/hosts:\/etc\/hosts\n    networks:\n      - elastic\n    depends_on:\n      elasticsearch:\n        condition: service_healthy\n    restart: unless-stopped\nvolumes:\n  certs:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_CERTS}\"\n  elasticsearch_data:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_DATA}\/elasticsearch\/01\"\n  kibana_data:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_DATA}\/kibana\/01\"\n  logstash_filters:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_CONFIGS}\/logstash\"\n  logstash_data:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_DATA}\/logstash\/01\"\n\nnetworks:\n  elastic:\n<\/code><\/pre>\n\n\n\n<p>The second compose file;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat elkstack-configs\/docker-compose-v2.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>services:\n  elasticsearch:\n    container_name: ${NODENN_NAME}\n    image: docker.elastic.co\/elasticsearch\/elasticsearch:${STACK_VERSION}\n    command: >\n      bash -c '\n      until [ -f \"${CERTS_PATH}\/elkstack-certs\/elkstack-certs.crt\" ]; do\n          sleep 10;\n        done;\n        exec \/usr\/local\/bin\/docker-entrypoint.sh\n      '\n    environment:\n      - node.name=${NODENN_NAME}\n      - network.publish_host=${NODENN_IP}\n      - cluster.name=${CLUSTER_NAME}\n      - bootstrap.memory_lock=true\n      - cluster.initial_master_nodes=${NODE01_NAME},${NODE02_NAME},${NODE03_NAME}\n      - discovery.seed_hosts=${NODE01_IP},${NODE02_IP},${NODE03_IP}\n      - \"ES_JAVA_OPTS=-Xms1g -Xmx1g\"\n      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}\n      - xpack.security.enabled=true\n      - xpack.security.http.ssl.enabled=true\n      - xpack.security.enrollment.enabled=false\n      - xpack.security.autoconfiguration.enabled=false        \n      - xpack.security.transport.ssl.enabled=true\n      - xpack.security.http.ssl.key=certs\/elkstack-certs\/elkstack-certs.key\n      - xpack.security.http.ssl.certificate=certs\/elkstack-certs\/elkstack-certs.crt\n      - xpack.security.http.ssl.certificate_authorities=certs\/ca\/ca.crt\n      - xpack.security.transport.ssl.key=certs\/elkstack-certs\/elkstack-certs.key\n      - xpack.security.transport.ssl.certificate=certs\/elkstack-certs\/elkstack-certs.crt\n      - xpack.security.transport.ssl.certificate_authorities=certs\/ca\/ca.crt\n      - KIBANA_USERNAME=${KIBANA_USERNAME}\n      - KIBANA_PASSWORD=${KIBANA_PASSWORD}\n    ulimits:\n      memlock:\n        soft: -1\n        hard: -1\n    volumes:\n      - elasticsearch_data:\/usr\/share\/elasticsearch\/data\n      - certs:\/usr\/share\/elasticsearch\/${CERTS_PATH}\n      - \/etc\/hosts:\/etc\/hosts\n    ports:\n      - ${ES_PORT}:9200\n      - ${ES_TS_PORT}:9300\n    networks:\n      - elastic\n    healthcheck:\n      test: [\"CMD-SHELL\", \"curl --fail -k -s -u elastic:${ELASTIC_PASSWORD} --cacert ${CERTS_PATH}\/ca\/ca.crt https:\/\/${NODENN_NAME}:9200\"]\n      interval: 30s\n      timeout: 10s\n      retries: 5\n    restart: unless-stopped\n\n  kibana:\n    image: docker.elastic.co\/kibana\/kibana:${STACK_VERSION}\n    container_name: kibana\n    environment:\n      - SERVER_NAME=${KIBANA_SERVER_HOST}\n      - ELASTICSEARCH_HOSTS=https:\/\/${NODENN_NAME}:9200\n      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=${CERTS_PATH}\/ca\/ca.crt\n      - ELASTICSEARCH_USERNAME=${KIBANA_USERNAME}\n      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}\n      - XPACK_REPORTING_ROLES_ENABLED=false\n      - XPACK_REPORTING_KIBANASERVER_HOSTNAME=localhost\n      - XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=${SAVEDOBJECTS_ENCRYPTIONKEY}\n      - XPACK_SECURITY_ENCRYPTIONKEY=${REPORTING_ENCRYPTIONKEY}\n      - XPACK_REPORTING_ENCRYPTIONKEY=${SECURITY_ENCRYPTIONKEY}\n    volumes:\n      - kibana_data:\/usr\/share\/kibana\/data\n      - certs:\/usr\/share\/kibana\/${CERTS_PATH}\n      - \/etc\/hosts:\/etc\/hosts\n    ports:\n      - ${KIBANA_PORT}:5601\n    networks:\n      - elastic\n    depends_on:\n      elasticsearch:\n        condition: service_healthy\n    restart: unless-stopped\n  \n  logstash:\n    image: docker.elastic.co\/logstash\/logstash:${STACK_VERSION}\n    container_name: logstash\n    environment:\n      - XPACK_MONITORING_ENABLED=false\n      - ELASTICSEARCH_USERNAME=${ES_USER}\n      - ELASTICSEARCH_PASSWORD=${ELASTIC_PASSWORD}\n      - NODE_NAME=${NODENN_NAME}\n      - CERTS_PATH=${CERTS_PATH}\n    ports:\n      - ${BEATS_INPUT_PORT}:5044\n    volumes:\n      - certs:\/usr\/share\/logstash\/${CERTS_PATH}\n      - logstash_filters:\/usr\/share\/logstash\/pipeline\/:ro\n      - logstash_data:\/usr\/share\/logstash\/data      \n      - \/etc\/hosts:\/etc\/hosts\n    networks:\n      - elastic\n    depends_on:\n      elasticsearch:\n        condition: service_healthy\n    restart: unless-stopped\n\nvolumes:\n  certs:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_CERTS}\"\n  elasticsearch_data:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_DATA}\/elasticsearch\/NN\"\n  kibana_data:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_DATA}\/kibana\/NN\"\n  logstash_filters:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_CONFIGS}\/logstash\"\n  logstash_data:\n    driver: local\n    driver_opts:\n      type: nfs\n      o: \"addr=${NFS_SVR_IP},nfsvers=4,rw\"\n      device: \":${NFS_ELK_DATA}\/logstash\/NN\"\n\nnetworks:\n  elastic:\n<\/code><\/pre>\n\n\n\n<p>The variables are defined in the Docker compose environment variable;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat elkstack-configs\/.env<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code># Version of Elastic products\nSTACK_VERSION=8.12.0\n\n# Set the cluster name\nCLUSTER_NAME=elk-docker-cluster\n\n# Set Elasticsearch Node Name\nNODE01_NAME=es01\nNODE02_NAME=es02\nNODE03_NAME=es03\n\n# Docker Host IP to advertise to cluster nodes\nNODE01_IP=192.168.122.60\nNODE02_IP=192.168.122.123\nNODE03_IP=192.168.122.152\n\n# Elasticsearch super user\nES_USER=elastic\n\n# Password for the 'elastic' user (at least 6 characters). No special characters, ! or @ or $.\nELASTIC_PASSWORD=ChangeME\n\n# Elasticsearch container name\nES_NAME=elasticsearch\n\n# Port to expose Elasticsearch HTTP API to the host\nES_PORT=9200\n#ES_PORT=127.0.0.1:9200\nES_TS_PORT=9300\n\n# Port to expose Kibana to the host\nKIBANA_PORT=5601\nKIBANA_SERVER_HOST=0.0.0.0\n\n# Kibana Encryption. Requires atleast 32 characters. Can be generated using `openssl rand -hex 16\nSAVEDOBJECTS_ENCRYPTIONKEY=ca11560aec8410ff002d011c2a172608\nREPORTING_ENCRYPTIONKEY=288f06b3a14a7f36dd21563d50ec76d4\nSECURITY_ENCRYPTIONKEY=62c781d3a2b2eaee1d4cebcc6bf42b48\n\n# Kibana - Elasticsearch Authentication Credentials for user kibana_system\n# Password for the 'kibana_system' user (at least 6 characters). No special characters, ! or @ or $.\nKIBANA_USERNAME=kibana_system\nKIBANA_PASSWORD=ChangeME\n\n# Domain Suffix for ES Wildcard SSL certs\nDOMAIN_SUFFIX=kifarunix-demo.com\n\n# Generated Certs Validity Period\nDAYS=3650\n\n# SSL\/TLS Certs Directory\nCERTS_PATH=config\/certs\n\n# Logstash Input Port\nBEATS_INPUT_PORT=5044\n\n# NFS Server\nNFS_SVR_IP=192.168.122.47\nNFS_ELK_CERTS=\/mnt\/elkstack\/certs\nNFS_ELK_DATA=\/mnt\/elkstack\/data\nNFS_ELK_CONFIGS=\/mnt\/elkstack\/configs\n<\/code><\/pre>\n\n\n\n<p>Docker environment variables will be same across the entire cluster. We will however replace NODENN with respective node variable. For example, on second node, NODENN will change to NODE02 and NODE03 on the third node. We also mount the NFS share data path based on each node&#8217;s number, signified by NN, the container is running on and hence, on node02, NN=02 and on node03, NN=03.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"copy-elk-stack-configs-and-docker-compose-files-to-respective-nodes\">Copy ELK Stack Configs and Docker Compose Files to Respective Nodes<\/h4>\n\n\n\n<p>Now, we need to copy the Docker compose files, environment variables and configs to the respective nodes.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/copy-docker-compose\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Create directory for ELK Stack Docker Compose files\n  file:\n    path: \"{{ elkstack_base_path }}\"\n    state: directory\n  when: inventory_hostname in groups['elk-stack']\n\n- name: Create Docker-Compose file for ELK Node 01\n  copy:\n    src: \"{{ src_elkstack_configs }}\/docker-compose-v1.yml\"\n    dest: \"{{ elkstack_base_path }}\/docker-compose.yml\"\n  when: ansible_host == 'node01'\n  \n- name: Create Docker-Compose file for ELK Node 02\/03\n  copy:\n    src: \"{{ src_elkstack_configs }}\/docker-compose-v2.yml\"\n    dest: \"{{ elkstack_base_path }}\/docker-compose.yml\"\n  when: ansible_host in [\"node02\",\"node03\"]\n    \n- name: Copy Environment Variables\n  copy:\n    src: \"{{ src_elkstack_configs }}\/.env\"\n    dest: \"{{ elkstack_base_path }}\"\n  when: inventory_hostname in groups['elk-stack']\n\n- name: Update the NODENN variable accordingly\n  replace:\n    path: \"{{ elkstack_base_path }}\/docker-compose.yml\"\n    regexp: 'NN'\n    replace: \"{{ '02' if ansible_host == 'node02' else ('03' if ansible_host == 'node03') }}\"\n  when: \"'node02' in ansible_host or 'node03' in ansible_host\"\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"deploying-elk-stack-8-cluster-on-docker-using-ansible-1\">Deploying ELK Stack 8 Cluster on Docker using Ansible<\/h4>\n\n\n\n<p>Next, create a task to deploy ELK stack 8 cluster on Docker.<\/p>\n\n\n\n<p>We are using docker compose to build the cluster containers.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/deploy-elk-cluster\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Deploy ELK Stack on Docker using Docker Compose\n  command: docker compose up -d\n  args:\n    chdir: \"{{elkstack_base_path}}\/\"\n  when: inventory_hostname in groups['elk-stack']\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"remove-cluster-initial-master-nodes-setting\">Remove <code>cluster.initial_master_nodes<\/code>&nbsp;setting<\/h4>\n\n\n\n<p>Accordingly to Elastic;<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"has-small-font-size\"><br>After the cluster has formed, remove the&nbsp;<code>cluster.initial_master_nodes<\/code>&nbsp;setting from each node\u2019s configuration. It should not be set for master-ineligible nodes, master-eligible nodes joining an existing cluster, or nodes which are restarting.<\/p>\n\n\n\n<p class=\"has-small-font-size\">If you leave&nbsp;<code>cluster.initial_master_nodes<\/code>&nbsp;in place once the cluster has formed then there is a risk that a future misconfiguration may result in bootstrapping a new cluster alongside your existing cluster. It may not be possible to recover from this situation without losing data.<\/p>\n<\/blockquote>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/remove_initial_master_nodes_setting\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Check ELK Stack 8 Cluster Status\n  shell: \"docker exec -it es01 bash -c 'curl -sk -XGET https:\/\/es01:9200 -u \\\"${ES_USER}:${ELASTIC_PASSWORD}\\\"'\"\n  register: es_cluster_status\n  run_once: true\n  when: inventory_hostname in groups['elk-stack']\n\n- name: Remove initial_master_nodes Setting After Cluster Formed\n  lineinfile:\n    path: \"{{ elkstack_base_path }}\/docker-compose.yml\"\n    regexp: '^(\\s*- cluster\\.initial_master_nodes=.*)$'\n    state: absent\n  when: inventory_hostname in groups['elk-stack'] and es_cluster_status.rc == 0\n\n- name: Restart Docker Compose Services\n  command: docker compose restart elasticsearch\n  args:\n    chdir: \"{{ elkstack_base_path }}\"\n  when: inventory_hostname in groups['elk-stack'] and es_cluster_status.rc == 0\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"docker-clean-up-role\">Docker Clean Up Role<\/h4>\n\n\n\n<p>Similarly, we have created a role to tear down the ELK stack cluster!<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat roles\/clean\/tasks\/main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- name: Check if ELK Stack Compose directory exists\n  stat:\n    path: \"{{ elkstack_base_path }}\"\n  register: dir_exists\n  when: inventory_hostname in groups['elk-stack']\n\n- name: Check if ELK Stack Docker Compose yml\/yaml file exists\n  find:\n    path: \"{{ elkstack_base_path }}\"\n    patterns: \"*.yml,*.yaml\"\n  register: compose_files\n  when: \"inventory_hostname in groups['elk-stack'] and dir_exists.stat.exists\"\n\n- name: Tear down the stack!\n  command: docker compose down -v\n  args:\n    chdir: \"{{ elkstack_base_path }}\"\n  when: \"inventory_hostname in groups['elk-stack'] and dir_exists.stat.exists and compose_files.matched > 0\"\n\n- name: Remove Docker compose and env files\n  file:\n    path: \"{{ elkstack_base_path }}\"\n    state: absent\n  when: inventory_hostname in groups['elk-stack']\n\n- name: Check if Docker service is active\n  command: systemctl is-active docker\n  ignore_errors: true\n  register: docker_service_status\n  when: inventory_hostname in groups['elk-stack']\n\n- name: Check if Docker images exist\n  command: docker image ls -q\n  ignore_errors: true\n  register: docker_images\n  when: inventory_hostname not in groups['nfs-server'] and docker_service_status.rc == 0\n\n- name: Remove Docker Images\n  shell: docker image rm -f {{ docker_images.stdout_lines | default([]) }}\n  ignore_errors: true\n  when: inventory_hostname in groups['elk-stack'] and docker_images.stdout_lines | default([]) | length > 0\n\n- name: Delete ELK Stack Data\/Certs\/Configs on NFS Share\n  file:\n    path: \"{{ elk_nfs_export }}\"\n    state: absent\n  when: inventory_hostname in groups['nfs-server']\n\n- name: Remove ELK Stack NFS Export from the NFS Server\n  lineinfile:\n    path: \/etc\/exports\n    state: absent\n    regexp: \"^{{ elk_nfs_export}}.*\"\n  when: inventory_hostname in groups['nfs-server']\n\n- name: Remove user from Docker group\n  user:\n    name: \"{{ target_user }}\"\n    groups: docker\n    append: no\n  when: inventory_hostname not in groups['nfs-server']\n\n- name: Remove Docker Engine Repo GPG Key on Ubuntu\/Debian\n  file:\n    path: \/etc\/apt\/trusted.gpg.d\/docker.gpg\n    state: absent\n  when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname not in groups['nfs-server']\n\n- name: Remove Docker Engine Repository on Ubuntu\/Debian\n  apt_repository:\n    repo: \"deb [arch=amd64] https:\/\/download.docker.com\/linux\/{{ ansible_distribution | lower }} {{ ansible_distribution_release }} stable\"\n    state: absent\n    update_cache: yes\n  when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname not in groups['nfs-server']\n\n- name: Remove Docker Repos on CentOS\/Rocky\/RHEL Derivatives\n  command: \"yum-config-manager --disable docker-ce\"\n  when: ansible_distribution in ['CentOS', 'Rocky'] or ansible_os_family == \"RedHat\" and inventory_hostname not in groups['nfs-server']\n\n- name: Uninstall Docker CE on Ubuntu\/Debian\n  apt:\n    name:\n      - docker-ce\n    state: absent\n  when: ansible_distribution in ['Debian', 'Ubuntu'] and inventory_hostname not in groups['nfs-server']\n\n- name: Uninstall Docker CE on CentOS\/RHEL\/Rocky\n  yum:\n    name:\n      - docker-ce\n    state: absent\n  when: ansible_distribution in ['CentOS', 'Rocky'] or ansible_os_family == \"RedHat\" and inventory_hostname not in groups['nfs-server']\n\n- name: Remove Docker Compose\n  file:\n    path: \/usr\/local\/bin\/docker-compose\n    state: absent\n  when: inventory_hostname not in groups['nfs-server']\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-main-ansible-playbook\">Create Main Ansible Playbook<\/h4>\n\n\n\n<p>It is now time to deploy ELK stack 8 cluster on Docker using playbooks\/tasks created above.<\/p>\n\n\n\n<p>To ensure that everything runs as expected, we need to run all plays\/tasks in a specific order.<\/p>\n\n\n\n<p>For example, in our case here, our order of execution is <strong>install docker &gt; extra-configs (update some of the settings on the Docker host) &gt; configure nfs share &gt; copy compose and env to Docker hosts &gt; Deploy ELK cluster using Docker compose<\/strong>.<\/p>\n\n\n\n<p>Thus, our main Ansible playbook;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- hosts: all\n  gather_facts: True\n  remote_user: \"{{ target_user }}\"\n  become: yes\n  roles:\n    - install-docker\n    - extra-configs\n    - configure-nfs-share\n    - copy-docker-compose\n    - deploy-elk-cluster\n    - remove_initial_master_nodes_setting\n#    - clean\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"perform-elk-stack-deployment-via-ansible-dry-run\">Perform ELK Stack Deployment via Ansible Dry Run<\/h3>\n\n\n\n<p>You can do a dry-run just to see how the plays will be executed without actually making changes on the nodes. This will also help you identify any issue and fix before running the deployment.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible-playbook main.yml --ask-become-pass -C<\/code><\/pre>\n\n\n\n<p>Where:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>ansible-playbook<\/code>: This is the Ansible command for running playbooks.<\/li>\n\n\n\n<li><code>main.yml<\/code>: This is the name of our main Ansible playbook file.<\/li>\n\n\n\n<li><code>--ask-become-pass<\/code>: This option tells Ansible to prompt you for the password that is required to become a privileged user (typically, to escalate to <code>sudo<\/code> or <code>root<\/code>). This is useful when running tasks that require elevated privileges on the target hosts. Without this option, Ansible might fail to execute tasks that require elevated privileges.<\/li>\n\n\n\n<li><code>-C<\/code> or <code>--check<\/code>: This option is used for a &#8220;dry-run&#8221; or check mode. When you use this option, Ansible will not make any changes to the target hosts. Instead, it will simulate the execution of tasks and report what changes would have been made. This is useful for previewing the impact of your playbook without actually applying changes.<\/li>\n<\/ul>\n\n\n\n<p>Some of the tasks may show as failed because, the destination files to make changes on have not been copied yet to the target hosts.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"deploy-elk-stack-8-cluster-on-docker-containers-using-ansible\">Deploy ELK Stack 8 Cluster on Docker Containers using Ansible<\/h3>\n\n\n\n<p>When ready to run the play, remove option -C.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible-playbook main.yml --ask-become-pass<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>BECOME password: \n[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details\n\nPLAY [all] *********************************************************************************************************************************************************\n\nTASK [Gathering Facts] *********************************************************************************************************************************************\nok: [192.168.122.47]\nok: [192.168.122.60]\nok: [192.168.122.123]\nok: [192.168.122.152]\n\nTASK [install-docker : Install Packages Required to install Various Repos and GPG Keys:Ubuntu\/Debian] **************************************************************\nskipping: [192.168.122.47]\nok: [192.168.122.152]\nok: [192.168.122.60]\nok: [192.168.122.123]\n\nTASK [install-docker : Install Docker Engine Repo GPG Key on Ubuntu\/Debian] ****************************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.123]\nchanged: [192.168.122.60]\nchanged: [192.168.122.152]\n\nTASK [install-docker : Install Docker Engine Repository on Ubuntu\/Debian] ******************************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.152]\nchanged: [192.168.122.60]\nchanged: [192.168.122.123]\n\nTASK [install-docker : Install Docker Repos on CentOS\/Rocky\/RHEL Derivatives] **************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.123]\nskipping: [192.168.122.152]\nskipping: [192.168.122.47]\n\nTASK [install-docker : Install Docker CE on Ubuntu\/Debian] *********************************************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.60]\nchanged: [192.168.122.123]\nchanged: [192.168.122.152]\n\nTASK [install-docker : Install Docker CE on CentOS\/RHEL\/Rocky] *****************************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.123]\nskipping: [192.168.122.152]\nskipping: [192.168.122.47]\n\nTASK [install-docker : Start and Enable Docker Engine] *************************************************************************************************************\nskipping: [192.168.122.47]\nok: [192.168.122.123]\nok: [192.168.122.60]\nok: [192.168.122.152]\n\nTASK [install-docker : Add User to Docker group] *******************************************************************************************************************\nskipping: [192.168.122.47]\nok: [192.168.122.60]\nok: [192.168.122.152]\nok: [192.168.122.123]\n\nTASK [install-docker : Install Docker Compose] *********************************************************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.60]\nchanged: [192.168.122.123]\nchanged: [192.168.122.152]\n\nTASK [extra-configs : Update Virtual Memory Settings on All Cluster Nodes] *****************************************************************************************\nskipping: [192.168.122.47]\nok: [192.168.122.60]\nok: [192.168.122.152]\nok: [192.168.122.123]\n\nTASK [extra-configs : Update hosts file] ***************************************************************************************************************************\nskipping: [192.168.122.47] => (item=192.168.122.60 es01.kifarunix-demo.com es01) \nskipping: [192.168.122.47] => (item=192.168.122.123 es02.kifarunix-demo.com es02) \nskipping: [192.168.122.47] => (item=192.168.122.152 es03.kifarunix-demo.com es03) \nskipping: [192.168.122.47]\nok: [192.168.122.123] => (item=192.168.122.60 es01.kifarunix-demo.com es01)\nok: [192.168.122.152] => (item=192.168.122.60 es01.kifarunix-demo.com es01)\nok: [192.168.122.60] => (item=192.168.122.60 es01.kifarunix-demo.com es01)\nok: [192.168.122.152] => (item=192.168.122.123 es02.kifarunix-demo.com es02)\nok: [192.168.122.60] => (item=192.168.122.123 es02.kifarunix-demo.com es02)\nok: [192.168.122.123] => (item=192.168.122.123 es02.kifarunix-demo.com es02)\nok: [192.168.122.60] => (item=192.168.122.152 es03.kifarunix-demo.com es03)\nok: [192.168.122.152] => (item=192.168.122.152 es03.kifarunix-demo.com es03)\nok: [192.168.122.123] => (item=192.168.122.152 es03.kifarunix-demo.com es03)\n\nTASK [configure-nfs-share : Check if NFS Share is already configured] **********************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.123]\nskipping: [192.168.122.152]\nchanged: [192.168.122.47]\n\nTASK [configure-nfs-share : Configure NFS Share] *******************************************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.123]\nskipping: [192.168.122.152]\nchanged: [192.168.122.47]\n\nTASK [configure-nfs-share : Create ELK Stack Docker Volumes Directories] *******************************************************************************************\nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/certs) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/configs\/logstash) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/certs) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/elasticsearch\/01) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/elasticsearch\/02) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/elasticsearch\/03) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/configs\/logstash) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/kibana\/01) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/elasticsearch\/01) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/kibana\/02) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/elasticsearch\/02) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/certs) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/kibana\/03) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/elasticsearch\/03) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/logstash\/01) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/kibana\/01) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/logstash\/02) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/kibana\/02) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/configs\/logstash) \nskipping: [192.168.122.60] => (item=\/mnt\/elkstack\/data\/logstash\/03) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/elasticsearch\/01) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/kibana\/03) \nskipping: [192.168.122.60]\nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/logstash\/01) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/elasticsearch\/02) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/logstash\/02) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/elasticsearch\/03) \nskipping: [192.168.122.123] => (item=\/mnt\/elkstack\/data\/logstash\/03) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/kibana\/01) \nskipping: [192.168.122.123]\nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/kibana\/02) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/kibana\/03) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/logstash\/01) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/logstash\/02) \nskipping: [192.168.122.152] => (item=\/mnt\/elkstack\/data\/logstash\/03) \nskipping: [192.168.122.152]\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/certs)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/configs\/logstash)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/elasticsearch\/01)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/elasticsearch\/02)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/elasticsearch\/03)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/kibana\/01)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/kibana\/02)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/kibana\/03)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/logstash\/01)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/logstash\/02)\nchanged: [192.168.122.47] => (item=\/mnt\/elkstack\/data\/logstash\/03)\n\nTASK [configure-nfs-share : Reload NFS Exports] ********************************************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.123]\nskipping: [192.168.122.152]\nchanged: [192.168.122.47]\n\nTASK [copy-docker-compose : Create directory for ELK Stack Docker Compose files] ***********************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.60]\nchanged: [192.168.122.123]\nchanged: [192.168.122.152]\n\nTASK [copy-docker-compose : Create Docker-Compose file for ELK Node 01] ********************************************************************************************\nskipping: [192.168.122.123]\nskipping: [192.168.122.152]\nskipping: [192.168.122.47]\nchanged: [192.168.122.60]\n\nTASK [copy-docker-compose : Create Docker-Compose file for ELK Node 02\/03] *****************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.47]\nchanged: [192.168.122.123]\nchanged: [192.168.122.152]\n\nTASK [copy-docker-compose : Copy Environment Variables] ************************************************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.152]\nchanged: [192.168.122.123]\nchanged: [192.168.122.60]\n\nTASK [copy-docker-compose : Update the NODENN variable accordingly] ************************************************************************************************\nskipping: [192.168.122.60]\nskipping: [192.168.122.47]\nchanged: [192.168.122.123]\nchanged: [192.168.122.152]\n\nTASK [deploy-elk-cluster : Deploy ELK Stack on Docker using Docker Compose] ****************************************************************************************\nskipping: [192.168.122.47]\nchanged: [192.168.122.123]\nchanged: [192.168.122.152]\nchanged: [192.168.122.60]\n\nPLAY RECAP *********************************************************************************************************************************************************\n192.168.122.123            : ok=15   changed=9    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   \n192.168.122.152            : ok=15   changed=9    unreachable=0    failed=0    skipped=7    rescued=0    ignored=0   \n192.168.122.47             : ok=5    changed=4    unreachable=0    failed=0    skipped=17   rescued=0    ignored=0   \n192.168.122.60             : ok=14   changed=8    unreachable=0    failed=0    skipped=8    rescued=0    ignored=0\n<\/code><\/pre>\n\n\n\n<p>And that is it!!<\/p>\n\n\n\n<p>Our ELK Stack 8 Cluster is now running on Docker containers!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"verify-elk-stack-8-cluster\">Verify ELK Stack 8 Cluster<\/h3>\n\n\n\n<p>Now, login to one of the ELK Stack nodes and check the cluster status;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>kifarunix@es01:~$ docker exec -it es01 bash -c 'curl -k -XGET https:\/\/es01:9200\/_cat\/nodes -u elastic'<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>Enter host password for user 'elastic':\n192.168.122.152 39 98 44 2.53 1.17 0.75 cdfhilmrstw - es03\n192.168.122.123 48 95 44 2.63 1.21 0.78 cdfhilmrstw - es02\n192.168.122.60  65 97 35 1.96 0.87 0.43 cdfhilmrstw * es01\n<\/code><\/pre>\n\n\n\n<p>Cluster is up!! ES01 is the master!<\/p>\n\n\n\n<p>Check health;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>kifarunix@es02:~$ docker exec -it es02 bash -c 'curl -k -XGET https:\/\/es01:9200\/_cat\/health?v -u elastic'\n<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>epoch      timestamp cluster            status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent\n1706642249 19:17:29  elk-docker-cluster green           3         3     57  28    0    0        0             0                  -                100.0%\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"accessing-elk-stack-kibana-interface\">Accessing ELK Stack Kibana Interface<\/h3>\n\n\n\n<p>The ELK stack cluster is now up!<\/p>\n\n\n\n<p>You can access on any nodes&#8217;s Kibana;<\/p>\n\n\n\n<p>http:\/\/node01:5601, http:\/\/node02:5601 or http:\/\/node03:5601.<\/p>\n\n\n\n<p>Same login credentials you defined in compose envs file!<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1360\" height=\"747\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-ui-elk-stack-docker-ansible.png?v=1706647601\" alt=\"\" class=\"wp-image-19983\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-ui-elk-stack-docker-ansible.png?v=1706647601 1360w, https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-ui-elk-stack-docker-ansible-768x422.png?v=1706647601 768w\" sizes=\"(max-width: 1360px) 100vw, 1360px\" \/><\/figure><\/div>\n\n\n<p>Dashboard<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1607\" height=\"779\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-interface.png?v=1706647636\" alt=\"\" class=\"wp-image-19984\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-interface.png?v=1706647636 1607w, https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-interface-768x372.png?v=1706647636 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2024\/01\/kibana-interface-1536x745.png?v=1706647636 1536w\" sizes=\"(max-width: 1607px) 100vw, 1607px\" \/><\/figure><\/div>\n\n\n<p>And that is it!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"tear-down-the-stack\">Tear Down the Stack<\/h3>\n\n\n\n<p>If you want to tear down, just use the clean play;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>vim main.yml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>---\n- hosts: all\n  gather_facts: True\n  remote_user: \"{{ target_user }}\"\n  become: yes\n  roles:\n#    - install-docker\n#    - extra-configs\n#    - configure-nfs-share\n#    - copy-docker-compose\n#    - deploy-elk-cluster\n#    - remove_initial_master_nodes_setting\n    - clean\n<\/code><\/pre>\n\n\n\n<p>Save and exit. Then run the play!<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ansible-playbook main.yml --ask-become-pass<\/code><\/pre>\n\n\n\n<p>Feel free to update all the configurations provided in this guide to suite your needs!<\/p>\n\n\n\n<p>That marks the end of our tutorial on how to deploying ELK stack 8 cluster on Docker using Ansible.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, you will learn how to deploy ELK stack 8 Cluster on Docker using Ansible, an an open-source automation tool used for configuration<\/p>\n","protected":false},"author":10,"featured_media":19989,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[72,1076,1077,910,121],"tags":[7368,7369,7367],"class_list":["post-19950","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-monitoring","category-containers","category-docker","category-elastic-stack","category-howtos","tag-ansible-elk-stack-8","tag-elk-ansible","tag-elk-stack-8-docker-ansible","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/19950"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=19950"}],"version-history":[{"count":29,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/19950\/revisions"}],"predecessor-version":[{"id":20906,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/19950\/revisions\/20906"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/19989"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=19950"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=19950"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=19950"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}