{"id":6563,"date":"2020-05-01T17:49:33","date_gmt":"2020-05-01T14:49:33","guid":{"rendered":"https:\/\/kifarunix.com\/?p=6563"},"modified":"2024-03-14T20:48:58","modified_gmt":"2024-03-14T17:48:58","slug":"create-kibana-visualization-dashboards-for-modsecurity-logs","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/create-kibana-visualization-dashboards-for-modsecurity-logs\/","title":{"rendered":"Create Kibana Visualization Dashboards for ModSecurity Logs"},"content":{"rendered":"\n<p>In this tutorial, you will learn how to create Kibana visualization dashboards for ModSecurity logs. This tutorial is a continuation of our previous tutorial on how to process and visualize ModSecurity Logs on ELK Stack where we covered various grok filters\/regular expressions for extracting various fields from the <a href=\"https:\/\/modsecurity.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">ModSecurity<\/a> audit logs. Hence, before you can proceed, ensure that you have checked the tutorial by following the link below.<\/p>\n\n\n\n<p><a aria-label=\"undefined (opens in a new tab)\" href=\"https:\/\/kifarunix.com\/process-and-visualize-modsecurity-logs-on-elk-stack\/\" target=\"_blank\" rel=\"noreferrer noopener\">Process and Visualize ModSecurity Logs on ELK Stack<\/a><\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"kibana-modsecurity-dashboards\"><a href=\"#kibana-modsecurity-dashboards\">Create Kibana Dashboards for ModSecurity Logs<\/a><\/h2>\n\n\n\n<p>Once you have created and verified your ModSecurity logstash filters, proceed to create visualization dashboards for your ModSecurity logs based on the fields extracted by your filters.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote td_pull_quote td_pull_center is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Kibana visualizations are based on Elasticsearch queries. By using a series of Elasticsearch&nbsp;<a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/7.8\/search-aggregations.html\" target=\"_blank\" rel=\"noopener\">aggregations<\/a>&nbsp;to extract and process your data, you can create charts that show you the trends, spikes, and dips you need to know about.<\/p>\n<\/blockquote>\n\n\n\n<p>There are different types of Kibana visualizations that you can use with the most fequently used including;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Line, area, and bar charts<\/strong>&nbsp;\u2014 Compares different series in X\/Y charts.<\/li>\n\n\n\n<li><strong>Pie chart<\/strong>&nbsp;\u2014 Displays each source contribution to a total.<\/li>\n\n\n\n<li><strong>Data table<\/strong>&nbsp;\u2014 Flattens aggregations into table format.<\/li>\n\n\n\n<li><strong>Metric<\/strong>&nbsp;\u2014 Displays a single number.<\/li>\n\n\n\n<li><strong>Goal and gauge<\/strong>&nbsp;\u2014 Displays a number with progress indicators.<\/li>\n\n\n\n<li><strong>Tag cloud<\/strong>&nbsp;\u2014 Displays words in a cloud, where the size of the word corresponds to its importance.<\/li>\n<\/ul>\n\n\n\n<p>You can create Visualizations on Kibana by navigating to <strong>Visualize<\/strong> menu.<\/p>\n\n\n\n<p>Hence, click on the three menu lines at the top left corner on Kibana web interface &gt; <strong>Visualize<\/strong> &gt;&nbsp;<strong>Create visualization<\/strong>.<\/p>\n\n\n\n<p>From there, you can now choose the type of Visualization you want to create.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1036\" height=\"653\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/new-visualization.png\" alt=\"\" class=\"wp-image-6568\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/new-visualization.png?v=1596205105 1036w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/new-visualization-768x484.png?v=1596205105 768w\" sizes=\"(max-width: 1036px) 100vw, 1036px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Top 10 Attacks<\/h3>\n\n\n\n<p>To create Kibana Visualization for the ModSecurity Top 10 attacks, we will use a pie chart.<\/p>\n\n\n\n<p>Therefore click <strong>Pie<\/strong> on the visualization window above.<\/p>\n\n\n\n<p>Choose the your data source index. In this case, we use our <strong>modsec-*<\/strong> Elasticsearch index.<\/p>\n\n\n\n<p>This opens up a default Pie chart with one slice as shown below;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1913\" height=\"917\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-pie-chart-settings.png\" alt=\"\" class=\"wp-image-6570\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-pie-chart-settings.png?v=1596205345 1913w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-pie-chart-settings-768x368.png?v=1596205345 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-pie-chart-settings-1536x736.png?v=1596205345 1536w\" sizes=\"(max-width: 1913px) 100vw, 1913px\" \/><\/figure>\n\n\n\n<p>Next, you need to define your Pie Chart Metrics and Buckets.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><code>Buckets<\/code><\/strong> are used to group data or sets of documents based on certain criteria. <strong><code>Bucket aggregation<\/code><\/strong> is used to specify the slices to display in a Pie chart. <strong>Aggregation<\/strong>&nbsp;refers to the collection of documents or a set of documents obtained from a particular search query or filter.<\/li>\n\n\n\n<li><strong><code>Metrics<\/code><\/strong> on the other hand refers to values extracted from the documents that are being aggregated. Example is the <strong><code>Numeric metrics aggregations<\/code><\/strong> which output numeric values.<\/li>\n<\/ul>\n\n\n\n<p>Under <strong>Data<\/strong> tab;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Choose the <strong>Metrics<\/strong> aggregation type. We use the default one here, <strong><code>Count<\/code><\/strong> (returns a raw count of the elements in the selected index pattern).<\/li>\n\n\n\n<li>On the <strong>Buckets<\/strong>, click <strong>Add<\/strong> &gt; <strong>Split slices<\/strong>.<\/li>\n\n\n\n<li>Select the type of Bucket Aggregation. Choose <strong>Terms<\/strong> (enables you to specify the top or bottom&nbsp;<em>n<\/em>&nbsp;elements of a given field to display, ordered by count or a custom metric).<\/li>\n\n\n\n<li>Select the field from which data should be extracted. In this case, we select <strong>attack.type<\/strong>.<\/li>\n\n\n\n<li>Order by <strong>Count<\/strong> Metrics.<\/li>\n\n\n\n<li>Set <strong>Descending<\/strong> order and set the size to <strong>10<\/strong>.<\/li>\n<\/ol>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1918\" height=\"957\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/chart-data-settings.png\" alt=\"\" class=\"wp-image-6569\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/chart-data-settings.png?v=1596205180 1918w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/chart-data-settings-768x383.png?v=1596205180 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/chart-data-settings-1536x766.png?v=1596205180 1536w\" sizes=\"(max-width: 1918px) 100vw, 1918px\" \/><\/figure>\n\n\n\n<p>Under the <strong>Options<\/strong> tab, are the chart customizations options. We disabled the <strong>Donut<\/strong> format for the chart in this section and also enabled <strong>Show labels<\/strong>.<\/p>\n\n\n\n<p>Click <strong>Update<\/strong> button at the bottom right corner to save you chart settings.<\/p>\n\n\n\n<p>Once done setting up the chart, click the <strong>Save<\/strong> button at the top left to name your chart and save it. We call the chart <strong>Top 10 Attacks<\/strong>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1907\" height=\"831\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-attack-types-chart-.png\" alt=\"\" class=\"wp-image-6572\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-attack-types-chart-.png?v=1596205835 1907w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-attack-types-chart--768x335.png?v=1596205835 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-attack-types-chart--1536x669.png?v=1596205835 1536w\" sizes=\"(max-width: 1907px) 100vw, 1907px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Top 10 User Agents<\/h3>\n\n\n\n<p>Similarly, we use the same approach above to create a new chart for the Top 10 user agents.<\/p>\n\n\n\n<p>However, when selecting the data field, we select <strong>user_agent.keyword<\/strong> as per our Elasticsearch index field.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1909\" height=\"918\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/Top-10-user-agents.png\" alt=\"\" class=\"wp-image-6573\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/Top-10-user-agents.png?v=1596205860 1909w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/Top-10-user-agents-768x369.png?v=1596205860 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/Top-10-user-agents-1536x739.png?v=1596205860 1536w\" sizes=\"(max-width: 1909px) 100vw, 1909px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Top 10 Attacker IPs<\/h3>\n\n\n\n<p>Next, let us create a visualization for the Top 10 Attacker source IPs and their count.<\/p>\n\n\n\n<p>In this visualization, we will use <strong>Data table<\/strong> aggregation type.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"create-kibana-data-table\"><a href=\"#create-kibana-data-table\">Create Kibana Data Table<\/a><\/h4>\n\n\n\n<p>To create Kibana Data table, navigate to Kibana <strong>Visualize<\/strong> menu &gt; <strong>Create visualization<\/strong> &gt; <strong>Data table<\/strong>.<\/p>\n\n\n\n<p>Select the Elasticsearch datasource index.<\/p>\n\n\n\n<p>This opens up a default table with just the <strong>Count<\/strong> aggregation metric enabled.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1918\" height=\"551\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-data-table.png\" alt=\"\" class=\"wp-image-6575\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-data-table.png?v=1596205946 1918w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-data-table-768x221.png?v=1596205946 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/default-data-table-1536x441.png?v=1596205946 1536w\" sizes=\"(max-width: 1918px) 100vw, 1918px\" \/><\/figure>\n\n\n\n<p>You can create your data table based on an existing saved search or a new search.<\/p>\n\n\n\n<p>Under <strong>Data<\/strong> tab;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Under <strong>Metrics<\/strong>, select an aggregation type. Again, we use the default one here, <strong><code>Count<\/code><\/strong>.<\/li>\n\n\n\n<li>Under the <strong>Buckets<\/strong>, this is where we add the columns to be displayed on the table. Hence, click <strong>Add<\/strong> &gt; <strong>Split rows<\/strong>.<\/li>\n\n\n\n<li>Select the Bucket Aggregation type. Choose <strong>Terms<\/strong> (enables you to specify the top or bottom&nbsp;<em>n<\/em>&nbsp;elements of a given field to display, ordered by count or a custom metric).<\/li>\n\n\n\n<li>Select the field from which data should be extracted. In this case, we select <strong>src_ip.keyword<\/strong> as per our ES index fields.<\/li>\n\n\n\n<li>Order by <strong>Count<\/strong> Metrics.<\/li>\n\n\n\n<li>Set <strong>Descending<\/strong> order and set the size to 10.<\/li>\n<\/ol>\n\n\n\n<p>If you like, you can see other settings under <strong>Options<\/strong> tab.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1906\" height=\"887\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-source-ips.png\" alt=\"\" class=\"wp-image-6574\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-source-ips.png?v=1596205932 1906w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-source-ips-768x357.png?v=1596205932 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-source-ips-1536x715.png?v=1596205932 1536w\" sizes=\"(max-width: 1906px) 100vw, 1906px\" \/><\/figure>\n\n\n\n<p>Click the <strong>Save<\/strong> button at the top left to name your data table and save it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Top 10 Request URIs<\/h3>\n\n\n\n<p>You can also create Top 10 request URIs visualizations. For this, we use <strong>Pie<\/strong> chart and hence, the approach is the the same as above.<\/p>\n\n\n\n<p>For the data field, we use <strong><code>request_uri.keyword<\/code><\/strong> as per our ES index fields.<\/p>\n\n\n\n<p>Save and name your chart accordingly.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1910\" height=\"867\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-request-uris.png\" alt=\"Create Kibana Visualization Dashboards for ModSecurity Logs\" class=\"wp-image-6576\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-request-uris.png?v=1596205976 1910w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-request-uris-768x349.png?v=1596205976 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/top-10-request-uris-1536x697.png?v=1596205976 1536w\" sizes=\"(max-width: 1910px) 100vw, 1910px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"create-kibana-dashboards\"><a href=\"#create-kibana-dashboards\">Add Visualization Charts\/Tables to Kibana Dashboard<\/a><\/h2>\n\n\n\n<p>Once you have created your visualization charts or tables, you can now create your own dashboard where you can put together all the visualizations.<\/p>\n\n\n\n<p>To add Kibana visualizations to Kibana dashboard;<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>On Kibana menu,  Click&nbsp;<strong>Dashboard<\/strong> &gt;&nbsp;<strong>Create dashboard<\/strong>.<\/li>\n\n\n\n<li><strong>Add an existing<\/strong> visualizations we already created above.<\/li>\n<\/ol>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1918\" height=\"411\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/add-existing-dashboards.png\" alt=\"\" class=\"wp-image-6577\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/add-existing-dashboards.png?v=1596206022 1918w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/add-existing-dashboards-768x165.png?v=1596206022 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/add-existing-dashboards-1536x329.png?v=1596206022 1536w\" sizes=\"(max-width: 1918px) 100vw, 1918px\" \/><\/figure>\n\n\n\n<p>Select the visualizations panel to add to the dashboard by clicking on it.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1890\" height=\"960\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/kibana-modsecurity-visualization-dashboard.png\" alt=\"\" class=\"wp-image-6578\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/kibana-modsecurity-visualization-dashboard.png?v=1596206166 1890w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/kibana-modsecurity-visualization-dashboard-768x390.png?v=1596206166 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2020\/07\/kibana-modsecurity-visualization-dashboard-1536x780.png?v=1596206166 1536w\" sizes=\"(max-width: 1890px) 100vw, 1890px\" \/><\/figure>\n\n\n\n<p>Click <strong>Save<\/strong> button at the top of the page to save your dashboard.<\/p>\n\n\n\n<p>And there you go. You can add more visualizations as you wish.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Further Reading<\/h3>\n\n\n\n<p><a aria-label=\"undefined (opens in a new tab)\" href=\"https:\/\/www.elastic.co\/guide\/en\/kibana\/current\/visualize.html\" target=\"_blank\" rel=\"noreferrer noopener\">Kibana visualization<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Other Tutorials<\/h3>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/install-elastic-stack-7-on-fedora-30-fedora-29-centos-7\/\" target=\"_blank\">Install Elastic Stack 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-configure-filebeat-7-on-ubuntu-18-04-debian-9-8\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Configure Filebeat 7 on Ubuntu 18.04\/Debian 9.8<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-elastic-stack-7-on-ubuntu-18-04-debian-9-8\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install Elastic Stack 7 on Ubuntu 18.04\/Debian 9.8<\/a><\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/kifarunix.com\/how-to-debug-logstash-grok-filters\/\" target=\"_blank\">How to Debug Logstash Grok Filters<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, you will learn how to create Kibana visualization dashboards for ModSecurity logs. This tutorial is a continuation of our previous tutorial on<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[121],"tags":[1852,1856,1854,1139,1855,1849],"class_list":["post-6563","post","type-post","status-publish","format-standard","hentry","category-howtos","tag-elk-stack","tag-kibana-dashboards","tag-kibana-visualization-dashboards-for-modsecurity","tag-modsecurity","tag-process-modsecurity-logs-on-elastic-stack","tag-visualize-modsecurity-logs-on-kibana","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/6563"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=6563"}],"version-history":[{"count":4,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/6563\/revisions"}],"predecessor-version":[{"id":21408,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/6563\/revisions\/21408"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=6563"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=6563"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=6563"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}