{"id":14647,"date":"2022-11-02T23:06:55","date_gmt":"2022-11-02T20:06:55","guid":{"rendered":"https:\/\/kifarunix.com\/?p=14647"},"modified":"2024-03-09T23:27:49","modified_gmt":"2024-03-09T20:27:49","slug":"how-to-integrate-elk-stack-with-thehive","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/how-to-integrate-elk-stack-with-thehive\/","title":{"rendered":"How to Integrate ELK Stack with TheHive"},"content":{"rendered":"\n<p>In this tutorial, we are going to learn how to integrate ELK stack with TheHive. ELK Stack can be configured to sent event alerts to <a href=\"https:\/\/thehive-project.org\/\" target=\"_blank\" rel=\"noreferrer noopener\">TheHive<\/a> case management system. This enables security personnel to create, investigate and follow up on various incidents or cases.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Integrating ELK Stack with TheHive<\/h2>\n\n\n\n<p>In order to integrate ELK stack with TheHive, proceed as follows;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Install ELK Stack<\/h3>\n\n\n\n<p>Follow the link below to learn how t install ELK Stack;<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/?s=install+elk\" target=\"_blank\" rel=\"noreferrer noopener\">Install and Configure ELK Stack<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Install TheHive<\/h3>\n\n\n\n<p>You can check the guide below on how to install TheHive;<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-thehive-on-ubuntu\/\" target=\"_blank\" rel=\"noreferrer noopener\">Install TheHive on Ubuntu<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Configure ELK Stack Alerting Via ElastAlert<\/h3>\n\n\n\n<p>&#8220;<em>ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch&#8221;.<\/em><\/p>\n\n\n\n<p>In our previous guide, we learnt how to <a href=\"https:\/\/kifarunix.com\/configure-elk-stack-alerting-with-elastalert\/\" target=\"_blank\" rel=\"noreferrer noopener\">Configure ELK Stack Alerting with ElastAlert<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Configure ELK Stack to Sent Alerts to TheHive<\/h3>\n\n\n\n<p>Based on the guide above on how to configure ELK stack alerting with ElastAlert, you need to configure ElastAlert to sent alerts to TheHive.<\/p>\n\n\n\n<p>ElastAlert can be configured to read specific Elasticsearch indices and check for specific events based on the defined query conditions. When those conditions are true, it can sent alerts, via the theHive alert type, to TheHive.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Create Organization and Administrative User on TheHive<\/h4>\n\n\n\n<p>To begin with, login to TheHive web interface as admin user and create your organization and an administrative user for that account, if you have not done so already.<\/p>\n\n\n\n<p>Follow the links below to learn how to create TheHive organization and an administrative user for that organization.<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-thehive-on-ubuntu\/#create-thehive-org\" target=\"_blank\" rel=\"noreferrer noopener\">Creating an Organization on TheHive<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-thehive-on-ubuntu\/#create-thehive-admin-user\" target=\"_blank\" rel=\"noreferrer noopener\">Create TheHive Organization Administrative User<\/a><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Create TheHive API User<\/h4>\n\n\n\n<p>Login as the administrative user to your TheHive organization and add another less privileged user with permissions to create alerts via API. Remember, ElastAlert will be configured to sent alerts to TheHive via API. Hence, we will use the profile <strong>analyst<\/strong> for the API user.<\/p>\n\n\n\n<p>Hence, under your specific&nbsp;Organization &gt; click Users &gt; +. Enter the login username, the name and choose the privileges (<strong>analyst<\/strong>).<\/p>\n\n\n\n<div><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-api-user.png\" class=\"td-modal-image\"><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1470\" height=\"945\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-api-user.png\" alt=\"How to Integrate ELK Stack with TheHive\" class=\"wp-image-14662\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-api-user.png?v=1667371070 1470w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-api-user-768x494.png?v=1667371070 768w\" sizes=\"(max-width: 1470px) 100vw, 1470px\" \/><\/figure><\/a><\/div>\n\n\n\n<p>Click <strong>Confirm<\/strong> to add the user.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Generate TheHive Alerts API Key for the API User<\/h4>\n\n\n\n<p>Next, you need to generate the API key for the API user.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>On the Organization users, hover your mouse over the API user and click <strong>Preview<\/strong>.<\/li>\n\n\n\n<li>Under the users settings wizard, API Key, click <strong>Create<\/strong> to generate API key.<\/li>\n<\/ul>\n\n\n\n<div><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/create-api-user-key.png\" class=\"td-modal-image\"><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1803\" height=\"936\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/create-api-user-key.png\" alt=\"How to Integrate ELK Stack with TheHive\" class=\"wp-image-14664\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/create-api-user-key.png?v=1667371338 1803w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/create-api-user-key-768x399.png?v=1667371338 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/create-api-user-key-1536x797.png?v=1667371338 1536w\" sizes=\"(max-width: 1803px) 100vw, 1803px\" \/><\/figure><\/a><\/div>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Once the key is generated, you can close the user settings wizard.<\/li>\n\n\n\n<li>You can access the key later by navigating to users <strong>Preview<\/strong> &gt; <strong>API Key &gt; Reveal<\/strong>.<\/li>\n<\/ul>\n\n\n\n<div><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/reveal-api-key.png\" class=\"td-modal-image\"><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1458\" height=\"650\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/reveal-api-key.png\" alt=\"How to Integrate ELK Stack with TheHive\" class=\"wp-image-14665\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/reveal-api-key.png?v=1667371554 1458w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/reveal-api-key-768x342.png?v=1667371554 768w\" sizes=\"(max-width: 1458px) 100vw, 1458px\" \/><\/figure><\/a><\/div>\n\n\n\n<h4 class=\"wp-block-heading\">Configure ElastAlert to Sent Alerts to TheHive<\/h4>\n\n\n\n<p>Note that we are using <a href=\"https:\/\/elastalert2.readthedocs.io\/en\/latest\/elastalert.html\" target=\"_blank\" rel=\"noreferrer noopener\">ElastAlert2<\/a> in our demo setup.<\/p>\n\n\n\n<p>Define ElastAlert configuration options;<\/p>\n\n\n\n<p>This is my sample Elastalert config for thehive;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/opt\/elastalert\/config-thehive.yaml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>rules_folder: example_rules\nrun_every:\n  minutes: 5\nbuffer_time:\n  minutes: 15\nes_host: 192.168.58.22\nes_port: 9200\nwriteback_index: elastalert_status\nwriteback_alias: elastalert_alerts\nalert_time_limit:\n  days: 2\n<\/code><\/pre>\n\n\n\n<p>Update your configuration accordingly;<\/p>\n\n\n\n<p>Create ElastAlert alerting rules for TheHive;<\/p>\n\n\n\n<p>For this, you will need to get the API key generated above.<\/p>\n\n\n\n<p>This is our sample rules file;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/opt\/elastalert\/example_rules\/thehive.yaml<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\nname: Sample SSH Rule\ntype: frequency\nnum_events: 3\ntimeframe:\n  minutes: 1\nfilter:\n- query:\n    query_string:\n      query: \"event.type:authentication_failure\"\nindex: filebeat-*\nrealert:\n  minutes: 1\nquery_key:\n  - source.ip\ninclude:\n  - source.ip\n  - host.hostname\n  - user.name\n\nalert: hivealerter\nhive_connection:\n  hive_host: http:\/\/thehive.kifarunix-demo.com\n  hive_port: 9000\n  hive_apikey: Uf0W20Mf9UTYrLuI\/hyn74ni9UzFZJvb\n  hive_proxies:\n    http: ''\n    https: ''\n\nhive_alert_config:\n  title: 'SSH Bruteforce Attacks'\n  type: 'external'\n  source: 'elastalert'\n  description: 'SSH Bruteforce Attacks'\n  severity: 2\n  tags: ['ssh', 'bruteforce', 'authentications']\n  tlp: 3\n  status: 'New'\n  follow: True\n\nhive_observable_data_mapping:\n    - ip: source.ip\n    - hostname: host.hostname\n<\/code><\/pre>\n\n\n\n<p>Test the rule;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elastalert-test-rule --config \/opt\/elastalert\/config-thehive.yaml \/opt\/elastalert\/example_rules\/thehive.yaml<\/code><\/pre>\n\n\n\n<p>Sample output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\nINFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.\n            To send them but remain verbose, use --verbose instead.\nGot 155 hits from the last 0 day\n\nAvailable terms in first hit:\n\tagent.hostname\n\tagent.name\n\tagent.id\n\tagent.ephemeral_id\n\tagent.type\n\tagent.version\n\tprocess.name\n\tprocess.pid\n\tlog.file.path\n\tlog.offset\n\tsource.port\n\tsource.ip\n\tfileset.name\n\tinput.type\n\t@timestamp\n\tsystem.auth.ssh.method\n\tsystem.auth.ssh.event\n\tecs.version\n\trelated.hosts\n\trelated.ip\n\trelated.user\n\tservice.type\n\thost.hostname\n\thost.os.kernel\n\thost.os.codename\n\thost.os.name\n\thost.os.type\n\thost.os.family\n\thost.os.version\n\thost.os.platform\n\thost.containerized\n\thost.ip\n\thost.name\n\thost.id\n\thost.mac\n\thost.architecture\n\tevent.ingested\n\tevent.timezone\n\tevent.kind\n\tevent.module\n\tevent.action\n\tevent.type\n\tevent.category\n\tevent.dataset\n\tevent.outcome\n\tuser.name\n\nINFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.\n                To send them but remain verbose, use --verbose instead.\nINFO:elastalert:1 rules loaded\nINFO:apscheduler.scheduler:Adding job tentatively -- it will be properly scheduled when the scheduler starts\nINFO:elastalert:Queried rule Sample SSH Rule from 2022-11-02 22:30 EAT to 2022-11-02 22:31 EAT: 4 \/ 4 hits\nINFO:elastalert:Alert for Sample SSH Rule at 2022-11-02T22:31:17+03:00:\nINFO:elastalert:Sample SSH Rule\n\nAt least 3 events occurred between 2022-11-02 22:30 EAT and 2022-11-02 22:31 EAT\n\n@timestamp: 2022-11-02T22:31:17+03:00\n_id: HK3TOYQBtG6LAFezqKdP\n_index: filebeat-7.17.0-2022.10.20-000001\n_type: _doc\nhost: {\n    \"hostname\": \"debian11\"\n}\nnum_hits: 4\nnum_matches: 1\nsource: {\n    \"ip\": \"192.168.58.50\"\n}\nuser: {\n    \"name\": \"socadmin\"\n}\n\n\nWould have written the following documents to writeback index (default is elastalert_status):\n\nsilence - {'exponent': 0, 'rule_name': 'Sample SSH Rule.192.168.58.50', '@timestamp': datetime.datetime(2022, 11, 2, 19, 31, 29, 378481, tzinfo=tzutc()), 'until': datetime.datetime(2022, 11, 2, 19, 36, 29, 378473, tzinfo=tzutc())}\n\nelastalert_status - {'rule_name': 'Sample SSH Rule', 'endtime': datetime.datetime(2022, 11, 2, 19, 31, 29, 222934, tzinfo=tzutc()), 'starttime': datetime.datetime(2022, 11, 2, 19, 30, 28, 622934, tzinfo=tzutc()), 'matches': 1, 'hits': 4, '@timestamp': datetime.datetime(2022, 11, 2, 19, 31, 29, 379893, tzinfo=tzutc()), 'time_taken': 0.008604764938354492}\n<\/code><\/pre>\n\n\n\n<p>As you can see from the test above, four events found related to failed SSH authentications within one minute.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Run ElastAlert Against the TheHive Rule<\/h4>\n\n\n\n<p>To check if alerts can be sent to TheHive, let&#8217;s run the rule;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\/usr\/local\/bin\/elastalert --verbose --config \/opt\/elastalert\/config-thehive.yaml --rule \/opt\/elastalert\/example_rules\/thehive.yaml<\/code><\/pre>\n\n\n\n<p>Now, simulate failed events to one your systems that are already collecting and sending logs to ELK stack;<\/p>\n\n\n\n<p>Sample rule output;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code>\nINFO:elastalert:1 rules loaded\nINFO:elastalert:Starting up\nINFO:elastalert:Disabled rules are: []\nINFO:elastalert:Sleeping for 299.999836 seconds\nINFO:elastalert:Queried rule Sample SSH Rule from 2022-11-02 22:27 EAT to 2022-11-02 22:32 EAT: 16 \/ 16 hits\nINFO:elastalert:Alert sent to TheHive\nINFO:elastalert:Ran Sample SSH Rule from 2022-11-02 22:27 EAT to 2022-11-02 22:32 EAT: 16 query hits (0 already seen), 5 matches, 1 alerts sent\nINFO:elastalert:Sample SSH Rule range 337\n<\/code><\/pre>\n\n\n\n<p>As you can see, an alert has been sent to TheHive based on the alert query event match.<\/p>\n\n\n\n<p>Login to TheHive and verify the alerts;<\/p>\n\n\n\n<div><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/the-hive-elastalerts.png\" class=\"td-modal-image\"><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1899\" height=\"694\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/the-hive-elastalerts.png\" alt=\"How to Integrate ELK Stack with TheHive\" class=\"wp-image-14676\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/the-hive-elastalerts.png?v=1667418168 1899w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/the-hive-elastalerts-768x281.png?v=1667418168 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/the-hive-elastalerts-1536x561.png?v=1667418168 1536w\" sizes=\"(max-width: 1899px) 100vw, 1899px\" \/><\/figure><\/a><\/div>\n\n\n\n<p>Hover the mouse over alert and click <strong>preview<\/strong> &gt; <strong>Go to details<\/strong> to see more details;<\/p>\n\n\n\n<div><a href=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-alert-details.png\" class=\"td-modal-image\"><figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1892\" height=\"812\" src=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-alert-details.png\" alt=\"How to Integrate ELK Stack with TheHive\" class=\"wp-image-14677\" title=\"\" srcset=\"https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-alert-details.png?v=1667418592 1892w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-alert-details-768x330.png?v=1667418592 768w, https:\/\/kifarunix.com\/wp-content\/uploads\/2022\/11\/thehive-alert-details-1536x659.png?v=1667418592 1536w\" sizes=\"(max-width: 1892px) 100vw, 1892px\" \/><\/figure><\/a><\/div>\n\n\n\n<p>And that is all on integrating ELK stack with TheHive. You can now proceed with further Hive actions based on the event.<\/p>\n\n\n\n<p>Other Tutorials;<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/detecting-malicious-files-with-wazuh-and-virustotal\/\" target=\"_blank\" rel=\"noreferrer noopener\">Detecting Malicious Files with Wazuh and VirusTotal<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/send-alert-when-clamav-finds-infected-files-on-linux-systems\/\" target=\"_blank\" rel=\"noreferrer noopener\">Send Alert When ClamAV Finds Infected Files on Linux Systems<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this tutorial, we are going to learn how to integrate ELK stack with TheHive. ELK Stack can be configured to sent event alerts to<\/p>\n","protected":false},"author":3,"featured_media":14679,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[121,72,34],"tags":[6039,6037,1852,6035,6036,6038],"class_list":["post-14647","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-howtos","category-monitoring","category-security","tag-case-management","tag-create-elk-event-alerts-on-thehive","tag-elk-stack","tag-how-to-integrate-elk-stack-with-thehive","tag-sent-elk-alerts-to-thehive","tag-thehive","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/14647"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=14647"}],"version-history":[{"count":11,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/14647\/revisions"}],"predecessor-version":[{"id":20664,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/14647\/revisions\/20664"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/14679"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=14647"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=14647"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=14647"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}