{"id":3533,"date":"2019-07-15T20:37:18","date_gmt":"2019-07-15T17:37:18","guid":{"rendered":"https:\/\/kifarunix.com\/?p=3533"},"modified":"2020-01-24T20:06:35","modified_gmt":"2020-01-24T17:06:35","slug":"send-windows-logs-to-elastic-stack-using-winlogbeat-and-sysmon","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/send-windows-logs-to-elastic-stack-using-winlogbeat-and-sysmon\/","title":{"rendered":"Send Windows logs to Elastic Stack using Winlogbeat and Sysmon"},"content":{"rendered":"\n

In this guide, we are going to learn how to send Windows logs to Elastic Stack using Winlogbeat and Sysmon. Winlogbeat<\/a> is an Elastic Beat that is used to collect windows system application, security, system or hardware events. Sysmon (System Monitor<\/em>) on the other hand is a windows application that is used to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time<\/p>\n\n\n\n

In order to visualize and analyze the events collected by Winlogbeat\/Sysmon, you need to have setup your Elastic Stack. See the links below on how to install and setup Elastic Stack on Fedora\/CentOS\/Ubuntu servers.<\/p>\n\n\n\n

Install Elastic Stack 7 on Ubuntu 18.04\/Debian 9.8<\/a><\/p>\n\n\n\n

Install Elastic Stack 7 on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n

Send Windows Logs to Elastic Stack Using Winlogbeat and Sysmon<\/h2>\n\n\n\n

Install Winlogbeat and Sysmon on Windows 7<\/h3>\n\n\n\n

In this guide, we are going to use Windows 7 as our Windows system. Therefore, you need to install both Winlogbeat and Sysmon on your Windows 7 system in order to ship events to Elastic stack.<\/p>\n\n\n\n

Install Winlogbeat on Windows 7<\/h3>\n\n\n\n

Navigate to Winlogbeat downloads page<\/a> and download Winlogbeat zip file.<\/p>\n\n\n\n

Once the download is done, extract the Winlogbeat zipped file, winlogbeat-7.2.0-windows-x86_64.zip<\/strong>.<\/p>\n\n\n\n

When you extract, you should get a folder, winlogbeat-7.2.0-windows-x86_64<\/strong>.<\/p>\n\n\n\n

Move the winlogbeat-7.2.0-windows-x86_64<\/strong> directory to C:\\Program Files<\/strong> and rename it to Winlogbeat<\/strong>. Your directory should look like as in below;<\/p>\n\n\n\n

\"Install<\/a><\/figure>\n\n\n\n

Next, to install Winlogbeat on Windows 7, you need to execute the install-service-winlogbeat.ps1<\/strong> installation script. Hence, open the Powershell<\/strong> as the administrator and change to Winlogbeat directory by executing the command below;<\/p>\n\n\n\n

cd C:\\'Program Files'\\Winlogbeat<\/code><\/pre>\n\n\n\n
Next, run the Winlogbeat installer as shown below;<\/p>\n\n\n\n
.\\install-service-winlogbeat.ps1<\/code><\/pre>\n\n\n\n
If you get the error, cannot be loaded because the execution of scripts is disabled on this system<\/strong>, as shown below, you need to enable the script execution.<\/p>\n\n\n\n
PS C:\\Program Files\\Winlogbeat> .\\install-service-winlogbeat.ps1\nFile C:\\Program Files\\Winlogbeat\\install-service-winlogbeat.ps1 cannot be loaded because the execution of scripts is disabled on this system. Please see \"get-help about_signing\" for more details.\nAt line:1 char:33\n+ .\\install-service-winlogbeat.ps1 <<<<\n    + CategoryInfo          : NotSpecified: (:) [], PSSecurityException\n    + FullyQualifiedErrorId : RuntimeException\n<\/code><\/pre>\n\n\n\n
As a result, you need to execute the Winlogbeat script with unrestricted execution policy as shown in the command below;<\/p>\n\n\n\n
PS C:\\Program Files\\Winlogbeat> PowerShell.exe -ExecutionPolicy UnRestricted -File .\\install-service-winlogbeat.ps1\n\nStatus   Name               DisplayName\n------   ----               -----------\nStopped  winlogbeat         winlogbeat\n<\/code><\/pre>\n\n\n\n
Install Sysmon on Windows 7<\/h3>\n\n\n\n
Download Sysmon from the downloads page<\/a>.<\/p>\n\n\n\n
Once the download is complete, extract the contents of the zipped file to C:\\Program Files<\/strong> directory.<\/p>\n\n\n\n
Once the extraction is done, you folder should look like as in below;<\/p>\n\n\n\n
\"Install<\/a><\/figure>\n\n\n\n
Install Sysmon with md5 and sha256 hashing of process created, log loading of modules and monitoring network connections, open a CMD<\/strong> as an administrator and navigate to C:\\Program Files\\Sysmon<\/strong> and execute the command below;<\/p>\n\n\n\n
cd C:\\Program Files\\Sysmon<\/code><\/pre>\n\n\n\n
C:\\Program Files\\Sysmon> sysmon -i -accepteula -h md5,sha256,imphash -l -n\n\nSystem Monitor v10.2 - System activity monitor\nCopyright (C) 2014-2019 Mark Russinovich and Thomas Garnier\nSysinternals - www.sysinternals.com\n\nSysmon installed.\nSysmonDrv installed.\nStarting SysmonDrv.\nSysmonDrv started.\nStarting Sysmon...\nSysmon started.<\/code><\/pre>\n\n\n\n
Configuring Winlogbeat<\/h3>\n\n\n\n
The main configuration file for Winlogbeat is C:\\Program Files\\Winlogbeat\\winlogbeat.yml<\/strong> with the reference config file being C:\\Program Files\\Winlogbeat\\winlogbeat.reference.yml<\/strong>.<\/p>\n\n\n\n
To edit this file, you can use Notepad++.<\/p>\n\n\n\n
By default, Winlogbeat is set to monitor application, security, and system logs, and logs from Sysmon.<\/p>\n\n\n\n
...\nwinlogbeat.event_logs:\n  - name: Application\n    ignore_older: 72h\n\n  - name: System\n\n  - name: Security\n    processors:\n      - script:\n          lang: javascript\n          id: security\n          file: ${path.home}\/module\/security\/config\/winlogbeat-security.js\n\n  - name: Microsoft-Windows-Sysmon\/Operational\n    processors:\n      - script:\n          lang: javascript\n          id: sysmon\n          file: ${path.home}\/module\/sysmon\/config\/winlogbeat-sysmon.js\n...<\/code><\/pre>\n\n\n\n
If you need to see more event types, you can execute the command Get-EventLog *<\/code> in PowerShell.<\/p>\n\n\n\n
Under the general settings, we are going to setup the optional name of the Beat and the Tags associated with the events.<\/p>\n\n\n\n
...\n#================================ General =====================================\n\n# The name of the shipper that publishes the network data. It can be used to group\n# all the transactions sent by a single shipper in the web interface.\nname: winlogbeat\n\n# The tags of the shipper are included in their own field with each\n# transaction published.\n#tags: [\"service-X\", \"web-tier\"]\ntags: [\"windows_systems\"]\n\n# Optional fields that you can specify to add additional information to the\n# output.\n#fields:\n#  env: staging\n...<\/code><\/pre>\n\n\n\n
Next, setup the Winlogbeat output. In this demo, we are sending the logs directly to Elasticsearch nodes.<\/p>\n\n\n\n
...\n#-------------------------- Elasticsearch output ------------------------------\noutput.elasticsearch:\n  # Array of hosts to connect to.\n  #hosts: [\"localhost:9200\"]\n  hosts: [\"192.168.43.104:9200\", \"192.168.43.105:9200\", \"192.168.43.106:9200\"]\n\n  # Optional protocol and basic auth credentials.\n  #protocol: \"https\"\n  #username: \"elastic\"\n  #password: \"changeme\"\n...<\/code><\/pre>\n\n\n\n
If Elasticsearch and Kibana are not running on the same host and you want to use Kibana Winlogbeat dashboards, you can specify Kibana URL. Kibana must be reachable on non-loopback address. For example;<\/p>\n\n\n\n
...\nsetup.kibana:\n\n  # Kibana Host\n  # Scheme and port can be left out and will be set to the default (http and 5601)\n  # In case you specify and additional path, the scheme is required: http:\/\/localhost:5601\/path\n  # IPv6 addresses should always be defined as: https:\/\/[2001:db8::1]:5601\n  #host: \"localhost:5601\"\n  host: \"192.168.43.104:5601\"\n...<\/code><\/pre>\n\n\n\n
Also, if you are using Logstash, you can comment out the Elasticsearch output and specify Logstash connection addresses.<\/p>\n\n\n\n
...\n#-------------------------- Elasticsearch output ------------------------------\n#output.elasticsearch:\n  # Array of hosts to connect to.\n  #hosts: [\"localhost:9200\"]\n  #hosts: [\"192.168.43.104:9200\", \"192.168.43.105:9200\", \"192.168.43.106:9200\"]\n\n  # Optional protocol and basic auth credentials.\n  #protocol: \"https\"\n  #username: \"elastic\"\n  #password: \"changeme\"\n\n#----------------------------- Logstash output --------------------------------\noutput.logstash:\n  # The Logstash hosts\n  #hosts: [\"localhost:5044\"]\n  hosts: [\"192.168.43.104:5044\"]\n...<\/code><\/pre>\n\n\n\n
Winlogbeat configuration checks<\/h3>\n\n\n\n
Once done with configuration, save the file and run the configuration checks. To run the configuration checks, open Powershell as an administrator and execute the command below;<\/p>\n\n\n\n
cd C:\\'Program Files'\\Winlogbeat\n.\\winlogbeat.exe test config -c .\\winlogbeat.yml -e<\/code><\/pre>\n\n\n\n
If there is no error with the configuration, you should see the Config Ok<\/strong>.<\/p>\n\n\n\n
...\n2019-07-15T14:05:08.164+0300    INFO    instance\/beat.go:292    Setup Beat: winlogbeat; Version: 7.2.0\n2019-07-15T14:05:08.192+0300    INFO    [index-management]      idxmgmt\/std.go:178      Set output.elasticsearch.index t\no 'winlogbeat-7.2.0' as ILM is enabled.\n2019-07-15T14:05:08.225+0300    INFO    elasticsearch\/client.go:166     Elasticsearch url: http:\/\/192.168.43.104:9200\n2019-07-15T14:05:08.258+0300    INFO    elasticsearch\/client.go:166     Elasticsearch url: http:\/\/192.168.43.105:9200\n2019-07-15T14:05:08.290+0300    INFO    elasticsearch\/client.go:166     Elasticsearch url: http:\/\/192.168.43.106:9200\n2019-07-15T14:05:08.325+0300    INFO    [publisher]     pipeline\/module.go:97   Beat name: winlogbeat\n2019-07-15T14:05:08.356+0300    INFO    beater\/winlogbeat.go:69 State will be read from and persisted to C:\\Program Files\\Winlogbeat\\data\\.winlogbeat.yml\nConfig OK<\/code><\/pre>\n\n\n\n
Load Elasticsearch templates<\/h3>\n\n\n\n
If Winlogbeat has a direction connection and is using Elasticearch as the output, it will automatically load the template. However, if you are using Logstash as the output, you need to manually load the Elasticsearch template. See example command to load the Elasticsearch template manually below;<\/p>\n\n\n\n
cd C:\\'Program Files'\\Winlogbeat\n.\\winlogbeat.exe setup --index-management -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=[\"192.168.43.104:9200\"]'<\/code><\/pre>\n\n\n\n
Setup Kibana Dashboards<\/h3>\n\n\n\n
To load Winlogbeat default visualization dashboards, you need to have created the index pattern. Hence, navigate to Kibana and create the Winlogbeat index pattern.<\/p>\n\n\n\n
\"create<\/a><\/figure>\n\n\n\n
Next, if you are using Elasticsearch as your output, you can load the dashboards by running the setup<\/strong> command or enabling dashboard loading in the winlogbeat.yml<\/strong> (setup.dashboards.enabled: true<\/strong>) configuration.<\/p>\n\n\n\n
cd C:\\'Program Files'\\Winlogbeat>\n.\\winlogbeat.exe setup --dashboards<\/code><\/pre>\n\n\n\n
Loading dashboards (Kibana must be running and reachable)\nLoaded dashboards<\/code><\/pre>\n\n\n\n
OR<\/p>\n\n\n\n
#============================== Dashboards =====================================\n# These settings control loading the sample dashboards to the Kibana index. Loading\n# the dashboards is disabled by default and can be enabled either by setting the\n# options here or by using the `setup` command.\n#setup.dashboards.enabled: false\nsetup.dashboards.enabled: true<\/code><\/pre>\n\n\n\n
If you are using Logstash as the output, run the command below to load the dashboards.<\/p>\n\n\n\n
cd C:\\'Program Files'\\Winlogbeat>\n.\\winlogbeat.exe setup -e -E output.logstash.enabled=false -E output.elasticsearch.hosts=['192.168.43.104:9200']<\/code><\/pre>\n\n\n\n
Running Winlogbeat<\/h3>\n\n\n\n
To start or stop Winlogbeat, navigate to install directory and execute the commands below respectively.<\/p>\n\n\n\n
PS C:\\Program Files\\Winlogbeat> Start-Service winlogbeat<\/code><\/pre>\n\n\n\n
PS C:\\Program Files\\Winlogbeat> Stop-Service winlogbeat<\/code><\/pre>\n\n\n\n
You can also manage the Winlogbeat from system services.<\/p>\n\n\n\n
View Winlogbeat Kibana Dashboard<\/h3>\n\n\n\n
When Winlogbeat runs, you should now have the events and the dashboards.<\/p>\n\n\n\n
\"send<\/a><\/figure>\n\n\n\n
You can view Winlogbeat dashboards by navigating to Dashboards.<\/p>\n\n\n\n
\"Winlogbeat<\/a><\/figure>\n\n\n\n
There you go. You can also check Winlogbeat events on Kibana SIEM.<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
That is all on how to send Windows logs to Elastic Stack using Winlogbeat and Sysmon. You can continue to explore Kibana SIEM and Winlogbeat dashboards to analyze your windows events. Enjoy.<\/p>\n\n\n\n
Related Tutorials:<\/h3>\n\n\n\n
Setup Multi-node Elasticsearch 7.x Cluster on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n
Install and Configure Elastic Auditbeat on Ubuntu 18.04<\/a><\/p>\n\n\n\n
Install Filebeat on Fedora 30\/Fedora 29\/CentOS 7<\/a><\/p>\n\n\n\n
How to Debug Logstash Grok Filters<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
In this guide, we are going to learn how to send Windows logs to Elastic Stack using Winlogbeat and Sysmon. Winlogbeat is an Elastic Beat<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[910,121,72],"tags":[923,912,913,1036,1035],"class_list":["post-3533","post","type-post","status-publish","format-standard","hentry","category-elastic-stack","category-howtos","category-monitoring","tag-beats","tag-elastic-stack","tag-elk","tag-sysmon","tag-winlogbeat","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/3533"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=3533"}],"version-history":[{"count":18,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/3533\/revisions"}],"predecessor-version":[{"id":4984,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/3533\/revisions\/4984"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=3533"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=3533"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=3533"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}