{"id":2385,"date":"2019-03-16T19:45:31","date_gmt":"2019-03-16T16:45:31","guid":{"rendered":"http:\/\/kifarunix.com\/?p=2385"},"modified":"2024-03-11T22:29:56","modified_gmt":"2024-03-11T19:29:56","slug":"configure-nxlog-to-forward-system-logs-to-rsyslog-server-on-ubuntu-18-04","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/configure-nxlog-to-forward-system-logs-to-rsyslog-server-on-ubuntu-18-04\/","title":{"rendered":"Configure NXLog to Forward System Logs to Rsyslog Server on Ubuntu"},"content":{"rendered":"\n<p>In this guide, we are going to learn how to configure NXLog to forward system logs to Rsyslog server on Ubuntu. There are various NXLog log collection solutions. In this guide, we are going to configure the opensource version of NXLog.<\/p>\n\n\n\n<div class=\"wp-block-rank-math-toc-block\" id=\"rank-math-toc\"><h2>Table of Contents<\/h2><nav><ul><li><a href=\"#configuring-nx-log-to-forward-to-rsyslog-server\">Configuring NXLog to Forward to Rsyslog Server<\/a><ul><li><a href=\"#configure-rsyslog-server\">Configure Rsyslog Server<\/a><\/li><li><a href=\"#install-nx-log-ce-on-ubuntu\">Install NXLog CE on Ubuntu<\/a><\/li><li><a href=\"#configure-nx-log-to-forward-logs-on-ubuntu\">Configure NXLog to Forward Logs on Ubuntu<\/a><\/li><li><a href=\"#define-the-global-directives\">Define the Global directives<\/a><\/li><li><a href=\"#define-input-module-directives\">Define Input Module Directives<\/a><\/li><li><a href=\"#define-the-processor-modules\">Define the Processor Modules<\/a><\/li><li><a href=\"#define-the-output-module\">Define the Output Module<\/a><\/li><li><a href=\"#define-the-route-directives\">Define the Route Directives<\/a><\/li><li><a href=\"#related-tutorials\">Related Tutorials<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"configuring-nx-log-to-forward-to-rsyslog-server\">Configuring NXLog to Forward to Rsyslog Server<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"configure-rsyslog-server\">Configure Rsyslog Server<\/h3>\n\n\n\n<p>You can check our previous articles on configuration of Rsyslog and Syslog by following the links below;<\/p>\n\n\n\n<p><a title=\"Configure Rsyslog on Solaris 11.4 to Send logs to Remote Log Server\" href=\"https:\/\/kifarunix.com\/configure-rsyslog-on-solaris-11-4-to-send-logs-to-remote-log-server\/\" target=\"_blank\" rel=\"bookmark noopener noreferrer\">Configure Rsyslog on Solaris 11.4 to Send logs to Remote Log Server<\/a><\/p>\n\n\n\n<p><a title=\"Configure Syslog on Solaris 11.4 for Remote Logging\" href=\"https:\/\/kifarunix.com\/how-to-configure-syslog-to-send-logs-to-remote-syslog-server-on-solaris-11-4\/\" target=\"_blank\" rel=\"bookmark noopener noreferrer\">Configure Syslog on Solaris 11.4 for Remote Logging<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-setup-rsyslog-server-on-ubuntu\" target=\"_blank\" rel=\"noreferrer noopener\">How to Configure Remote Logging with Rsyslog on Ubuntu<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"install-nx-log-ce-on-ubuntu\">Install NXLog CE on Ubuntu<\/h3>\n\n\n\n<p>Check how to install NXLog CE on Ubuntu by following the link below;<\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/?s=install+nxlog+ce+on+ubuntu\">Install NXLog CE on Ubuntu<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"configure-nx-log-to-forward-logs-on-ubuntu\"><a href=\"#configurenxlogforwarding\">Configure NXLog to Forward Logs on Ubuntu<\/a><\/h3>\n\n\n\n<p>Now that NXLog CE has been installed, you need to configure it to forward logs to the remote Rsyslog server.<\/p>\n\n\n\n<p>The default configuration file for NXLog CE is <code>\/etc\/nxlog\/nxlog.conf<\/code>.<\/p>\n\n\n\n<p>This is how the default configuration file looks like;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/etc\/nxlog\/nxlog.conf<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>## This is a sample configuration file. See the nxlog reference manual about the\n## configuration options. It should be installed locally under\n## \/usr\/share\/doc\/nxlog-ce\/ and is also available online at\n## http:\/\/nxlog.org\/docs\n\n########################################\n# Global directives                    #\n########################################\nUser nxlog\nGroup nxlog\n\ninclude \/etc\/nxlog\/nxlog.d\/*.conf\nLogFile \/var\/log\/nxlog\/nxlog.log\nLogLevel INFO\n\n########################################\n# Modules                              #\n########################################\n&lt;Extension _syslog&gt;\n    Module      xm_syslog\n&lt;\/Extension&gt;\n\n&lt;Input in1&gt;\n    Module\tim_udp\n    Port\t514\n    Exec\tparse_syslog_bsd();\n&lt;\/Input&gt;\n\n&lt;Input in2&gt;\n    Module\tim_tcp\n    Port\t514\n&lt;\/Input&gt;\n\n&lt;Output fileout1&gt;\n    Module\tom_file\n    File\t\"\/var\/log\/nxlog\/logmsg.txt\"\n    Exec\tif $Message =~ \/error\/ $SeverityValue = syslog_severity_value(\"error\");\n    Exec\tto_syslog_bsd();\n&lt;\/Output&gt;\n\n&lt;Output fileout2&gt;\n    Module\tom_file\n    File\t\"\/var\/log\/nxlog\/logmsg2.txt\"\n&lt;\/Output&gt;\n\n########################################\n# Routes                               #\n########################################\n&lt;Route 1&gt;\n    Path\tin1 =&gt; fileout1\n&lt;\/Route&gt;\n\n&lt;Route tcproute&gt;\n    Path\tin2 =&gt; fileout2\n&lt;\/Route&gt;\n<\/code><\/pre>\n\n\n\n<p>NXLog can be configured to receive and read logs from different types of sources including;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>log data received over the network<\/li>\n\n\n\n<li>events stored in databases<\/li>\n\n\n\n<li>messages read from files<\/li>\n\n\n\n<li>data retrieved using executables<\/li>\n<\/ul>\n\n\n\n<p>This guide focuses on configuring NXLog CE to receive, read and forward logs from system log files to a remote logging server.<\/p>\n\n\n\n<p>Therefore, make a backup of the original configuration so that you can make any adjustments that suit your environment.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mv \/etc\/nxlog\/nxlog.conf{,.original}<\/code><\/pre>\n\n\n\n<p>Create a new configuration file.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>touch \/etc\/nxlog\/nxlog.conf<\/code><\/pre>\n\n\n\n<p>The NXLog configuration file consists of;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>global directives<\/code><\/li>\n\n\n\n<li><code>module instances<\/code>, and<\/li>\n\n\n\n<li><code>routes<\/code><\/li>\n<\/ul>\n\n\n\n<p>To begin with, set the ROOT to the main directory of NXLog configuration.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code># Set the NXLog main directory\ndefine ROOT \/etc\/nxlog<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"define-the-global-directives\">Define the Global directives<\/h3>\n\n\n\n<p>There are quite a number of global directives that can be set.<\/p>\n\n\n\n<p>However, in its simplest, we will define;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>the NXLog modules directory<\/li>\n\n\n\n<li>directory to write the cached data<\/li>\n\n\n\n<li>the logging level,<\/li>\n\n\n\n<li>NXLog PID file,<\/li>\n\n\n\n<li>NXLog working directory,<\/li>\n\n\n\n<li>NXLog log file etc<\/li>\n<\/ul>\n\n\n\n<p>This can be done by using the following directives respectively; <code>ModuleDir<\/code>, <code>CacheDir<\/code>, <code>LogLevel<\/code>, <code>PidFile<\/code>, <code>SpoolDir<\/code> respectively.<\/p>\n\n\n\n<pre class=\"scroll-box\"><code># Global Directives\nModuledir \/usr\/lib\/nxlog\/modules\nCacheDir %ROOT%\/data\nSpoolDir %ROOT%\/data\nPidfile \/tmp\/nxlog.pid\nLogFile \/var\/log\/nxlog\/nxlog.log\n<\/code><\/pre>\n\n\n\n<p>Some of the other important global directives include <code>User<\/code> and <code>Group<\/code>.<\/p>\n\n\n\n<p>NXLog runs as user <code>nxlog<\/code> by default with the limitation being this user cannot read the <code>\/var\/log<\/code> directory where most system logs are written to.<\/p>\n\n\n\n<p>To circumvent this, NXlog can be set to run as root by omitting the <code>User<\/code> option. However, it is more secure if you can add NXlog user to a group with permissions to read the log file.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"define-input-module-directives\">Define Input Module Directives<\/h3>\n\n\n<div class=\"gb-container gb-container-17525c71\">\n\n<p>Basically, we are going to define basic input modules directives that read various log files for sending to a remote log server.<\/p>\n\n<\/div>\n\n\n<pre class=\"scroll-box\"><code># Define Input Modules\n&lt;Input in1&gt;\n    Module      im_file\n    File        \"\/var\/log\/auth.log\"\n    SavePos TRUE\n    ReadFromLast TRUE\n&lt;\/Input&gt;\n&lt;Input in2&gt;\n    Module      im_file\n    File        \"\/var\/log\/syslog\"\n    SavePos TRUE\n    ReadFromLast TRUE\n&lt;\/Input&gt;\n<\/code><\/pre>\n\n\n\n<p>The <code>SavePos<\/code> directive ensures that log file state is cached when NXLog exits. The log file will be read from that position when NXLog starts. This can however be turned off by using <code>NoCache<\/code> directive.<\/p>\n\n\n\n<p>The <code>ReadFromLast<\/code> directive ensures that NXLog reads the logs received after NXLog started.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"define-the-processor-modules\">Define the Processor Modules<\/h3>\n\n\n\n<p>Processor modules are used process logs between the Input and Output modules. This can be achieved by use of <code>pm_buffer<\/code> modules which supports both <code>disk<\/code> and <code>memory<\/code> log buffering. In this guide, we are going to set disk buffering. You also need to set the maximum size of the logs that can be buffered.<\/p>\n\n\n\n<pre class=\"scroll-sz\"><code>#Define Processor Modules\n&lt;Processor buffer>\n   Module      pm_buffer\n   MaxSize    512000   # Buffer logs upto 512MB\n   Type         Disk   # Disk buffering\n&lt;\/Processor>\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"define-the-output-module\">Define the Output Module<\/h3>\n\n\n\n<p>There are different types of output modules. We are going to set UDP as our output module. You can check about other modules <a href=\"https:\/\/nxlog.co\/docs\/nxlog-ce\/nxlog-reference-manual.html#output-modules\" target=\"_blank\" rel=\"noopener noreferrer\">here<\/a>. You need to set the remote <code>Host<\/code> IP and <code>Port<\/code>.<\/p>\n\n\n\n<pre class=\"scroll-sz\"><code># Define Output Modules\n&lt;Output udp>\n    Module      om_udp\n    Host        192.168.43.208\n    Port        514\n&lt;\/Output>\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"define-the-route-directives\">Define the Route Directives<\/h3>\n\n\n\n<p>Define the data flow using the <code>Path<\/code> directive. More than one Input feeding logs into the route are comma separated. The list of Input modules is followed by an arrow (<code>=&gt;<\/code>). Processor modules or Output modules follow after. Multiple Processors are separated by arrows. The syntax is;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Path INPUT1&#91;, INPUT2...] =&gt; &#91;PROCESSOR1 &#91;=&gt; PROCESSOR2...] =&gt;] OUTPUT1&#91;, OUTPUT2...]<\/code><\/pre>\n\n\n\n<p>Hence, we can define this in our configuration file as;<\/p>\n\n\n\n<pre class=\"scroll-sz\"><code># Route definition\n&lt;Route 1>\n    Path      in1,in2 => buffer => udp\n&lt;\/Route>\n<\/code><\/pre>\n\n\n\n<p>That is all about our configuration in its simplest form.<\/p>\n\n\n\n<p>In general, it should look like;<\/p>\n\n\n\n<pre class=\"scroll-box\"><code># Set the NXLog main directory\ndefine ROOT \/etc\/nxlog\n# Global Directives\nModuledir \/usr\/lib\/nxlog\/modules\nCacheDir %ROOT%\/cache_dir\nSpoolDir %ROOT%\/spool_dir\nPidfile \/tmp\/nxlog.pid\nLogFile \/var\/log\/nxlog\/nxlog.log\n# Define Input Modules\n&lt;Input in1&gt;\n    Module      im_file\n    File        \"\/var\/log\/auth.log\"\n    SavePos TRUE\n    ReadFromLast TRUE\n&lt;\/Input&gt;\n&lt;Input in2&gt;\n    Module      im_file\n    File        \"\/var\/log\/syslog\"\n    SavePos TRUE\n    ReadFromLast TRUE\n&lt;\/Input&gt;\n#Define Processor Modules\n&lt;Processor buffer&gt;\n   Module      pm_buffer\n   MaxSize     512000\n   Type        Disk\n&lt;\/Processor&gt;\n# Define Output Modules\n&lt;Output udp&gt;\n    Module      om_udp\n    Host        0.0.0.0 # IP of Rsyslog Server\n    Port        514\n&lt;\/Output&gt;\n# Route Definition\n&lt;Route 1&gt;\n    Path      in1,in2 =&gt; buffer =&gt; udp\n&lt;\/Route&gt;\n<\/code><\/pre>\n\n\n\n<p>Create the Cache and Spool Directories.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>mkdir -p \/etc\/nxlog\/{cache_dir,spool_dir}<\/code><\/pre>\n\n\n\n<p>Check the configuration to verify the syntax;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>nxlog -v<\/code><\/pre>\n\n\n\n<p>If you get <strong>INFO configuration OK<\/strong>, you are good to proceed.<\/p>\n\n\n\n<p>Restart NXLog and set it to run on system boot.<\/p>\n\n\n\n<pre id=\"block-bf506d80-2e05-491e-b056-0f19fb69adb5\" class=\"wp-block-preformatted\">systemctl restart nxlog<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl enable nxlog<\/code><\/pre>\n\n\n\n<p>Check the status.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><code>systemctl status nxlog<\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>\u25cf nxlog.service - NXLog daemon\n     Loaded: loaded (\/lib\/systemd\/system\/nxlog.service; enabled; vendor preset: enabled)\n     Active: active (running) since Mon 2023-08-28 19:48:20 UTC; 29min ago\n    Process: 2692 ExecStartPre=\/usr\/bin\/nxlog -v (code=exited, status=0\/SUCCESS)\n   Main PID: 2693 (nxlog)\n      Tasks: 7 (limit: 2257)\n     Memory: 1.9M\n     CGroup: \/system.slice\/nxlog.service\n             \u2514\u25002693 \/usr\/bin\/nxlog -f\n\nAug 28 19:48:20 focal systemd[1]: Starting NXLog daemon...\nAug 28 19:48:20 focal nxlog[2692]: 2023-08-28 19:48:20 INFO configuration OK\nAug 28 19:48:20 focal systemd[1]: Started NXLog daemon.\n<\/code><\/pre>\n\n\n\n<p>Confirm that you can receive logs on the remote server.<\/p>\n\n\n\n<p>As a POC, ssh into Ubuntu 18.04 server with nxlog running from a different server.<\/p>\n\n\n\n<p>At the same time, tail the logs on remote Rsyslog server and there you go.<\/p>\n\n\n\n<pre id=\"block-1a6e8b77-ca6e-4870-9149-b9e2776a7430\" class=\"wp-block-preformatted\">tail -f \/var\/log\/remotelogs\/192.168.43.203.log <\/code><\/pre>\n\n\n\n<pre class=\"scroll-box\"><code>2022-10-16T19:32:40-04:00 u18svr sshd[21327]: Connection closed by 127.0.0.1 port 50630 [preauth]\n2022-10-16T19:34:12-04:00 u18svr sshd[21335]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.43.149  user=kifarunix\n2022-10-16T19:34:14-04:00 u18svr sshd[21335]: Failed password for kifarunix from 192.168.43.149 port 48224 ssh2\n2022-10-16T19:34:18-04:00 u18svr sshd[21335]: Accepted password for kifarunix from 192.168.43.149 port 48224 ssh2\n2022-10-16T19:34:18-04:00 u18svr sshd[21335]: pam_unix(sshd:session): session opened for user kifarunix by (uid=0)\n2022-10-16T19:34:18-04:00 u18svr systemd-logind[581]: New session 26 of user kifarunix.\n<\/code><\/pre>\n\n\n\n<p>Feel free to read more about NXLog on their <a href=\"https:\/\/nxlog.co\/docs\/nxlog-ce\/nxlog-reference-manual.html\" target=\"_blank\" rel=\"noopener noreferrer\">reference manual<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"related-tutorials\">Related Tutorials<\/h3>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/install-and-configure-nxlog-ce-on-ubuntu-20-04\/\" target=\"_blank\" rel=\"noopener noreferrer\" aria-current=\"page\">Install and Configure NXLog CE on Ubuntu 20.04<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/configure-rsyslog-on-solaris-11-4-to-send-logs-to-remote-log-server\/\" target=\"_blank\" rel=\"noreferrer noopener\">Configure Rsyslog on Solaris 11.4 to Send logs to Remote Log Server<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/how-to-configure-syslog-to-send-logs-to-remote-syslog-server-on-solaris-11-4\/\" target=\"_blank\" rel=\"noreferrer noopener\">Configure Syslog on Solaris 11.4 for Remote Logging<\/a><\/p>\n\n\n\n<p><a href=\"https:\/\/kifarunix.com\/how-to-configure-remote-logging-with-rsyslog-on-ubuntu-18-04\/\" target=\"_blank\" rel=\"noreferrer noopener\">How to Configure Remote Logging with Rsyslog on Ubuntu 18.04<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this guide, we are going to learn how to configure NXLog to forward system logs to Rsyslog server on Ubuntu. There are various NXLog<\/p>\n","protected":false},"author":1,"featured_media":6044,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[121,72,191,331,64],"tags":[5984,5983,1645,5982,1644,351,5985,186,192,67],"class_list":["post-2385","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-howtos","category-monitoring","category-rsyslog","category-syslog","category-ubuntu-18-04","tag-check-nxlog-configuration-syntax","tag-configure-nxlog-on-ubuntu","tag-forward-syslog-logs-using-nxlog","tag-install-nxlog-on-ubuntu","tag-install-nxlog-ubuntu-18-04","tag-nxlog","tag-nxlog-v","tag-rsyslog","tag-syslog","tag-ubuntu-18-04","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","resize-featured-image"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/2385"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=2385"}],"version-history":[{"count":11,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/2385\/revisions"}],"predecessor-version":[{"id":21126,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/2385\/revisions\/21126"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media\/6044"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=2385"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=2385"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=2385"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}