{"id":993,"date":"2018-10-20T17:38:49","date_gmt":"2018-10-20T14:38:49","guid":{"rendered":"http:\/\/kifarunix.com\/?p=993"},"modified":"2024-03-11T20:04:17","modified_gmt":"2024-03-11T17:04:17","slug":"a-basic-introduction-to-rsyslog-filters","status":"publish","type":"post","link":"https:\/\/kifarunix.com\/a-basic-introduction-to-rsyslog-filters\/","title":{"rendered":"A Basic Introduction to Rsyslog Filters"},"content":{"rendered":"\n<p>Hello folks, welcome to our tutorial on a basic introduction to rsyslog filters. In this tutorial, we are going to learn different ways of filtering log messages on a system using rsyslog.<\/p>\n\n\n\n<p>Rsyslog is the default logging utility on most Linux systems. Its main configuration file is&nbsp;<strong>\/etc\/rsyslog.conf <\/strong>where <em><strong>global directives<\/strong><\/em>, <em><strong>modules<\/strong><\/em>, and <em><strong>rules<\/strong><\/em> that consist of <em><strong>filter<\/strong> <\/em>and <em><strong>action<\/strong><\/em> parts can be specified.<\/p>\n\n\n\n<p>A <em><strong>filter<\/strong><\/em> can be used to specify a rule applied on syslog messages while <em><strong>action<\/strong><\/em> specifies what to do with the selected messages.<\/p>\n\n\n\n<p><strong>Rsyslog<\/strong> offers different methods for filtering syslog messages;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Facility\/Priority-based filter method<\/li>\n\n\n\n<li>Property-based filter method<\/li>\n\n\n\n<li>Expression-based filter method<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Facility\/Priority-based method<\/h3>\n\n\n\n<p>This is the most common method of filtering messages on rsyslog. It filters messages based on <em><strong>facility <\/strong>and<strong> priority <\/strong><\/em>conditions. The facility priority condition is separated by a dot with the syntax:<\/p>\n\n\n\n<p><strong><em>FACILITY.PRIORITY ACTION<\/em><\/strong><\/p>\n\n\n\n<p>where:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><em>facility <\/em><\/strong>specifies type of program or tool that generated the message to be logged.<\/li>\n\n\n\n<li><strong><em>priority<\/em><\/strong> is the parameter for the importance of this message.<\/li>\n\n\n\n<li><strong><em>action<\/em><\/strong> is a file, remote computer, or other location that\u2019s to accept the message.<\/li>\n<\/ul>\n\n\n\n<p>The<strong> facility<\/strong> and <strong>priority<\/strong> are often referred to collectively as the <em><strong>selector<\/strong><\/em>.<\/p>\n\n\n\n<p>Valid parameters for the facility and their numerical codes are as follows:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>kern (0)<\/strong> &#8211; Kernel messages<\/li>\n\n\n\n<li><strong>user (1)<\/strong> &#8211; User-level messages<\/li>\n\n\n\n<li><strong>mail (2)<\/strong> &#8211; Mail system messages<\/li>\n\n\n\n<li><strong>daemon (3)<\/strong> &#8211; Various system daemon messages<\/li>\n\n\n\n<li><strong>auth (4)<\/strong> &#8211; Security and authorization messages<\/li>\n\n\n\n<li><strong>syslog (5)<\/strong> &#8211; Internally generated syslog messages<\/li>\n\n\n\n<li><strong>lpr (6)<\/strong> &#8211; Printing system messages<\/li>\n\n\n\n<li><strong>news (7)<\/strong> &#8211; News daemon messages<\/li>\n\n\n\n<li><strong>uucp (8)<\/strong> &#8211; Unix-to-Unix copy program (uucp) daemon messages<\/li>\n\n\n\n<li><strong>cron (9)<\/strong> &#8211; cron daemon messages<\/li>\n\n\n\n<li><strong>authpriv<\/strong> (10) -Security and authorization messages<\/li>\n\n\n\n<li><strong>ftp(11)<\/strong> &#8211; FTP daemon messages<\/li>\n\n\n\n<li><strong>ntp (12)<\/strong> &#8211; NTP messages<\/li>\n\n\n\n<li><strong>security (13)<\/strong> &#8211; log audit<\/li>\n\n\n\n<li><strong>console (14)<\/strong> &#8211; log alert<\/li>\n\n\n\n<li><strong>solaris-cron (15)<\/strong> &#8211; scheduling daemon<\/li>\n\n\n\n<li><strong>local0&nbsp;<\/strong>&#8211; <strong>local7<\/strong> &#8211; Locally defined application messages<\/li>\n<\/ul>\n\n\n\n<p>Note that the mapping between facility code and keyword is not uniform in different operating systems and syslog implementations.<\/p>\n\n\n\n<p>Valid parameters for priority, their key words and numerical codes are as follows:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Emergency (emerg, 0)<\/strong> &#8211; Panic messages indicating system is unusable (most important)<\/li>\n\n\n\n<li><strong>Alert (alert,1) <\/strong>&#8211; Indicates that action must be taken immediately<\/li>\n\n\n\n<li><strong>Critical (crit , 2) <\/strong>&#8211; Critical conditions<\/li>\n\n\n\n<li><strong>Error (err, 3)<\/strong> &#8211; Non-urgent failure and error messages<\/li>\n\n\n\n<li><strong>Warning (warning, 4)<\/strong> &#8211; Warning conditions<\/li>\n\n\n\n<li><strong>Notice (notice, 5)<\/strong> &#8211; Normal but significant conditions<\/li>\n\n\n\n<li><strong>Informational (info,6)<\/strong> &#8211; Informational messages<\/li>\n\n\n\n<li><strong>Debug (debug, 7)<\/strong> -Debug level messages<\/li>\n<\/ul>\n\n\n\n<p>Points to note:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Preceding any priority keyword with an <strong>equal sign (=)<\/strong>, selects syslog messages with the specified priority only. All other priorities will be ignored.<\/li>\n\n\n\n<li>Preceding a priority keyword with an <strong>exclamation mark (!)<\/strong> selects all syslog messages<br>except those with the defined priority.<\/li>\n\n\n\n<li>An <strong>asterisk (*)<\/strong> can be used to define all facilities or priorities.<\/li>\n\n\n\n<li>Specifying the priority keyword <strong>none<\/strong> serves for facilities with no given priorities.<\/li>\n\n\n\n<li>Both <strong>facility<\/strong> and <strong>priority<\/strong> conditions are <strong>case-insensitive<\/strong>.<\/li>\n\n\n\n<li>Multiple facilities and priorities can be defined by separating them with a <strong>comma (,)<\/strong>.<\/li>\n\n\n\n<li>multiple selectors can be defined on one line by separating them with a <strong>semi-colon (;)<\/strong>.<\/li>\n\n\n\n<li>Note that each selector in the selector field is capable of overwriting the preceding ones, which can exclude some priorities from the pattern<\/li>\n\n\n\n<li>Rsyslog logs messages with the <strong>defined or higher<\/strong> priority e.g if you define <strong>alert<\/strong> level, the system will log messages that are classified as <strong>alert<\/strong> or <strong>emerg<\/strong> but not messages of crit or below.<\/li>\n\n\n\n<li>In most cases, the <em><strong>action<\/strong> <\/em>parameter is a <em><strong>filename,<\/strong><\/em> typically in the \/var\/log directory tree e.g <em>messages<\/em>, <em>syslog<\/em>, and <em>secure<\/em> files.<\/li>\n\n\n\n<li>Remote logging location can be specified by preceding the remote machine with an <strong>at sign (@)<\/strong>.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Examples of Facility\/Priority-based Filters<\/h4>\n\n\n\n<p>The following examples can be defined in <strong>\/etc\/rsyslog.conf<\/strong><\/p>\n\n\n\n<p>Send critical kernel messages to remote.example.com<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">kern.crit @remote.example.com<\/pre>\n\n\n\n<p>Send mail related messages of all security level to \/var\/log\/mail<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">mail.* \/var\/log\/mail<\/pre>\n\n\n\n<p>Send all emergency-level messages to all users who are logged into the computer using text-mode tools.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">*.emerg   *<\/pre>\n\n\n\n<p>Send all kernel related messaged to \/var\/log\/kernel<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">kern.* \/var\/log\/kernel<\/pre>\n\n\n\n<p>Log kernel messages of <strong>emerg, alert, crtitical, warning, informational<\/strong> level but not error level.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">kern.info;kern.!err \/var\/log\/kernel-info<\/pre>\n\n\n\n<p>To select all cron syslog messages except those with the info or debug priority, set the<br>configuration in the following form:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">cron.!info,!debug<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\"><\/h3>\n\n\n\n<!--nextpage-->\n\n\n\n<h3 class=\"wp-block-heading\">Property-based filters<\/h3>\n\n\n\n<p>Property based filters allow you to filter syslog messages using syslog properties such as hostname, msg, timegenerated or sysylogtag. Specified property value can be compared to one of the compare-operations described below.<\/p>\n\n\n\n<p>Property-based filter must start with a colon (:). To define the filter, use the following syntax:<\/p>\n\n\n\n<p><code class=\"docutils literal\"><span class=\"pre\">:property,<\/span> <span class=\"pre\">[!]compare-operation,<\/span> <span class=\"pre\">\"value\"<\/span><\/code><\/p>\n\n\n\n<p>The<strong> optional<\/strong> exclamation point (!) negates the output of the compare-operation. Other Boolean operators are currently not supported in property-based filters.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Property-based Properties<\/h4>\n\n\n\n<p>Some of the commonly used rsyslog properties include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>msg &#8211; <\/strong>the MSG part of the message.<\/li>\n\n\n\n<li><strong>hostname &#8211; <\/strong>hostname from the message<\/li>\n\n\n\n<li><strong>source &#8211; <\/strong>alias for HOSTNAME<\/li>\n\n\n\n<li><strong>timegenerated &#8211; <\/strong>timestamp when the message was RECEIVED. Always in high resolution<\/li>\n\n\n\n<li><strong>fromhost &#8211; <\/strong>hostname of the system the message was received from.<\/li>\n\n\n\n<li><strong>fromhost-ip &#8211; <\/strong>The same as fromhost, but always as an IP address.<\/li>\n\n\n\n<li><strong>syslogtag- <\/strong>TAG from the message<\/li>\n\n\n\n<li><strong>programname &#8211; <\/strong>the \u201cstatic\u201d part of the tag, as defined by BSD syslogd. For example, when TAG is \u201cnamed[12345]\u201d, programname is \u201cnamed\u201d.<\/li>\n<\/ul>\n\n\n\n<p>For a comprehensive list and description&nbsp;all currently-supported properties, you can check <a href=\"https:\/\/www.rsyslog.com\/doc\/v8-stable\/configuration\/properties.html\" target=\"_blank\" rel=\"noopener\">ryslog properties<\/a>.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Property-based Compare-Operations<\/h4>\n\n\n\n<p>Some of the property-based compare operations include;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>contains &#8211; <\/strong>Checks if the string provided in value is contained in the property. There must be an exact match, wildcards are not supported. A case insensitive option is <strong>contains_i<\/strong>.<\/li>\n\n\n\n<li><strong>isequal &#8211; <\/strong>Compares the \u201cvalue\u201d string provided and the property contents. These two values must be exactly equal to match. isequal is most useful for fields like syslogtag or FROMHOST, where you probably know the exact contents.<\/li>\n\n\n\n<li><strong>startswith &#8211; <\/strong>Checks if the value is found exactly at the beginning of the property value. For example, if you search for \u201cval\u201d with <code class=\"docutils literal\"><span class=\"pre\">:msg,<\/span> <span class=\"pre\">startswith,<\/span> <span class=\"pre\">\"val\"<\/span><\/code> it will be a match if msg contains \u201cvalues are in this message\u201d but it won\u2019t match if the msg contains \u201cThere are values in this message\u201d. To perform case insensitive comparisons, use <em><strong>startswith_i<\/strong><\/em>.<\/li>\n\n\n\n<li><strong>regex &#8211; <\/strong>Compares the property against the provided POSIX BRE regular expression.<\/li>\n\n\n\n<li><strong>ereregex &#8211; <\/strong>Compares the property against the provided POSIX ERE regular expression.<\/li>\n\n\n\n<li><strong>isempty<\/strong> &#8211; Checks if the property is empty.<\/li>\n<\/ul>\n\n\n\n<p>To select syslog messages which contain the string error in their message text;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">:msg, contains, \"error\"<\/pre>\n\n\n\n<p>The following filter selects syslog messages received from the host name test.example.com:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">:hostname, isequal, \"test.example.com\"<\/pre>\n\n\n\n<p>To write all messages including messages with string error to a log file,&nbsp;<span class=\"o\">\/<\/span><span class=\"n\">var<\/span><span class=\"o\">\/<\/span><span class=\"n\">log<\/span><span class=\"o\">\/<\/span><span class=\"n\">all-msgs-<\/span><span class=\"o\">with-error<\/span><span class=\"o\">.<\/span><span class=\"n\">log<\/span> and then write all messages without the string <strong>error<\/strong> to a<span class=\"n\">all-msgs-<\/span><span class=\"o\">with-error<\/span><span class=\"o\">.<\/span><span class=\"n\">log<\/span><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"><span class=\"o\">*.*<\/span> <span class=\"o\">\/<\/span><span class=\"n\">var<\/span><span class=\"o\">\/<\/span><span class=\"n\">log<\/span><span class=\"o\">\/<\/span><span class=\"n\">all-msgs-<\/span><span class=\"o\">with-error<\/span><span class=\"o\">.<\/span><span class=\"n\">log<\/span>\n<span class=\"p\">:<\/span><span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">contains<\/span><span class=\"p\">,<\/span> <span class=\"s2\">\"error\"<\/span>&nbsp; <span class=\"o\">~<\/span>\n<span class=\"o\">*.*<\/span> <span class=\"o\">\/<\/span><span class=\"n\">var<\/span><span class=\"o\">\/<\/span><span class=\"n\">log<\/span><span class=\"o\">\/<\/span><span class=\"n\">all-msgs-<span class=\"o\">without-error<\/span><span class=\"o\">.<\/span>log<\/span><\/pre>\n\n\n\n<p>The tilde on line 2 discards any message with string <strong>error.<\/strong><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Expression-based filters<\/h3>\n\n\n\n<p>Expression based filters allow filtering on arbitrary complex expressions, which can include boolean, arithmetic and string operations.<\/p>\n\n\n\n<p>These filters use rsyslog&#8217;s own scripting language called RainerScript to build complex filters.<\/p>\n\n\n\n<p>The basic syntax of expression-based filter looks as follows:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">if EXPRESSION then ACTION else ACTION<\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u201c<strong>if<\/strong>\u201d and \u201c<strong>then<\/strong>\u201d are fixed keywords that mus be present.<\/li>\n\n\n\n<li>The <strong>EXPRESSION<\/strong> attribute represents an expression to be evaluated, for example: <strong>$msg <\/strong><strong>startswith &#8216;DEVNAME&#8217;<\/strong> or <strong>$syslogfacility-text == &#8216;local0&#8217;<\/strong>.<br>You can specify more than one expression in a single filter by using and and or operators.<\/li>\n\n\n\n<li>The <strong>ACTION<\/strong> attribute represents an action to be performed if the expression returns the<br>value true. This can be a single action, or an arbitrary complex script enclosed in curly<br>braces.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Expression-based Filters Examples<\/h4>\n\n\n\n<p>To log messages that contains an string error in a log file called \/var\/log\/errors.log;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">if $msg contains 'error' then \/var\/log\/errors.log<\/pre>\n\n\n\n<p>To split&nbsp; the log file created by <strong>progA <\/strong>into two files based on the presence of the &#8220;<strong>informational<\/strong>&#8221; string in the messages;<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">if $programname == 'progA' then {\n   action(type=\"omfile\" file=\"\/var\/log\/progA.log\")\n   if $msg contains 'informational' then\n     action(type=\"omfile\" file=\"\/var\/log\/progAinfo.log\")\n   else\n     action(type=\"omfile\" file=\"\/var\/log\/progAnoinfo.log\")\n}<\/pre>\n\n\n\n<p>That is all about rsyslog filters. Hope you found this helpful.<\/p>\n\n\n\n<p>In our next tutorial, we will learn about introduction to rsyslog actions and templates configuration.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hello folks, welcome to our tutorial on a basic introduction to rsyslog filters. In this tutorial, we are going to learn different ways of filtering<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[121,191],"tags":[186,190,188,187,189],"class_list":["post-993","post","type-post","status-publish","format-standard","hentry","category-howtos","category-rsyslog","tag-rsyslog","tag-rsyslog-actions","tag-rsyslog-facility","tag-rsyslog-filters","tag-rsyslog-priority","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50"],"_links":{"self":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/993"}],"collection":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/comments?post=993"}],"version-history":[{"count":9,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/993\/revisions"}],"predecessor-version":[{"id":21033,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/posts\/993\/revisions\/21033"}],"wp:attachment":[{"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/media?parent=993"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/categories?post=993"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kifarunix.com\/wp-json\/wp\/v2\/tags?post=993"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}