{"id":2015,"date":"2022-12-07T23:12:51","date_gmt":"2022-12-07T23:12:51","guid":{"rendered":"https:\/\/linuxdigest.com\/?p=2015"},"modified":"2023-03-16T22:41:28","modified_gmt":"2023-03-16T22:41:28","slug":"counting-unique-results-from-grep-output","status":"publish","type":"post","link":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/","title":{"rendered":"Counting Unique Results from grep Output \ud83d\udd0e"},"content":{"rendered":"\n

As a Linux admin, you will probably find yourself running grep quite frequently. It is a quick way to find whatever you need from a text file. At least in my case, I probably use grep daily.<\/p>\n\n\n\n

Just finding the results is not always enough. Sometimes you also need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared in a log file.<\/p>\n\n\n\n

How do I count unique matches from grep?<\/h2>\n\n\n\n

There are actually two small programs that I would use in this scenario. They are sort and uniq. Both of them are a part of the GNU coreutils package and should be installed by default on any Linux distribution. Sort sorts the lines alphabetically, and uniq finds repetitions in the output.<\/p>\n\n\n\n

Let’s try a simple example. I have made a small mock log file, that has some simple error messages. Here is a snippet from the file:<\/p>\n\n\n\n

...\nERROR: Error code 8\nERROR: Error code 16\nSome other output\nSome other output\nERROR: Error code 19\nERROR: Error code 32\nERROR: Error code 32\n\n...<\/pre>\n\n\n\n
Now we want to find all the errors in the file using grep. That should be simple enough.<\/p>\n\n\n\n
grep ERROR myfile.log<\/pre>\n\n\n\n
Now we should get only the error messages. But we want to know how often each of them appears.<\/p>\n\n\n\n
To do this, we first pass it through the sort command. Using the pipe character (|). This will sort all the lines and make sure each repetition appears with the lines that are like it.<\/p>\n\n\n\n
Next, we will do the actual counting with the uniq command. By default, the uniq command removes duplicates. It also has a helpful argument to count how many duplicates there were. This argument is -c.<\/p>\n\n\n\n
So the full command will be:<\/p>\n\n\n\n
grep ERROR myfile.log | sort | uniq -c<\/pre>\n\n\n\n
Running this will show us each message only once with the count prepended to the line.<\/p>\n\n\n\n
      6 ERROR: Error code 16\n      7 ERROR: Error code 19\n     17 ERROR: Error code 32\n      5 ERROR: Error code 8<\/pre>\n\n\n\n
This is very helpful in our situation since we can see that “error code 32” was the most common message. But it would help even more if we could see the results ordered. <\/p>\n\n\n\n
We can run sort again to achieve this. But this time, we will add two arguments. To ensure that it is sorting by numbers, we add n, and to show the results in descending order, we add r.<\/p>\n\n\n\n
grep ERROR myfile.txt | sort | uniq -c | sort -nr<\/pre>\n\n\n\n
The results are now ordered by frequency.<\/p>\n\n\n\n
     17 ERROR: Error code 32\n      7 ERROR: Error code 19\n      6 ERROR: Error code 16\n      5 ERROR: Error code 8<\/pre>\n\n\n\n
Count the total number of results from grep<\/h2>\n\n\n\n
In some cases, you might just want the total number of results. Not the number of unique results. The example we just went through might be a bit too convoluted for these cases.<\/p>\n\n\n\n
This time we could use the wc command. It is also a part of the coreutils package. <\/p>\n\n\n\n
The wc command will print the number of lines, words and bytes in the text you pass to it. In our example, we are only interested in the number of lines. So we will pass the -l argument.<\/p>\n\n\n\n
grep \"Error code 32\" myfile.txt | wc -l<\/pre>\n\n\n\n
This will output the number of times error code 32 appeared: 17<\/p>\n\n\n\n
How do I count matches within a line?<\/h2>\n\n\n\n
I’ll just add this as a bonus, as I just learned this myself. You can also grep for multiple occurrences within a line. Using the -o (–only-matching) option, grep will only output the exact match and continue searching the current line. Each match will be printed on a separate line.<\/p>\n\n\n\n
Let’s try this. In keeping with the error theme of the previous examples, the string is the word “error” printed multiple times.<\/p>\n\n\n\n
echo \"error error error error\" | grep -o error<\/pre>\n\n\n\n
The output should look like this:<\/p>\n\n\n\n
error\nerror\nerror\nerror<\/pre>\n\n\n\n
We can now pipe the output to wc to count the total number of results.<\/p>\n\n\n\n
echo \"error error error error\" | grep -o error | wc -l<\/pre>\n\n\n\n
Count unique results within a line<\/h2>\n\n\n\n
We can also count unique values in a similar way as we did in our first example with a regular expression. To do this we add the “-e” argument to the grep command. This regular expression is quite simple. All we do is add a space and the “[[:digit:]]” expression to search for entries that look like this: “error n”. Where n is a single digit:<\/p>\n\n\n\n
echo \"error 1 error 2 error 1 error 2\" | grep -o -e 'error [[:digit:]]<\/pre>\n\n\n\n
The output should look like this:<\/p>\n\n\n\n
error 1\nerror 2\nerror 1\nerror 2<\/pre>\n\n\n\n
Using our trusty utilities, sort and uniq, we can now count each unique result.<\/p>\n\n\n\n
echo \"error 1 error 2 error 1 error 2\" | grep -o -e 'error [[:digit:]]' | sort | uniq -c<\/pre>\n\n\n\n
From this one-liner we can see that there are two of each value.<\/p>\n\n\n\n
2 error 1\n2 error 2<\/pre>\n\n\n\n
That’s it for now. You should be able to your own counting now. Good luck!<\/p>\n","protected":false},"excerpt":{"rendered":"
As a Linux admin, you will probably find yourself running grep quite frequently. It is a…<\/p>\n","protected":false},"author":1,"featured_media":2018,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[19,28],"yoast_head":"\nCounting Unique Results from grep Output \ud83d\udd0e - Linux Digest<\/title>\n<meta name=\"description\" content=\"Sometimes you need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Counting Unique Results from grep Output \ud83d\udd0e - Linux Digest\" \/>\n<meta property=\"og:description\" content=\"Sometimes you need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\" \/>\n<meta property=\"og:site_name\" content=\"Linux Digest\" \/>\n<meta property=\"article:published_time\" content=\"2022-12-07T23:12:51+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-03-16T22:41:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/linuxdigest.com\/wp-content\/uploads\/2022\/12\/Counting-Unique-Results-from-gre.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"800\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"The Linux Digest Guy\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"The Linux Digest Guy\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\"},\"author\":{\"name\":\"The Linux Digest Guy\",\"@id\":\"https:\/\/linuxdigest.com\/#\/schema\/person\/29c97230aa94affab929a88c6a10adb2\"},\"headline\":\"Counting Unique Results from grep Output \ud83d\udd0e\",\"datePublished\":\"2022-12-07T23:12:51+00:00\",\"dateModified\":\"2023-03-16T22:41:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\"},\"wordCount\":699,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/linuxdigest.com\/#organization\"},\"keywords\":[\"Command line\",\"grep\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\",\"url\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\",\"name\":\"Counting Unique Results from grep Output \ud83d\udd0e - Linux Digest\",\"isPartOf\":{\"@id\":\"https:\/\/linuxdigest.com\/#website\"},\"datePublished\":\"2022-12-07T23:12:51+00:00\",\"dateModified\":\"2023-03-16T22:41:28+00:00\",\"description\":\"Sometimes you need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared.\",\"breadcrumb\":{\"@id\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/linuxdigest.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Counting Unique Results from grep Output \ud83d\udd0e\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/linuxdigest.com\/#website\",\"url\":\"https:\/\/linuxdigest.com\/\",\"name\":\"Linux Digest\",\"description\":\"Linux tutorials for everyone\",\"publisher\":{\"@id\":\"https:\/\/linuxdigest.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/linuxdigest.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/linuxdigest.com\/#organization\",\"name\":\"Linux Digest\",\"url\":\"https:\/\/linuxdigest.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/linuxdigest.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/linuxdigest.com\/wp-content\/uploads\/2019\/11\/logo1.png\",\"contentUrl\":\"https:\/\/linuxdigest.com\/wp-content\/uploads\/2019\/11\/logo1.png\",\"width\":1102,\"height\":170,\"caption\":\"Linux Digest\"},\"image\":{\"@id\":\"https:\/\/linuxdigest.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/linuxdigest.com\/#\/schema\/person\/29c97230aa94affab929a88c6a10adb2\",\"name\":\"The Linux Digest Guy\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/linuxdigest.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/ac6bcf745dec6961360ccf2d2711f26c?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/ac6bcf745dec6961360ccf2d2711f26c?s=96&d=mm&r=g\",\"caption\":\"The Linux Digest Guy\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Counting Unique Results from grep Output \ud83d\udd0e - Linux Digest","description":"Sometimes you need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/","og_locale":"en_US","og_type":"article","og_title":"Counting Unique Results from grep Output \ud83d\udd0e - Linux Digest","og_description":"Sometimes you need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared.","og_url":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/","og_site_name":"Linux Digest","article_published_time":"2022-12-07T23:12:51+00:00","article_modified_time":"2023-03-16T22:41:28+00:00","og_image":[{"width":1200,"height":800,"url":"https:\/\/linuxdigest.com\/wp-content\/uploads\/2022\/12\/Counting-Unique-Results-from-gre.jpg","type":"image\/jpeg"}],"author":"The Linux Digest Guy","twitter_card":"summary_large_image","twitter_misc":{"Written by":"The Linux Digest Guy","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#article","isPartOf":{"@id":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/"},"author":{"name":"The Linux Digest Guy","@id":"https:\/\/linuxdigest.com\/#\/schema\/person\/29c97230aa94affab929a88c6a10adb2"},"headline":"Counting Unique Results from grep Output \ud83d\udd0e","datePublished":"2022-12-07T23:12:51+00:00","dateModified":"2023-03-16T22:41:28+00:00","mainEntityOfPage":{"@id":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/"},"wordCount":699,"commentCount":0,"publisher":{"@id":"https:\/\/linuxdigest.com\/#organization"},"keywords":["Command line","grep"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/","url":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/","name":"Counting Unique Results from grep Output \ud83d\udd0e - Linux Digest","isPartOf":{"@id":"https:\/\/linuxdigest.com\/#website"},"datePublished":"2022-12-07T23:12:51+00:00","dateModified":"2023-03-16T22:41:28+00:00","description":"Sometimes you need to count unique values that grep returns. One example is when I need to find out how often a particular error has appeared.","breadcrumb":{"@id":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/linuxdigest.com\/howto\/counting-unique-results-from-grep-output\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/linuxdigest.com\/"},{"@type":"ListItem","position":2,"name":"Counting Unique Results from grep Output \ud83d\udd0e"}]},{"@type":"WebSite","@id":"https:\/\/linuxdigest.com\/#website","url":"https:\/\/linuxdigest.com\/","name":"Linux Digest","description":"Linux tutorials for everyone","publisher":{"@id":"https:\/\/linuxdigest.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/linuxdigest.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/linuxdigest.com\/#organization","name":"Linux Digest","url":"https:\/\/linuxdigest.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/linuxdigest.com\/#\/schema\/logo\/image\/","url":"https:\/\/linuxdigest.com\/wp-content\/uploads\/2019\/11\/logo1.png","contentUrl":"https:\/\/linuxdigest.com\/wp-content\/uploads\/2019\/11\/logo1.png","width":1102,"height":170,"caption":"Linux Digest"},"image":{"@id":"https:\/\/linuxdigest.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/linuxdigest.com\/#\/schema\/person\/29c97230aa94affab929a88c6a10adb2","name":"The Linux Digest Guy","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/linuxdigest.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/ac6bcf745dec6961360ccf2d2711f26c?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ac6bcf745dec6961360ccf2d2711f26c?s=96&d=mm&r=g","caption":"The Linux Digest Guy"}}]}},"_links":{"self":[{"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/posts\/2015"}],"collection":[{"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/comments?post=2015"}],"version-history":[{"count":3,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/posts\/2015\/revisions"}],"predecessor-version":[{"id":2069,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/posts\/2015\/revisions\/2069"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/media\/2018"}],"wp:attachment":[{"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/media?parent=2015"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/categories?post=2015"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/linuxdigest.com\/wp-json\/wp\/v2\/tags?post=2015"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}