{"id":93,"date":"2015-03-30T04:21:48","date_gmt":"2015-03-30T04:21:48","guid":{"rendered":"http:\/\/svops.com\/blog\/?p=93"},"modified":"2015-05-20T16:50:22","modified_gmt":"2015-05-20T16:50:22","slug":"processing-common-event-information-with-grok","status":"publish","type":"post","link":"http:\/\/svops.com\/blog\/processing-common-event-information-with-grok\/","title":{"rendered":"Processing common event information with grok{}"},"content":{"rendered":"<p>If you&#8217;re lucky, most of your log messages in a given input will arrive in a standard format, typically with a set of common fields at the front (date, time, server, etc).<\/p>\n<p>Rather than multiple grok{} patterns that are looking across the entire message, like these:<\/p>\n<pre>grok {\r\n    match =&gt; [\"message\", \"%{SYSLOGTIMESTAMP:syslogtime} %{HOSTNAME:sysloghost} Save this %{WORD:word1}\"]\r\n    tag_on_failure =&gt; [\"_grokparsefailure_match1\"]\r\n}\r\n\r\ngrok {\r\n    match =&gt; [\"message\", \"%{SYSLOGTIMESTAMP:syslogtime} %{HOSTNAME:sysloghost} Save this other %{WORD:word2}\"]\r\n    tag_on_failure =&gt; [\"_grokparsefailure_match2\"]\r\n}<\/pre>\n<p>I like to split off the common stuff:<\/p>\n<pre>grok {\r\n    match =&gt; [\"message\", \"%{SYSLOGTIMESTAMP:syslogtime} %{HOSTNAME:sysloghost} %{GREEDYDATA:message}\"]\r\n    overwrite =&gt; [ \"message\" ]\r\n    tag_on_failure =&gt; [\"_grokparsefailure_syslog\"]\r\n}\r\n<\/pre>\n<p>Note that the last pattern puts the results into the field &#8220;message&#8221;. \u00a0Since that field already exists, we have to use the &#8220;overwrite&#8221; setting to update it.<\/p>\n<p>Then use smaller patterns against this smaller &#8220;message&#8221; for your application specific info:<\/p>\n<pre>grok {\r\n    match =&gt; [\"message\", \"Save this %{WORD:word1}\"]\r\n    tag_on_failure =&gt; [\"_grokparsefailure_match1\"]\r\n}\r\n<\/pre>\n<p>This is easier to read, and the later grok{}s will be running smaller regexps<br \/>\nagainst smaller input, which should be faster.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you&#8217;re lucky, most of your log messages in a given input will arrive in a standard format, typically with a set of common fields at the front (date, time, server, etc). Rather than multiple grok{} patterns that are looking &hellip; <a href=\"http:\/\/svops.com\/blog\/processing-common-event-information-with-grok\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[14],"tags":[],"_links":{"self":[{"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/posts\/93"}],"collection":[{"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/comments?post=93"}],"version-history":[{"count":4,"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/posts\/93\/revisions"}],"predecessor-version":[{"id":118,"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/posts\/93\/revisions\/118"}],"wp:attachment":[{"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/media?parent=93"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/categories?post=93"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/svops.com\/blog\/wp-json\/wp\/v2\/tags?post=93"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}