The multiline filter is designed to combine messages that span lines into a single event that can be easily processed with other logstash filters. \u00a0Systems that throw large exceptions (e.g. Java) are the standard use-case for this filter.<\/p>\n
At the most basic, you need to provide three pieces of information to the filter:<\/p>\n
When ‘negate’ is set to true, read it as “when my PATTERN doesn’t match, do WHAT”; when false, read it as “when my PATTERN does match, do WHAT”.<\/p>\n
In this example, ‘negate’ is true, so we read it as “when my timestamp pattern doesn’t match, keep the line with the previous entry”:<\/p>\n
filter {\r\n multiline {\r\n negate => 'true'\r\n pattern => \"^%{TIMESTAMP_ISO8601} \"\r\n what => 'previous'\r\n }\r\n}<\/pre>\nThis filter should be used first, so that other filters will see the single event.<\/p>\n
Until a new line matches the pattern, logstash is expecting more lines to join, so it won’t release the combined event. \u00a0There is an enable_flush option, but it should not be used in production. \u00a0In logstash version 1.5, the flush will be “production ready”.<\/p>\n
When using multiline, you cannot use multiple filter workers, as each worker would be reading a different line. \u00a0If you attempt this configuration, logstash will not start.<\/p>\n
If your application writes log entries in a way where they can overlap with each other, the basic filter can’t help you. \u00a0However, if your system prints a common string in each message (a UUID, etc), you can use that to combine messages. \u00a0See the ‘stream_identity’ option.<\/p>\n
You should also consider using the multiline{} codec, so that messages are combined in the input{} phase. \u00a0Note that the codec doesn’t offer the ‘stream_identity’ option.<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
The multiline filter is designed to combine messages that span lines into a single event that can be easily processed with other logstash filters. \u00a0Systems that throw large exceptions (e.g. Java) are the standard use-case for this filter. At the … Continue reading