You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
emit_invalid_record_to_error in the parser filter is a setting which allows users to decide on the outcome that happens when a record is unable to parsed for one of three reasons.
If set to true it says that the record will be given the @ERROR label, and "If you want to ignore these errors, set false".
Notably, the source code clearly has an extra condition that if reserve_data is true then it will not discard these errors (as I would of expected), but instead it allows them to pass but with an unparsed record. I fully understand why this behaviour exists, but I think the documentation around this could be included.
I would expect the documentation for emit_invalid_record_to_error to clearly say that if reserve_data is set to true, then the record will not be discarded but will instead be passed through without any parsing happening.
Additional context
An example configuration which was not clear from the documentation is:
Describe the bug
emit_invalid_record_to_error
in the parser filter is a setting which allows users to decide on the outcome that happens when a record is unable to parsed for one of three reasons.If set to true it says that the record will be given the
@ERROR
label, and "If you want to ignore these errors, set false".However, I was seeing different behaviour, and looking at the relevant piece of source code https://github.com/fluent/fluentd/blob/c48ca04a593f2f4628c545d74c90f6caba513bee/lib/fluent/plugin/filter_parser.rb#L72-L95
it became clear that the documentation isn't quite clear.
Notably, the source code clearly has an extra condition that if
reserve_data
is true then it will not discard these errors (as I would of expected), but instead it allows them to pass but with an unparsed record. I fully understand why this behaviour exists, but I think the documentation around this could be included.Link to the problematic documentation
https://docs.fluentd.org/filter/parser#emit_invalid_record_to_error
https://docs.fluentd.org/filter/parser#reserve_data
Expected explanation
I would expect the documentation for
emit_invalid_record_to_error
to clearly say that ifreserve_data
is set to true, then the record will not be discarded but will instead be passed through without any parsing happening.Additional context
An example configuration which was not clear from the documentation is:
I am happy to open a PR myself, but given this is my first open-source contribution I figured I would put an issue in.
The text was updated successfully, but these errors were encountered: