Multiline parser not following the list order

[JaredTan95]

Bug Report

Describe the bug

defined by Multiline.parser in the INPUT plugin do not parse in the order indicated by the documentation.

https://docs.fluentbit.io/manual/pipeline/inputs/tail#multiline-and-containers-v1.8

For example, it will first try docker, and if docker does not match, it will then try cri

To Reproduce

Use the following input to parse the multi-line log generated by cri:

[INPUT]
    Name     tail
    Read_from_Head True
    Path     /var/lib/docker/containers/*.log
    Multiline.parser   docker,cri,go,python,java

• cri format logs as follows:

2024-11-29T14:51:04.509635495+08:00 stdout F 2024-11-29 14:51:04.509 56488331 [TID:N/A] [12762363e4d2d153] [INFO ] com.xxxxxx.domain.storage.handler.CommonHelpHandler - stmt mt rmsLastTime=1732850560244 orderTimeStamp=1732850586000 traceId=18486001256466054
2024-11-29T14:51:04.509639621+08:00 stdout F 2024-11-29 14:51:04.509 56488331 [TID:N/A] [12762363e4d2d153] [INFO ] com.xxxxxx.domain.storage.handler.CommonHelpHandler - stmt mt rmsLastTime=1732850560244 orderTimeStamp=1732850586000 traceId=18486001256466054
2024-11-29T14:51:04.509790301+08:00 stdout F 2024-11-29 14:51:04.509 56488331 [TID:N/A] [12762363e4d2d153] [ERROR] com.xxxxxx.domain.storage.handler.StockHandler：
2024-11-29T14:51:04.50979503+08:00 stdout F com.deepexi.util.extension.ApplicationException:
2024-11-29T14:51:04.509802084+08:00 stdout F 	at com.xxxxxx.domain.storage.handler.CommonHelpHandler.checkRmsTimesWithSTMT(CommonHelpHandler.java:1265)
2024-11-29T14:51:04.509804111+08:00 stdout F 	at com.xxxxxx.domain.storage.handler.StockUnLockHandler.lambda$unlockStockPrivateWithSTMT$6(StockUnLockHandler.java:198)
2024-11-29T14:51:04.509806406+08:00 stdout F 	at java.lang.Iterable.forEach(Iterable.java:75)
2024-11-29T14:51:04.509808497+08:00 stdout F 	at com.xxxxxx.domain.storage.handler.StockUnLockHandler.unlockStockPrivateWithSTMT(StockUnLockHandler.java:193)
2024-11-29T14:51:04.509810942+08:00 stdout F 	at com.xxxxxx.domain.storage.handler.StockHandler.unlockStockWithSTMT(StockHandler.java:779)
2024-11-29T14:51:04.509812971+08:00 stdout F 	at com.xxxxxx.domain.storage.handler.StockHandler$$FastClassBySpringCGLIB$$de12a81c.invoke(<generated>)
2024-11-29T14:51:04.509815268+08:00 stdout F 	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
2024-11-29T14:51:04.509817242+08:00 stdout F 	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:750)
2024-11-29T14:51:04.509819177+08:00 stdout F 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
2024-11-29T14:51:04.50982133+08:00 stdout F 	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
2024-11-29T14:51:04.509823378+08:00 stdout F 	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
2024-11-29T14:51:04.509825527+08:00 stdout F 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
2024-11-29T14:51:04.509827452+08:00 stdout F 	at com.baomidou.dynamic.datasource.aop.DynamicDataSourceAnnotationInterceptor.invoke(DynamicDataSourceAnnotationInterceptor.java:51)
2024-11-29T14:51:04.509829411+08:00 stdout F 	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
2024-11-29T14:51:04.509831293+08:00 stdout F 	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
2024-11-29T14:51:04.509833492+08:00 stdout F 	at com.xxxxxx.domain.storage.handler.StockHandler$$EnhancerBySpringCGLIB$$c4a2c4d5.unlockStockWithSTMT(<generated>)
2024-11-29T14:51:04.509835442+08:00 stdout F 	at com.xxxxxx.domain.storage.service.impl.StockChangeServiceImpl.unlockStockWithSTMTData(StockChangeServiceImpl.java:423)
2024-11-29T14:51:04.509837393+08:00 stdout F 	at com.xxxxxx.domain.storage.service.impl.StockChangeServiceImpl.unlockStockWithSTMTFor(StockChangeServiceImpl.java:386)
2024-11-29T14:51:04.509839379+08:00 stdout F 	at com.xxxxxx.domain.storage.service.impl.StockChangeServiceImpl.unlockStockWithSTMT(StockChangeServiceImpl.java:369)
2024-11-29T14:51:04.509841463+08:00 stdout F 	at com.xxxxxx.domain.storage.service.impl.StockChangeServiceImpl.lambda$doSyncDepotStock$1(StockChangeServiceImpl.java:336)
2024-11-29T14:51:04.509843342+08:00 stdout F 	at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:67)
2024-11-29T14:51:04.509845304+08:00 stdout F 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
2024-11-29T14:51:04.509847538+08:00 stdout F 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2024-11-29T14:51:04.50984966+08:00 stdout F 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2024-11-29T14:51:04.509851543+08:00 stdout F 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2024-11-29T14:51:04.509853454+08:00 stdout F 	at java.lang.Thread.run(Thread.java:750)
2024-11-29T14:51:04.513162444+08:00 stdout F 2024-11-29 14:51:04.513 56488335 [TID:N/A] [12762363e4d2d153] [ERROR] com.xxxxxx.domain.storage.service.impl.StockChangeServiceImpl - orderNo:678987654567,traceId:6a91d146b9ba4b749f3dc40145c6fc2b

When I ran fluentbit, I found that the data written to es did not combine multiple data such as ApplicationException into one document:

When I ran it again after removing go,python,java from muliline-parser(Keep only docker and cri), ApplicationException was merged into a single document:

[INPUT]
    Name     tail
    Read_from_Head True
    Path     /var/lib/docker/containers/*.log
    Multiline.parser   docker,cri

Expected behavior

The following configuration takes effect according to the document description:

[INPUT]
    Name     tail
    Read_from_Head True
    Path     /var/lib/docker/containers/*.log
    Multiline.parser   docker,cri,go,python,java

Screenshots

Your Environment

• Version used:

fluentbit 2.0.8 or fluentbit 3.2.4

[patrick-stephens]

No, the parsers are applied in order and the first one to apply is then used with no others tried.

You need a secondary filter to then parse the additional steps, e.g. https://docs.fluentbit.io/manual/pipeline/filters/multiline-stacktrace

[JaredTan95]

No, the parsers are applied in order and the first one to apply is then used with no others tried.

This is what makes me confused, when I add go,python,java after docker and cri, according to your statement, it will not affect the normal work of the cri parser, But in fact, the cri failed.

I don't feel the need to add extra filter