Client timeout in Producer longevity test

[tjtelan]

This test run failed due to a client error. The test was trying to produce messages at a fixed rate against a k8 cluster in CI.

It is possible that this is stress from the low-resource github runner. However, I do hit this in my local dev environment after a few minutes.

 + echo 1629480679,9/10,338/600,Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Error: 
   0: Fluvio client error
   1: Fluvio socket error
   2: time out in serial: 0 request: 1

[sehz]

Lets's re-test after #1453 and #1454 has been fixed

[sehz]

hours tests passes now. @tjtelan can you close after testing this?

[tjtelan]

The hourly tests have been running for more than 24 hours now, and their pass rate has significantly increased.

I think we may still see this issue in high CPU load environments like in the github actions hosted runners, but this looks practically fixed.

[tjtelan]

I'm able to reproduce this locally if I run the test for long enough. This is the error I got after attempting to run a 10 min test. Failed after 9.5 minutes.

+ echo 1630431825,1/20,570/600,oKUdUqV2ItEW9beukZfXaypHS41zEVAiUAo9nOT03F4rukXJ85nCmwTzfg83XSYWsZ4IN8H5Xz7j9Myd7cTjKRen5pGGsHlrNBlNayGZ3joZ17bvNnnB7JnlqD03AZa4AeFhiMDlMoE6tpSgFzLpWpTD3fUFX35PvIXfPXDnHFZJwGRbF3uzn5yuyVanjBkPTY8N52UCuMmfWnXSCQjTdLZjWTngRBiRpA7c8IUVX5ijPDF8Jnxh3gACXmiwSw93VNHffZb6Y7yIL0wltXcBSO4XIcu8MMTclRYTYfgVOgTDxQ8PPVlI5s1XeygGFXMxNVs8otUCEYc2oFDUUvjkXaZ7WWKdt29KG9muymvNKQOQ1cR6KUWKUg814UNsDSoCWN4PBCYeJAGOJYJBIc3rJZ8UvYLMpFN6hTJA5AqICqrrX2Kvke63uK9aJASWGvA5ve31ggMVnZ7LQ6eP3ycPMJZqs633jwHfLHnSqpA1h9lAEGrVQuCnvwPmYqAxJlNWjTdVrDBYXcbiL7rYjghdVseGDe2Nn0afOivsqM8p6g4nV8zHP57dkmGw4PnHw3VJECL6lqASeNSxs5fNndLYr3vXcH0rk2Vhtdsns6KhTOx9Vk1QjU3LNviYNBLRJlkYop6IlKCDnkskGWMHHKDeAZpSMLQMajqVY7EKF4KuLlqhZj2EM6dlGjKi49SDUwbMsnzKwCBOBmFXQBt1qS9edmeFkvF3LzXFKd50bft9iWgGxAx0p5bDvyXBHbhKKJOYFk2Tpf65T2jb5IUwzT7rczXZgApV0HiW7zeKNF1LW9kyX7UbxFd4yjbf0TEeTKMZYZj7geDeZ5o0lKeP3T1fGDu86gh8MkdGCCMrGi9bbaGqv0k8towAfKWhyqddll8TtXo4eo5MnkLGQ3iIIkthtoGl5x18OAC7mM2KVgr62DvkdubPahi8UYoxyQnTAQrnJDBalC9VP3qEFMrlXe5O9oJlagkQX9zXY2oHKqoW
+ /home/telant/.fluvio/bin/fluvio produce longevity-existing
Error: 
   0: Fluvio client error
   1: Fluvio socket error
   2: Connection refused (os error 111)

[ajhunyady]

This is likely related to #1526

• once there is a fix, we should re-test both.

[sehz]

longevity test should use develop version of fluvio if not run in CI

[tjtelan]

Maybe #1476 (comment) is a different issue than #1476 (comment). Perhaps I should reopen a separate issue.

The error messages are different