Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error events take too much time to emit when fluentd is not available #167

Open
Shantanu35 opened this issue Feb 24, 2020 · 2 comments
Open

Comments

@Shantanu35
Copy link

Hey,
We are using your library to send logs from our lambda functions to fluentd via Bunyan Logger.

// Instance style fluent-logger creation
const sender = require('fluent-logger').createFluentSender('tag-prefix', {
            enableReconnect: false,
            host: 'localhost',
            port: 24224,
            reconnectInterval: 3000,
            requireAckResponse: true,
            timeout: 3.0
        });

// Listener for detecting error events
sender.on('error', (error) => {
      console.log('Error found on stream here. ' + error);
      // Printing the logs on stdout using the buffer.
});

// bunyan logger instance for sending logs to fluentd.
logger = bunyan.createLogger({
            name: 'FluentdLogger',
            streams: [{ stream: sender.toStream('fluentd') }]
        });

logger.info('this log record is sent to fluent daemon');
  1. Using the above mentioned approach, the logs are successfully being sent to fluentd from lambda.
  2. However, when fluentd is not available, then the expected behavior is to catch the error event during the creation of bunyan instance and send the logs to stdout.

However, we have noticed that it takes a significant amount of time for the error event to be emitted when fluentd is not available, before which the lambda is completing its execution thus not reaching the error handler.

setTimeout(()=> sender.end(), 5000);

The above snippet seems to not work for me.

Therefore, is there any way to:

  1. Avoid delaying the lambda execution in the case when fluentd is not available and error event needs to be caught?
  2. If not, then what is the best way to perform a timeout so as to make the lambda quit after the error listener code has been executed?

It would be great if anyone could advise me on it.

@okkez
Copy link
Contributor

okkez commented May 30, 2020

Why you set timeout: 3.0 ?

Set the socket to timetout after timeout milliseconds of inactivity on the socket.

This library will reconnect to fluentd if fluentd is not available.
If you want to shutdown your Lambda function when Fluentd is down, you can throw an error to re-run your Lambda.

Please check the documentation for AWS Lambda about error handling and retry.

@Shantanu35
Copy link
Author

timeout: 3.0 is the default timeout and as far as I know, it just indicates the amount of time after which socket will timeout.
I have set enableReconnect: false, so that it does not try reconnecting.

The aim is that, if fluentd is not available, then the 'error' event emitted takes a significant amount of time, which causes the lambda to wait for it.
Is there any efficient or faster way to send error events from the library in case fluentd is down?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants