reread auth file

[ppbrown]

2023-04-06 19:53:20 +0000 [warn]: #0 failed to flush the buffer. retry_times=10 next_retry_time=2023-04-06 20:10:35 +0000 chunk="5f8b00246b400a668bd3d83a9bb6ea8b" error_class=Aws::TimestreamWrite::Errors::ExpiredTokenException error="The security token included in the request is expired"

It would be nice if the plugin attempted to re-read the credentials.
In this case, the old cred had been updated... but the plugin just kept to the old ones and failed writes, even though a usable one was sitting in the appropriate location.

[valerauko]

So you want to be able to change the credentials at runtime without restarting the process?

[ppbrown]

yup. i want to have a cron in the background that keeps replacing the cred before it expires every 6 hours. Meanwhile i want fluentd to keep running

…

On Thu, Apr 6, 2023 at 6:32 PM valerauko ***@***.***> wrote: So you want to be able to change the credentials at runtime without restarting the process? — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANEV6JZUS663YQ3VW6M4OTW75VCVANCNFSM6AAAAAAWV2ITG4> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[mtakeda15]

If the aws sdk which this plugin uses can re-read the credentials, we may be able to support it.
However, I don't think aws sdk supports such a feature. I read some documents, but I could not find it.

[ppbrown]

maybe what is needed is to open a new session

…

On Tue, Apr 11, 2023 at 3:03 AM mtakeda ***@***.***> wrote: If the aws sdk which this plugin uses can re-read the credentials, we may be able to support it. However, I don't think aws sdk supports such a feature. I read some documents, but I could not find it. — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANEV6JF4TZS4GXGLINPAOTXAUT5HANCNFSM6AAAAAAWV2ITG4> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[mtakeda15]

Would a workaround like this work for you? It'd be helpful if you could try it out.

lib/fluent/plugin/out_timestream.rb

      def write_records(records)
        return if records.empty?
+       options = credential_options
+       options[:region] = @region if @region
+       options[:endpoint] = @endpoint if @endpoint
+       options[:ssl_verify_peer] = @ssl_verify_peer
+       @client = Aws::TimestreamWrite::Client.new(options)
        @client.write_records(
          database_name: @database,
          table_name: @table,
          records: records
        )

[ppbrown]

hmm what would that change?

…

On Fri, Apr 21, 2023 at 2:02 AM mtakeda ***@***.***> wrote: Would a workaround like this work for you? It'd be helpful if you could try it out. lib/fluent/plugin/out_timestream.rb def write_records(records) return if records.empty?+ options = credential_options+ options[:region] = @region if @region+ options[:endpoint] = @endpoint if @endpoint+ options[:ssl_verify_peer] = @ssl_verify_peer+ @client = Aws::TimestreamWrite::Client.new(options) @client.write_records( database_name: @database, table_name: @table, records: records ) — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANEV6LD7AEBFXJEYH4PSILXCJEI3ANCNFSM6AAAAAAWV2ITG4> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[mtakeda15]

Creating a timestream client every time when plugin send data to Timestream.
When client is recreated, new credential could be re-read.

By the way, how do you have your credential?
At least, if you specify the credentials by fluent.conf, this code does not work. (I guessed you don't do it)

[ppbrown]

Right. I export environment vars which are recognized by the AWS library used by the fluentd plugin

…

On Sun, Apr 23, 2023 at 8:18 PM mtakeda ***@***.***> wrote: Creating a timestream client every time when plugin send data to Timestream. When client is recreated, new credential could be re-read. By the way, how do you have your credential? At least, if you specify the credentials by fluent.conf, this code does not work. (I guessed you don't do it) — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANEV6KC3FWFNBSSE6F3GW3XCXWJZANCNFSM6AAAAAAWV2ITG4> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[mtakeda15]

Environment vars are read only when a process starts up, so I don't think this code works.
I expected that you have credential by ~/.aws/credentials or ~/.aws/config.

In this readme, they explain:

Shared configuration is loaded only a single time, and credentials are provided statically at client creation time. Shared credentials do not refresh.

So I expected if you have credentials by these files and update periodically, this code could be helpful.

[ppbrown]

Ah, I see what you are saying now. Yes, I think that any time it drops a connection, and then later creates a new one, that would be a good time to reread the cred file. Which would happen when you "create a new client" Good plan perhaps. I guess the only thing is to somehow make sure that it periodically drops and then recreates, at least (X often)

…

On Tue, Apr 25, 2023 at 6:45 PM mtakeda ***@***.***> wrote: Environment vars are read only when a process starts up, so I don't think this code works. I expected that you have credential by ~/.aws/credentials or ~/.aws/config . In this readme <https://github.com/aws/aws-sdk-ruby#configuration>, they explain: Shared configuration is loaded only a single time, and credentials are provided statically at client creation time. Shared credentials do not refresh. So I expected if you have credentials by these files and update periodically, this code could be helpful. — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANEV6JQ5D6SOS23RN466C3XDB44ZANCNFSM6AAAAAAWV2ITG4> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[mtakeda15]

@ppbrown
I wanted to know if the experimental code works in your environment or not.
If it works, I will consider how recreate client.

Could you confirm that code work or not in your environment?