connecting bottledwater with remote amazon rds postgres

[gyan]

Is there a way to connect bottlewater with a remote db as amazon rds?

[jong-eatsa]

+1

How can we capture CDC stream in a Postgres RDS instance in general? I need to capture this and send it to an AWS Kinesis Stream.

The only way I can see is to use AWS DMS continuous replication into an S3 target, then use a Lambda function to route this data into Kinesis. This seems rather convoluted (unlesss DMS can directly inject into Kinesis or trigger a Lambda)

[ewencp]

I believe anything that gets raw data from logical decoding requires an extra plugin since the only output plugin that ships with Postgres is simple test plugin. This is why both bottledwater and debezium have their own plugin.

The alternative would be to try to get changes indirectly, e.g. via the JDBC connector. This isn't as good as what a CDC connector can provide, but works generically across all databases with JDBC support, without requiring any modification of the database server itself.

[mwarkentin]

RDS postgres now supports wal2json output on postgres 9.5.7 and 9.6.3: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.FeatureSupport.Extensions

[gunnarmorling]

You might take a look at Debezium's Postgres connector. We also support wal2json as a logical decoding plug-in now.