Get Data from Flink to Database

Introduction

In this page I want to show you how to get events to a database.

see the class DataToDatabase for reference.

Walkthrough

To get an event into a database we assume it is a SpecificRecord (org.apache.avro.specific.SpecificRecord).

Because of that we can extract its attributes (specified in its schema). Thanks to that we are able to extract the record values and put both the attributes and their to-be values in strings so they can be inserted in a sql query of the form INSERT INTO table (col1, col2, ...) VALUES (val1, val2, ...).

To actually put the data in the database we need a connection which we acquire from a ConnectionPool. after that we stick the query to the connection with a ComputableFuture. For the query to be executeg we have to get the QueryResult from the future CompletableFuture<QueryResult> future = connection.sendPreparedStatement(query).

Be aware you already need to have the database and table you want to store the data in.