Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support more SSL related configurations in Kafka connector #18256

Open
lmatz opened this issue Aug 27, 2024 · 2 comments
Open

support more SSL related configurations in Kafka connector #18256

lmatz opened this issue Aug 27, 2024 · 2 comments

Comments

@lmatz
Copy link
Contributor

lmatz commented Aug 27, 2024

right now, when connecting via SSL, the user is expected to input 4 fields:

  1. properties.ssl.ca.location
  2. properties.ssl.certificate.location
  3. properties.ssl.key.location
  4. properties.ssl.key.password

When RW is deployed in the cloud environment, the file needs to be mounted to the file system in the pod first and then the user can specify the path in the 3 location configs

Since we have supported create secret, it is easier if the user can just copy & paste the content of these files into RW as a secret. The Cloud user can still upload the file, but this uploading is just a widget on Cloud UI to copy & paste the content.

The change is that there is no longer a need to mount the file to the pod.

To support this, it requires the Kafka API, i.e. librdkafka to support configs of reading in credential content directly.

By checking https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md

each of the first three options has its corresponding API for reading in credential content, which is listed as follows:

  1. properties.ssl.ca.location
    ->
    ssl.ca.pem
    ssl_ca

  2. properties.ssl.certificate.location
    ->
    ssl.certificate.pem
    ssl_certificate

  3. properties.ssl.key.location
    ->
    ssl.key.pem
    ssl_key

This makes the cloud deployment and usage easier

@tabVersion
Copy link
Contributor

each of the first three options has its corresponding API for reading in credential content, which is listed as follows:

  1. properties.ssl.ca.location
    ->
    ssl.ca.pem
    ssl_ca
  2. properties.ssl.certificate.location
    ->
    ssl.certificate.pem
    ssl_certificate
  3. properties.ssl.key.location
    ->
    ssl.key.pem
    ssl_key

This makes the cloud deployment and usage easier

Upvote for the option changes, but they are mostly for on-perm users. Cloud Kafka vendors seldom require using custom certs to connect to brokers. We need to be aware of the case.

@lmatz
Copy link
Contributor Author

lmatz commented Sep 3, 2024

I will keep the current ones, and add the "*.pem" ones for all three options, which is offered in other DB/ETL saas products. We will also expose these fields in RW Cloud

I will not add the ones without extension, they are in the form of special DER-encoded binary bytes.

cc: @neverchanje

@lmatz lmatz modified the milestones: release-2.1, release-2.2 Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants