Skip to content

FAQ

Jump to bottom
Geoff Williams edited this page Jan 5, 2024 · 8 revisions
When I try to compile I get an error about a missing SNAPSHOT dependency.

The error looks something like this:

[ERROR] Failed to execute goal on project kafka-connect-jdbc: Could not resolve dependencies for project
io.confluent:kafka-connect-jdbc:jar:3.1.0-SNAPSHOT: Could not find artifact 
org.apache.kafka:connect-api:jar:0.10.1.0-SNAPSHOT in confluent (http://packages.confluent.io/maven/) -> [Help 1]

During development of the JDBC connector we sometimes use a SNAPSHOT version of Kafka in order to build and test against new features. If you want to build a development version, you may need to build and install these dependencies to your local Maven repository in order to build the connector:

Out of Memory Errors

The connector sets the JDBC fetch size to be equal to batch.max.rows. This provides a hint to the JDBC driver as to the number of rows to fetch. However, in practice, drivers may require this option to be set in conjunction with other settings to properly stream results and avoid OOM. Here are some dialects and configuration required gathered from Issue 34:

  • MySQL: Attach to your JDBC connection string the parameter: ?useCursorFetch=true
  • PostgreSQL: Should work out of the box
  • MS SQL: Attach to connection string the parameter: ;selectMethod=cursor
Cannot run tests/most tests failing

Make sure to use Java 11 to compile and run tests, otherwise get errors like java.base does not "opens java.lang" to unnamed module

Unexpected test failure(s). I'm using Windows

Running under WSL2 with source repository on Windows C drive mounted into the WSL via /mnt/c, 100% of the time, specific test will fail unexpectedly (it passes on Linux):

[ERROR] Failures:
[ERROR]   JdbcSourceConnectorTest.testStartStop:148
  Unexpected method calls:
    CachedConnectionProvider.close()

Speculate the cause could be due to either very slow host filesystem access with the 9P server which is at least 10x slower then normal EXT4 access or some unimplemented filesystem attribute.

The only fix seems to be to move the source code under test into the WSL2 filesystem. This causes all tests to pass but means you now have "files you care about" inside of the WSL2 filesystem.

How to run tests?
  • Unit tests: mvn test
  • Integration tests: mvn integration-test
Clone this wiki locally