Allow replacing the DocumentDBSchemaService #477

normanj-bitquill · 2023-01-05T18:59:37Z

Summary

Want to be able to choose an implementation of DocumentDBSchemaService.

Description

Working on an application that manages collection metadata internally. As a result, it is not desirable to store the collections schemas in the database. The changes here make it possible to replace the implementation of DocumentDBSchemaService.

Related Issue

None

Additional Reviewers

@affonsoBQ
@alinaliBQ
@andiem-bq
@alexey-temnikov
@birschick-bq

* [AD-1014] Developer Guide. * Commit Code Coverage Badge * [AD-1014] Updates to use existing GETTING_STARTED.md and added schema-caching.md * Commit Code Coverage Badge Co-authored-by: birschick-bq <[email protected]>

* Bump JDBC version from 1.4.1 to 1.4.2 * Commit Code Coverage Badge Co-authored-by: affonsoBQ <[email protected]>

* Bump commons-text from 1.9 to 1.10.0 Bumps commons-text from 1.9 to 1.10.0. --- updated-dependencies: - dependency-name: org.apache.commons:commons-text dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Bump com.github.spotbugs from 5.0.10 to 5.0.13 Bumps com.github.spotbugs from 5.0.10 to 5.0.13. --- updated-dependencies: - dependency-name: com.github.spotbugs dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Bump mockito-core from 4.8.0 to 4.8.1 Bumps [mockito-core](https://github.com/mockito/mockito) from 4.8.0 to 4.8.1. - [Release notes](https://github.com/mockito/mockito/releases) - [Commits](mockito/mockito@v4.8.0...v4.8.1) --- updated-dependencies: - dependency-name: org.mockito:mockito-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Commit Code Coverage Badge * Bump sslcontext-kickstart from 7.4.7 to 7.4.8 Bumps [sslcontext-kickstart](https://github.com/Hakky54/sslcontext-kickstart) from 7.4.7 to 7.4.8. - [Release notes](https://github.com/Hakky54/sslcontext-kickstart/releases) - [Changelog](https://github.com/Hakky54/sslcontext-kickstart/blob/master/CHANGELOG.md) - [Commits](Hakky54/sslcontext-kickstart@v7.4.7...v7.4.8) --- updated-dependencies: - dependency-name: io.github.hakky54:sslcontext-kickstart dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> * Bump checkstyle from 10.3.4 to 10.4 Bumps [checkstyle](https://github.com/checkstyle/checkstyle) from 10.3.4 to 10.4. - [Release notes](https://github.com/checkstyle/checkstyle/releases) - [Commits](checkstyle/checkstyle@checkstyle-10.3.4...checkstyle-10.4) --- updated-dependencies: - dependency-name: com.puppycrawl.tools:checkstyle dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Revert "Bump checkstyle from 10.3.4 to 10.4" This reverts commit b4b048d. * Adding mongo driver information * fixing build.gradle * adding missing class * Revert "adding missing class" This reverts commit 5a53d21. * adding missing class * adding license * fixing check style * adding final test fixture * Commit Code Coverage Badge * Commit Code Coverage Badge * Commit Code Coverage Badge * [AD-1032] Driver / Application information for CloudWatch. Simplify application and version handling. * Commit Code Coverage Badge * [AD-1032] Refactored creating the MongoClient for cleaner use. * [AD-1032] Fixed style issues. * [AD-1032] Fixed style issues aws#2. * [AD-1032] Fixed style issues aws#3. * Commit Code Coverage Badge Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Bruce Irschick <[email protected]> Co-authored-by: birschick-bq <[email protected]> Co-authored-by: affonsoBQ <[email protected]>

* [AD-1017] Implement single-instance SSH tunnel. * Handle late-bound invocation (DbVisualizer) * Ensure support for multiple hash algorithms. * Properly embed new SSH tunnel component. * [AD-1017] Attempt to debug multi-process test. * [AD-1017] Attempt to debug multi-process test. * [AD-1017] Attempt to debug multi-process test - aws#3. * Commit Code Coverage Badge * [AD-1017] Attempt to find correct location for test source file. * [AD-1017] Attempt to find correct location for test source file - aws#2. * [AD-1017] Fix spotBugs error; code review improvements. * [AD-1017] Ignore IO exceptions when reading spawned processes. * Commit Code Coverage Badge * [AD-1017] Update dependency for mongo to 4.8.1 * Commit Code Coverage Badge * [AD-1017] Attempt improve stability of SSH Tunnel client tests. * [AD-1017] Attempt improve stability of multiprocess connection test. * Commit Code Coverage Badge * Update src/main/java/software/amazon/documentdb/jdbc/DocumentDbConnectionProperties.java Co-authored-by: Alexey Temnikov <[email protected]> * [AD-1017] Properly rename parameter. * Commit Code Coverage Badge * [AD-1017] Fix issue with output not being sent to the console (i.e., change level from INFO to ERROR). * Commit Code Coverage Badge * [AD-1017] Attempt to increase code coverage. * [AD-1017] Fix style issue. * Commit Code Coverage Badge Co-authored-by: birschick-bq <[email protected]> Co-authored-by: Alexey Temnikov <[email protected]>

* [AD-1039] Updated release version and dependencies. * Commit Code Coverage Badge Co-authored-by: birschick-bq <[email protected]>

…ws#468) * [Diagnosis] attempt to diagnose test failure * [Diagnosis] attempt to diagnose test failure * Commit Code Coverage Badge * [AD-1043] Mark tests so they only run when integration testing. * Commit Code Coverage Badge * [AD-1043] Attempt to resolve "Commit Code Coverage Badge" when running mavenFilesPreparation * [AD-1043] Add upload of raw-test-results. * [AD-1043] attempt to resolve bug. * Commit Code Coverage Badge Co-authored-by: birschick-bq <[email protected]>

* The schema metadata service can now be changed. A supplier can be passed in when the Driver is created.

alinaliBQ · 2023-01-05T19:27:02Z

Hi, there are 51 files shown in the files tab for this PR, could you please rebase or merge from the develop branch?

normanj-bitquill · 2023-01-05T22:00:52Z

@alinaliBQ It looks like @birschick-bq already fixed up my branch. It now shows 16 files changed. I'm fine iterating on this PR with feedback.

To begin with though, can you look over the PR and see if the overall design makes sense. I'd like to start planning the code that will depend on this.

alinaliBQ · 2023-01-06T18:37:22Z

@alinaliBQ It looks like @birschick-bq already fixed up my branch. It now shows 16 files changed. I'm fine iterating on this PR with feedback.

To begin with though, can you look over the PR and see if the overall design makes sense. I'd like to start planning the code that will depend on this.

Thanks for creating the PR and putting it together, but I'm not primarily working on DocumentDB, perhaps other team members can have a look.

birschick-bq · 2023-01-06T18:43:08Z

@alinaliBQ It looks like @birschick-bq already fixed up my branch. It now shows 16 files changed. I'm fine iterating on this PR with feedback.

To begin with though, can you look over the PR and see if the overall design makes sense. I'd like to start planning the code that will depend on this.

@normanj-bitquill I'll have a look at the changes/design over the next week.

normanj-bitquill · 2023-01-23T19:17:36Z

@birschick-bq Have you had a chance to look over this PR?

* The schema metadata service can now be changed. A supplier can be passed in when the Driver is created.

…bitquill/amazon-documentdb-jdbc-driver into override-schema-service

src/main/java/software/amazon/documentdb/jdbc/metadata/DocumentDbSchema.java

birschick-bq · 2023-01-30T23:46:08Z

src/main/java/software/amazon/documentdb/jdbc/DocumentDbDriver.java

+     * Constructs a new DocumentDbDriver with the provided metadata service.
+     * @param metadataServiceProvider metadata service to use
+     */
+    public DocumentDbDriver(final Supplier<DocumentDbMetadataService> metadataServiceProvider) {


@normanj-bitquill I'm assuming this will be the entry point to override the schema service for a future local file implementation. Correct?

@birschick-bq That is correct. The goal is to supply my own implementation of the DocumentDbMetadataService that will be used for all connections made using the driver.

src/main/java/software/amazon/documentdb/jdbc/DocumentDbConnection.java

birschick-bq · 2023-01-30T23:49:54Z

src/main/java/software/amazon/documentdb/jdbc/DocumentDbConnection.java

            throws SQLException {
        super(connectionProperties);
        this.connectionProperties = connectionProperties;
+        this.metadataService = metadataService;


I think we might have to check for nulls since the public interface from DocumentDbDriver might return a null from the Supplier.

Would it be sufficient to add a not null check in the DocumentDbDriver constructor so that the DocumentDbMetadataService is never null?

Added a null check in DocumentDbDriver just before this constructor is called.

normanj-bitquill · 2023-02-14T16:54:45Z

@sunnie629 @CassieLyu Can you take a look at this PR? I'd like to help move this forward.

jlarue26 · 2023-03-06T22:25:07Z

@sunnie629 @CassieLyu how can we help get this PR merged? @normanj-bitquill has responded to all conversations.

sunnie629 · 2023-03-06T22:32:04Z

@normanj-bitquill @jlarue26 our product team is working on taking a look at this PR to sign off if we want to merge it

jlarue26 · 2023-03-07T15:01:20Z

Much appreciated @sunnie629

sunnie629 · 2023-03-16T19:14:36Z

just to be clear, no custom implementation of the metadata service has been done yet right? this is just changing the code to set that up?

normanj-bitquill · 2023-03-16T21:05:11Z

just to be clear, no custom implementation of the metadata service has been done yet right? this is just changing the code to set that up?

@sunnie629 I have worked on a custom implementation. The custom implementation would live in a different project that uses this driver. It will be adjusted as needed based on this PR. In short we have a server that maintains its own view of the metadata, so there is no reason for us to write it to the MongoDB database.

Bruce Irschick and others added 8 commits December 27, 2022 11:18

[AD-1014] Developer Guide. (aws#451)

543b65a

* [AD-1014] Developer Guide. * Commit Code Coverage Badge * [AD-1014] Updates to use existing GETTING_STARTED.md and added schema-caching.md * Commit Code Coverage Badge Co-authored-by: birschick-bq <[email protected]>

Bump JDBC version from 1.4.1 to 1.4.2 (aws#457)

c1281a5

* Bump JDBC version from 1.4.1 to 1.4.2 * Commit Code Coverage Badge Co-authored-by: affonsoBQ <[email protected]>

[AD-1039] Updated release version and dependencies. (aws#467)

a341734

* [AD-1039] Updated release version and dependencies. * Commit Code Coverage Badge Co-authored-by: birschick-bq <[email protected]>

Refactored to allow setting a schema metadata service

196ab46

* The schema metadata service can now be changed. A supplier can be passed in when the Driver is created.

Exposed two methods for subclasses

6aeab4a

Bruce Irschick added 2 commits January 5, 2023 12:10

Merge remote-tracking branch 'aws/develop' into override-schema-service

87c4190

Attempting fix on branch retrieval.

8db4f2b

birschick-bq mentioned this pull request Jan 6, 2023

Enable local persistence of detected schemas #438

Open

birschick-bq linked an issue Jan 6, 2023 that may be closed by this pull request

Enable local persistence of detected schemas #438

Open

birschick-bq and others added 5 commits January 30, 2023 10:10

Merge remote-tracking branch 'aws/develop' into override-schema-service

78eac43

Changed setGetTableFunction() to be a noop if tables is already set.

b7e4346

Refactored to allow setting a schema metadata service

30bf479

* The schema metadata service can now be changed. A supplier can be passed in when the Driver is created.

Exposed two methods for subclasses

c5899db

Changed setGetTableFunction() to be a noop if tables is already set.

9a61559

normanj-bitquill force-pushed the override-schema-service branch from 337f412 to b7e4346 Compare January 30, 2023 19:03

Fix a compile error in a test file.

e7998c8

normanj-bitquill force-pushed the override-schema-service branch from b7e4346 to e7998c8 Compare January 30, 2023 19:17

birschick-bq added 3 commits January 30, 2023 11:19

Merge branch 'override-schema-service' of https://github.com/normanj-…

42438a1

…bitquill/amazon-documentdb-jdbc-driver into override-schema-service

Fix merge issues

61640dd

Fix merge issues - aws#2

3328ff5

birschick-bq reviewed Jan 30, 2023

View reviewed changes

Added a null check on the DocumentDbMetadataService

0612175

birschick-bq requested review from affonsov, CassieLyu and sunnie629 January 31, 2023 18:19

Added a comment for when setGetTablesFunction is a noop.

bb115b9

sunnie629 approved these changes Mar 16, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow replacing the DocumentDBSchemaService #477

Allow replacing the DocumentDBSchemaService #477

normanj-bitquill commented Jan 5, 2023

alinaliBQ commented Jan 5, 2023

normanj-bitquill commented Jan 5, 2023

alinaliBQ commented Jan 6, 2023

birschick-bq commented Jan 6, 2023

normanj-bitquill commented Jan 23, 2023

birschick-bq Jan 30, 2023

normanj-bitquill Jan 30, 2023

birschick-bq Jan 30, 2023

normanj-bitquill Jan 30, 2023

normanj-bitquill Jan 31, 2023

normanj-bitquill commented Feb 14, 2023

jlarue26 commented Mar 6, 2023

sunnie629 commented Mar 6, 2023

jlarue26 commented Mar 7, 2023

sunnie629 commented Mar 16, 2023

normanj-bitquill commented Mar 16, 2023

Allow replacing the DocumentDBSchemaService #477

Are you sure you want to change the base?

Allow replacing the DocumentDBSchemaService #477

Conversation

normanj-bitquill commented Jan 5, 2023

Summary

Description

Related Issue

Additional Reviewers

alinaliBQ commented Jan 5, 2023

normanj-bitquill commented Jan 5, 2023

alinaliBQ commented Jan 6, 2023

birschick-bq commented Jan 6, 2023

normanj-bitquill commented Jan 23, 2023

birschick-bq Jan 30, 2023

Choose a reason for hiding this comment

normanj-bitquill Jan 30, 2023

Choose a reason for hiding this comment

birschick-bq Jan 30, 2023

Choose a reason for hiding this comment

normanj-bitquill Jan 30, 2023

Choose a reason for hiding this comment

normanj-bitquill Jan 31, 2023

Choose a reason for hiding this comment

normanj-bitquill commented Feb 14, 2023

jlarue26 commented Mar 6, 2023

sunnie629 commented Mar 6, 2023

jlarue26 commented Mar 7, 2023

sunnie629 commented Mar 16, 2023

normanj-bitquill commented Mar 16, 2023