conduit-connector-sql-server Pipeline Issue

[Om-Jumaili]

Good Day, all,

I am trying to set up Conduit as a pipeline for MS SQL table replication, as Kafka has been giving me non-stop headaches.

I set up my docker container, loaded the conduit-connector-sql-server plugin, and then tried to write my pipeline. here it is:

version: 2.2

pipelines:
  - id: SqlServer
    status: running
    name: MSSQL
    description: MSSQL Pipeline
    connectors:
      - id: sql-source-1
        type: source
        plugin: standalone:[email protected]
        name: sql-source
        settings:
          connection: sqlserver://****:****@192.168.1.150:1433?database=kayanhr-uat
          table: TimeAttendance.EmployeesPunches
          primaryKey: RowID
          orderingColumn: RowID
      - id: sql-dest-1
        type: destination
        plugin: standalone:[email protected]
        name: sql-destination
        settings:
          connection: sqlserver://****:****@192.168.1.152:1433?database=kayanhr-uat
          table: TimeAttendance.EmployeesPunches

I end up with this error:

2024-11-24T08:37:56+00:00 ERR node stopped error="node SqlServer:sql-source-1 stopped with error: could not open source connector: could not open source connector plugin: create combined iterator: setup cdc: create tracking table: mssql: Incorrect syntax near ')'." component=lifecycle.Service node_id=SqlServer:sql-source-1 stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743},{"file":"/app/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run","line":120},{"file":"/app/pkg/connector/source.go","func":"github.com/conduitio/conduit/pkg/connector.(*Source).open","line":308}]

if I switch the table to EmployeesPunches only, I get this error instead:

2024-11-24T08:19:57+00:00 ERR node stopped error="node SqlServer:sql-source-1 stopped with error: could not open source connector: could not open source connector plugin: create combined iterator: setup cdc: add trigger catch insert: mssql: The object 'EmployeesPunches' does not exist or is invalid for this operation." component=lifecycle.Service node_id=SqlServer:sql-source-1 stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743},{"file":"/app/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run","line":120},{"file":"/app/pkg/connector/source.go","func":"github.com/conduitio/conduit/pkg/connector.(*Source).open","line":308}]

I'm most likely writing my pipeline wrong, but I can't find any documentation on this specific connector. I would greatly appreciate any help.

[hariso]

@Om-Jumaili Thanks for letting us know about this issue!

The error create tracking table: mssql: Incorrect syntax near ')' means that the connector tried to create a tracking table, but the query wasn't correct. Unfortunately, the query is not being logged 🤦 so I'll add some logs to the connector. Would you mind trying out the connector after that?

I'm most likely writing my pipeline wrong, but I can't find any documentation on this specific connector.

Is there any particular part that you're interesting and you couldn't find? We're currently working on streamlining how we generate docs, so hopefully it should be easier to find the relevant docs once we're done with that.