Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI: conduit pipelines describe ID #2020

Closed
raulb opened this issue Dec 18, 2024 · 2 comments · Fixed by #2078
Closed

CLI: conduit pipelines describe ID #2020

raulb opened this issue Dec 18, 2024 · 2 comments · Fixed by #2078
Assignees
Labels
feature New feature or request

Comments

@raulb
Copy link
Member

raulb commented Dec 18, 2024

Feature description

This command will describe the topology of the pipeline.

It requires having Conduit previously running.

It requires the pipeline id as argument.

Arguments

Name Description Required Default Value
id pipeline id to describe Yes

Flags

Name Description Required Default Value
grpc.address address for serving the gRPC API No ":8084"

--help

$ conduit pipelines describe ID

EXAMPLE:

$ conduit pipelines describe my-pipeline
ID: generator-to-log
State:
  Status: STATUS_STOPPED
Config:
  Name: generator-to-log
  Description: Postgres source, file destination
Connector IDs:
  - generator-to-log:postgres-source
  - generator-to-log:destination
Processor IDs:
  - my-processor:avro.encode
Created At: 2024-10-09T13:55:17.113068Z
Updated At: 2024-10-09T13:55:17.113802Z

Part of #1911

@raulb raulb added the feature New feature or request label Dec 18, 2024
@raulb raulb mentioned this issue Dec 18, 2024
39 tasks
@raulb raulb moved this from Triage to Todo in Conduit Main Dec 18, 2024
@lovromazgon
Copy link
Member

Given the issue that made us think we have a bug, I think that a user would expect to see the whole pipeline including processors attached to connectors in this command's output. In this sense, describing a pipeline would mean seeing the whole pipeline topology (all connecotrs and processors). If we go a step further, we could even visualize the pipeline using ASCII art. Maybe not as part of the first release though, the scope if big enough already.

@hariso
Copy link
Contributor

hariso commented Jan 13, 2025

Given the issue that made us think we have a bug, I think that a user would expect to see the whole pipeline including processors attached to connectors in this command's output. In this sense, describing a pipeline would mean seeing the whole pipeline topology (all connecotrs and processors). If we go a step further, we could even visualize the pipeline using ASCII art. Maybe not as part of the first release though, the scope if big enough already.

This makes a lot of sense especially in simpler pipelines, e.g. with one source/destination, where there's practically no difference between a pipeline processor and a connector processor.

@hariso hariso self-assigned this Jan 16, 2025
@hariso hariso moved this from Todo to In Progress in Conduit Main Jan 16, 2025
@github-project-automation github-project-automation bot moved this from In Progress to Done in Conduit Main Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants