Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support networkx to represent components as a graph #331

Merged
merged 41 commits into from
Sep 18, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
ee29743
Add networkx compatibility
irux Aug 22, 2023
0ace374
Fix formatting
irux Aug 22, 2023
bb61f5f
Save progress with ci working
irux Aug 22, 2023
862aba0
Linting
irux Aug 22, 2023
8f46969
Fix pyright
irux Aug 22, 2023
769bd04
Test new dependency
irux Aug 28, 2023
a97148a
Fix lock
irux Aug 28, 2023
d929459
Fix problem when names are not overriten
irux Aug 28, 2023
7408e02
Delete pl show
irux Aug 28, 2023
a1ae585
Deleting ploting graph
irux Aug 28, 2023
36a86b0
Add test and fix some edge cases
irux Aug 29, 2023
c3aba1d
remove unused dependency
irux Aug 29, 2023
7476784
Delete more unused depencendies
irux Aug 29, 2023
f364f00
Delete matploit for testing
irux Aug 29, 2023
c197969
Test and changes
irux Aug 30, 2023
00da489
Implement comments
irux Sep 5, 2023
d84e0ea
Linting
irux Sep 5, 2023
147ff74
Format
irux Sep 5, 2023
e9f8712
Add more graph tests
irux Sep 5, 2023
d35bc41
Remove unused
irux Sep 5, 2023
5262f17
Implement comments
irux Sep 5, 2023
f81d573
Reformat
irux Sep 5, 2023
3ced2f6
Implement changes
irux Sep 5, 2023
35b7d8c
Implement changes
irux Sep 11, 2023
20e0c09
Replace link
irux Sep 11, 2023
510de2f
Fix link to kpops-examples (#357)
sujuka99 Sep 11, 2023
f1dbe1f
Fix docstring
irux Sep 12, 2023
931731b
Join tests
irux Sep 12, 2023
0052cd1
Increase timeout
irux Sep 12, 2023
f9cc88a
Save progress
irux Sep 12, 2023
b4e6a9e
Merge last state
irux Sep 12, 2023
184bc43
Implement changes
irux Sep 12, 2023
bd724e1
Delete unused imports
irux Sep 12, 2023
133005a
Implement changes
irux Sep 13, 2023
b956ebc
Fix test
irux Sep 18, 2023
df16d00
Solve linting
irux Sep 18, 2023
8b87409
Fix pyright
irux Sep 18, 2023
d73f169
Fix linting
irux Sep 18, 2023
a2a3192
Delete from implemented methods
irux Sep 18, 2023
5432c8a
Implement changes
irux Sep 18, 2023
1e89223
Change to id
irux Sep 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions kpops/pipeline_generator/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from collections.abc import Iterator
from contextlib import suppress
from pathlib import Path
import matplotlib.pyplot as plt
disrupted marked this conversation as resolved.
Show resolved Hide resolved

import networkx as nx
import yaml
Expand Down
59 changes: 59 additions & 0 deletions tests/components/test_streams_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -429,3 +429,62 @@ async def test_should_clean_streams_app_and_deploy_clean_up_job_and_delete_clean
"test-namespace", self.STREAMS_APP_CLEAN_NAME, dry_run
),
]

@pytest.mark.asyncio
async def test_get_topics(
disrupted marked this conversation as resolved.
Show resolved Hide resolved
self, config: PipelineConfig, handlers: ComponentHandlers
):
streams_app = StreamsApp(
name=self.STREAMS_APP_NAME,
config=config,
handlers=handlers,
**{
"namespace": "test-namespace",
"app": {
"streams": {"brokers": "fake-broker:9092"},
},
"from": {
"topics": {
"example-input": {"type": "input"},
"b": {"type": "input"},
"a": {"type": "input"},
"topic-extra2": {"type": "extra", "role": "role2"},
"topic-extra3": {"type": "extra", "role": "role2"},
"topic-extra": {"type": "extra", "role": "role1"},
".*": {"type": "input-pattern"},
"example.*": {
"type": "extra-pattern",
"role": "another-pattern",
},
}
},
},
)
assert streams_app.get_input_topics() == ["example-input", "b", "a"]
assert streams_app.get_extra_input_topics() == {
"role1": ["topic-extra"],
"role2": ["topic-extra2", "topic-extra3"],
}

@pytest.mark.asyncio
async def test_get_output_topic(
self, config: PipelineConfig, handlers: ComponentHandlers
):
streams_app = StreamsApp(
name=self.STREAMS_APP_NAME,
config=config,
handlers=handlers,
**{
"namespace": "test-namespace",
"app": {
"streams": {"brokers": "fake-broker:9092"},
},
"from": {
"topics": {
"example-input": {"type": "input"},
}
},
"to": {"topics": {"example-output": {"type": "output"}}},
},
)
assert streams_app.get_output_topic() == "example-output"
2 changes: 1 addition & 1 deletion tests/pipeline/resources/pipeline-with-loop/defaults.yaml
sujuka99 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ streams-app:
labels:
pipeline: ${pipeline_name}
streams:
optimizeLeaveGroupBehavior: false
optimizeLeaveGroupBehavior: false
2 changes: 1 addition & 1 deletion tests/pipeline/resources/pipeline-with-loop/pipeline.yaml
sujuka99 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,4 @@
to:
topics:
my-output-topic:
type: output
type: output