Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make koza able to handle yamls with multiple Source configurations #149

Open
DnlRKorn opened this issue Oct 3, 2024 · 0 comments
Open
Labels
enhancement New feature or request

Comments

@DnlRKorn
Copy link
Contributor

DnlRKorn commented Oct 3, 2024

Here's an excalidraw I whipped up of the current Koza pipeline.

image

It may be desirable to make it so "transform.yaml" has two different configurations for two different writers/filters. I have a few ideas how we could do this;

  1. One simple one may be just making a different yaml file and having two yaml files. This would include the least amount of rewriting.
  2. Refactor "transform_source" into an entry point which converts the yaml file pointer into a koza.Source object and then another helper function which runs and processes koza.Source objects. Then write another function called "transform_sources" which does specialized processes on the yaml file to make multiple koza.Source objects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant