Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instructions to run a local Airflow server #110

Merged
merged 5 commits into from
Feb 15, 2024

Conversation

will-cromar
Copy link
Contributor

Description

See README diff

Tests

Tested instructions locally with a WIP DAG.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run one-shot tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed.

Change-Id: I700f597f799b48b592658d4292270d764a7f0eec
@will-cromar will-cromar force-pushed the wcromar/airflow-standalone branch from 66f2fc4 to 0318879 Compare February 12, 2024 21:39
Copy link
Collaborator

@RissyRan RissyRan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Will! How will this feature look like?

I ran this standalone before, and recall I need to do some setup. Is this the same for you? Could you provide one simple example for the run? I think this will be helpful for users as well.

README.md Outdated Show resolved Hide resolved
@will-cromar
Copy link
Contributor Author

Thanks Will! How will this feature look like?

I ran this standalone before, and recall I need to do some setup. Is this the same for you? Could you provide one simple example for the run? I think this will be helpful for users as well.

I actually didn't have to do any other setup besides authenticating with GCP gcloud auth login --update-adc . airflow standalone does the one-time setup and prints a link to a local server. I didn't try all of the DAGs, so I don't expect them all to work. Some might be coupled to the composer environment, or people might not have the right permissions (e.g. to directly create QR in the project).

I'm contributing this as a quick hack that may or may not work, but we should expand it over time to work with more task types.

Change-Id: I065805700b4979357cfeb1261749bab221efcd39
docs/contributing.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@RissyRan RissyRan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with a nit.

@will-cromar will-cromar merged commit 7a6f9d5 into master Feb 15, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants