Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add S3 relation to mimir coordinator #21

Merged
merged 14 commits into from
Jan 5, 2024
Merged

Add S3 relation to mimir coordinator #21

merged 14 commits into from
Jan 5, 2024

Conversation

IbraAoad
Copy link
Contributor

@IbraAoad IbraAoad commented Dec 13, 2023

Description

Add the ability to integrate the Mimir cluster with external S3 Storage

Rationale
Mimir is a highly scalable, highly available solution. As such, using host path- or PVC-backed storage won’t be enough to fulfill the resilience and availability requirements when deploying a cluster with multiple units or applications.

metadata.yaml Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Show resolved Hide resolved
Copy link
Contributor

@PietroPasotti PietroPasotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see the ball rolling but I see a few issues with the implementation.
Some stylistic feedback, and some charming best practices we should enforce.

Great job overall! Looking forward to the next steps :)

src/charm.py Outdated Show resolved Hide resolved
requirements.txt Show resolved Hide resolved
metadata.yaml Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/mimir_config.py Outdated Show resolved Hide resolved
src/mimir_config.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
Copy link
Contributor

@PietroPasotti PietroPasotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some issues are not resolved.

src/charm.py Outdated Show resolved Hide resolved
src/mimir_config.py Outdated Show resolved Hide resolved
src/mimir_config.py Outdated Show resolved Hide resolved
src/mimir_config.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
Copy link
Contributor

@PietroPasotti PietroPasotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to pass around typed data structures instead of untyped, opaque ones whenever possible.

You get better type hints and type checks if you pass around _S3ConfigData rather than its dictified version. I'd advise to postpone dumping it to dict until you really need the raw data structure.

To see how much this simplification helps with typing, add

reportUnknownParameterType = true

to pyproject.toml[tool.pyright] and run tox -e static-charm

(which we should put there in a different PR)

src/mimir_config.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
src/mimir_coordinator.py Outdated Show resolved Hide resolved
Copy link
Contributor

@PietroPasotti PietroPasotti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! We're getting there.
No need for another pass from my side, but please check that you haven't forgotten that default arg in the _S3ConfigData object :)

src/charm.py Show resolved Hide resolved
src/mimir_config.py Show resolved Hide resolved
@PietroPasotti
Copy link
Contributor

(and before we merge, let's make sure the integration tests are green)

@IbraAoad
Copy link
Contributor Author

IbraAoad commented Jan 5, 2024

Integration tests are failing and are out of this PR scope, merging this PR and Integration tests will be tackled separately

@IbraAoad IbraAoad merged commit a0a2f08 into main Jan 5, 2024
12 of 13 checks passed
@IbraAoad IbraAoad deleted the mimir-s3 branch January 5, 2024 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants