Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ web-server: Refactor users domain for improved layer separation and upgrading to asyncpg #6937

Open
wants to merge 105 commits into
base: master
Choose a base branch
from

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Dec 10, 2024

What do these changes do?

The users plugin has quite some old functionality so before adding further logic into it, I decided to do some refactoring to align with the conventions and libraries we should be using.

  • Layer Separation:

    • Clear separation of rest-controller, service, and repository layers within the users domain.
    • The api is now a pure interface for other domains.
    • Sub-domains were renamed for clarity (further refactoring to follow in separate PRs).
  • Schema and Domain Model Separation:

    • Introduced a clear distinction between schemas and domain models.
    • Implemented type adapters for seamless conversion between schemas and domain models:
      • schema.to_model() and schema.from_model().
      • The schema layer is aware of domain models, but domain models remain agnostic of schemas.
  • Repository Migration:

    • Replaced aiopg with asyncpg across all repository code.
    • Addressed shared code (e.g., groups_extra_properties) where necessary, though some duplication exists for now.
  • **Misc: **

  • User Enums: Moved user-related enums to the common-library for broader reuse.

Related issue/s

How to test

cd services/web/server
make install-dev
pytest -vv --pdb tests/unit/**/test_*user*.py 

Dev-ops

None

@pcrespov pcrespov self-assigned this Dec 10, 2024
@pcrespov pcrespov added a:webserver issue related to the webserver service a:database associated to postgres service and postgres-database package labels Dec 10, 2024
Copy link

codecov bot commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 74.62185% with 151 lines in your changes missing coverage. Please review.

Project coverage is 83.36%. Comparing base (4eeaa5c) to head (a9b48dc).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6937      +/-   ##
==========================================
- Coverage   88.10%   83.36%   -4.75%     
==========================================
  Files        1592     1616      +24     
  Lines       62326    63258     +932     
  Branches     2012     2068      +56     
==========================================
- Hits        54912    52732    -2180     
- Misses       7078    10179    +3101     
- Partials      336      347      +11     
Flag Coverage Δ
integrationtests 64.90% <60.30%> (-0.10%) ⬇️
unittests 82.03% <73.44%> (-4.30%) ⬇️
Components Coverage Δ
api 76.84% <ø> (ø)
pkg_aws_library 93.49% <ø> (ø)
pkg_dask_task_models_library 97.09% <ø> (ø)
pkg_models_library 91.36% <90.75%> (+0.12%) ⬆️
pkg_notifications_library 84.57% <ø> (ø)
pkg_postgres_database 88.26% <100.00%> (+0.20%) ⬆️
pkg_service_integration 70.02% <ø> (ø)
pkg_service_library 74.58% <ø> (ø)
pkg_settings_library 90.60% <ø> (ø)
pkg_simcore_sdk 85.38% <ø> (ø)
agent 97.00% <ø> (ø)
api_server 90.13% <ø> (ø)
autoscaling 95.42% <ø> (ø)
catalog 90.57% <ø> (ø)
clusters_keeper 99.48% <ø> (ø)
dask_sidecar 91.26% <ø> (ø)
datcore_adapter 93.18% <ø> (ø)
director 76.40% <ø> (ø)
director_v2 91.40% <ø> (ø)
dynamic_scheduler 96.99% <ø> (ø)
dynamic_sidecar 89.75% <ø> (ø)
efs_guardian 90.12% <ø> (ø)
invitations 93.44% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.58% <ø> (-0.07%) ⬇️
storage 89.54% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 72.53% <64.69%> (-15.25%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4eeaa5c...a9b48dc. Read the comment docs.

@pcrespov pcrespov force-pushed the is1779/refactor-users branch from ed2a0d5 to b358fb4 Compare December 10, 2024 22:43
@pcrespov pcrespov added this to the Event Horizon milestone Dec 11, 2024
@pcrespov pcrespov force-pushed the is1779/refactor-users branch from b358fb4 to 8e3bedb Compare December 11, 2024 19:46
@pcrespov pcrespov requested review from bisgaard-itis and removed request for jsaq007 and ignapas December 12, 2024 23:54
Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, I like it. left a few comments. Thanks


async with pass_or_acquire_connection(engine, connection) as conn:
result = await conn.stream(
sa.select(*(users.columns[name] for name in return_column_names)).where(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens if I pass [] ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point. I would say that the current code would returns None, but it has been a waste to transmit that ...

Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 👍 All good

@pcrespov pcrespov requested a review from GitHK December 13, 2024 14:54
Copy link

sonarcloud bot commented Dec 13, 2024

@pcrespov pcrespov mentioned this pull request Dec 13, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:database associated to postgres service and postgres-database package a:models-library a:webserver issue related to the webserver service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants