Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement pagination behavior for grouped models #125

Merged
merged 2 commits into from
Nov 8, 2024

Conversation

lu-pl
Copy link
Contributor

@lu-pl lu-pl commented Oct 29, 2024

Closes #114.

@lu-pl lu-pl force-pushed the lupl/grouped-pagination branch 5 times, most recently from 97d281b to 988f836 Compare October 30, 2024 12:05
@b1rger b1rger self-requested a review November 4, 2024 08:23
@lu-pl lu-pl force-pushed the lupl/grouped-pagination branch from 988f836 to 9205a38 Compare November 7, 2024 10:12
@lu-pl lu-pl marked this pull request as ready for review November 7, 2024 10:28
Copy link
Collaborator

@b1rger b1rger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems a bit overengineered to me ;)

Why even bother with a query constructor? You could combine the get_items_query_constructor and the construct_ methods into one:

get_query(model, query, limit, offset):
   if group_by_value := getattr(model, "model_config", {}).get("group_by", None):
      return construct_grouped_pagination_query(query, group_by_value, limit, offset)
   return construct_ungrouped_pagination_query(quer, limit, offset)

@lu-pl
Copy link
Contributor Author

lu-pl commented Nov 8, 2024

That seems a bit overengineered to me ;)

Why even bother with a query constructor? You could combine the get_items_query_constructor and the construct_ methods into one:

get_query(model, query, limit, offset):
   if group_by_value := getattr(model, "model_config", {}).get("group_by", None):
      return construct_grouped_pagination_query(query, group_by_value, limit, offset)
   return construct_ungrouped_pagination_query(quer, limit, offset)

I deliberately implemented this using match case because this allows pattern matching on model_config and to trigger different query constructor code paths depending on the model config. See the commit message.

It is very likely that additional constructors will be needed in the future (e.g. for filtering and ordering etc), so I feel like this should be pluggable.

I am aware that query construction is kind of untidy at the moment, this is why I will implement #134 as soon as these changes are merged.

@b1rger
Copy link
Collaborator

b1rger commented Nov 8, 2024

I deliberately implemented this using match case because this allows pattern matching on model_config and to trigger different query constructor code paths depending on the model config. See the commit message.

The commit message starts with: "The get_items_query_constructor should be implemented using match/case;" - I'm not sure what this means The get_items_query_constructor is implemented using match/case. Maybe it should be rather be saying "The get_items_query_constructor is implemented using match/case, because this alllows ...."

It is very likely that additional constructors will be needed in the future (e.g. for filtering and ordering etc), so I feel like this should be pluggable.

We can easily make this pluggable once this is actually needed. We don't have to make the code overly complex so we can cater to every possible scenario that might come up in the future.

@lu-pl
Copy link
Contributor Author

lu-pl commented Nov 8, 2024

The commit message starts with: "The get_items_query_constructor should be implemented using match/case;" - I'm not sure what this means The get_items_query_constructor is implemented using match/case.

True, this is misleading, will change the commit message.

We can easily make this pluggable once this is actually needed. We don't have to make the code overly complex so we can cater to every possible scenario that might come up in the future.

I essentially agree, still I am much in favor of using the suggested match/case version because I feel like this is a rather elegant way to implement the logic and it is also an idiomatic use case for Structural Pattern Matching.

The alternative to using the partial pattern matching mechanism on the model.model_config dict is to either use KeyError exception handling or dict.get lookups - i.e. explicit checking for entry existence.

The match/case solution allows for an exact implementation of the intended behavior in a very concise way. The fact that this is easily extensible too, is more of an additional benefit.

@lu-pl lu-pl force-pushed the lupl/grouped-pagination branch 2 times, most recently from 259e558 to 0ba0f09 Compare November 8, 2024 09:59
@b1rger
Copy link
Collaborator

b1rger commented Nov 8, 2024

The match/case solution allows for an exact implementation of the intended behavior in a very concise way

The dict.get lookup does this too, just with less lines. But this is not only about the match/case statement. What you basically want is to modify a string. Instead of just doing that, you implement a method that returns a constructor that has to implement a protocol. Do you see how that is overengineering?
Throw away the ItemsQueryConstructor, throw away the get_items_query_constructor and instead implement what you actually need - 3 lines of code directly in SPARQLModelAdapter.query

(Another example is the construct_ungrouped_pagination_query that basically mimics what f-strings do, just instead of using f-strings it uses a template, and in this commit you're also replacing that template with a method that runs substitute on that template for you)

@lu-pl
Copy link
Contributor Author

lu-pl commented Nov 8, 2024

The match/case solution allows for an exact implementation of the intended behavior in a very concise way

The dict.get lookup does this too, just with less lines. But this is not only about the match/case statement. What you basically want is to modify a string. Instead of just doing that, you implement a method that returns a constructor that has to implement a protocol. Do you see how that is overengineering? Throw away the ItemsQueryConstructor, throw away the get_items_query_constructor and instead implement what you actually need - 3 lines of code directly in SPARQLModelAdapter.query

This might be (a mild case of) overengineering now.
But the proposed solution is the base for a functional + pluggable design I know will be needed soon.

I would agree on implementing get_items_query_constructor with dict.get (for now), but the general functional + pluggable design deems me a good way of clearly defining responsibilities in the process of query construction. Also, the ItemsQueryConstructor protocol is not only a way of communicating intent to developers (including myself) but also important for static type checking.

Generally, I see your point and you are correct. But if I completely tear this down now, I will have to re-implement everything for issue #134 again. I have a pretty clear picture now of how the query construction logic should be organized. So the current PR is more of a documented transition to the (urgently needed) #134 refactoring.

(Another example is the construct_ungrouped_pagination_query that basically mimics what f-strings do, just instead of using f-strings it uses a template, and in this commit you're also replacing that template with a method that runs substitute on that template for you)

I totally agree, this is a remnant from an earlier implementation. I still want construct_ungrouped_pagination_query to be a callable though; as a top-level entity it is easier to communicate its intent and responsibility, it can have a docstring and it can be used in the constructor factory dispatcher (get_items_query_constructor).

@lu-pl lu-pl force-pushed the lupl/grouped-pagination branch 2 times, most recently from ccdac22 to b1ddf09 Compare November 8, 2024 11:09
@lu-pl lu-pl requested a review from b1rger November 8, 2024 11:31
@lu-pl lu-pl force-pushed the lupl/grouped-pagination branch from b1ddf09 to 5ec7256 Compare November 8, 2024 15:12
@lu-pl lu-pl force-pushed the lupl/grouped-pagination branch from 5ec7256 to 6598e6c Compare November 8, 2024 15:13
@lu-pl
Copy link
Contributor Author

lu-pl commented Nov 8, 2024

Ok, here is another version: I encapsulated the ItemsQueryConstructor dispatch in construct_items_query; this significantly tidies up the query retrieval in SPARQLModelAdapter:

class SPARQLModelAdapter(Generic[_TModelInstance]):
    # ...
    def query(
        self,
        *,
        page: int = 1,
        size: int = 100,
    ) -> Page[_TModelInstance]:
        """Run a query against an endpoint and return a Page model object."""
        count_query: str = construct_count_query(query=self._query, model=self._model)
        items_query: str = construct_items_query(
            query=self._query,
            model=self._model,
            limit=size,
            offset=calculate_offset(page, size),
        )
       # ...

This might still be considered overengineered or at least "eagerly modular", but as I said, I consider this PR more as a documented transition towards an rdfproxy.constructor.QueryConstructor class that will handle all query construction requirements in a concise and pluggable way.

@lu-pl lu-pl merged commit d8e307b into main Nov 8, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement correct pagination behavior for grouped models
2 participants