Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Torii grpc entities query #1196

Merged
merged 19 commits into from
Nov 22, 2023
Merged

Torii grpc entities query #1196

merged 19 commits into from
Nov 22, 2023

Conversation

broody
Copy link
Collaborator

@broody broody commented Nov 19, 2023

This PR exposes a RetrieveEntities grpc api, only implemented querying by keys clause, querying by attribute and composite clause to be implemented.

Also, map_row_to_ty method is not currently used, will eventually be used by graphql

Query format is similar to graphql, supply keys array and it will return all entities matching those keys. For wildcard, leave key part blank. Only pagination supported is limit offset.

{
    "query": {
        "clause": {
            "keys": {
                "keys": [
                    [], <-- wildcard
                    [3]
                ]
            }
        },
        "limit": 2,
        "offset": 0
    }
}

Return format

{
    "entities": [
        {
            "models": [
                {
                    "members": [
                        {
                            "name": "record_id",
                            "value": {
                                "uint_value": "2"
                            }
                        },
                        {
                            "name": "subrecord_id",
                            "value": {
                                "uint_value": "3"
                            }
                        },
                        {
                            "name": "type_u8",
                            "value": {
                                "uint_value": "0"
                            }
                        },
                        {
                            "name": "random_u8",
                            "value": {
                                "uint_value": "69"
                            }
                        }
                    ],
                    "name": "Subrecord"
                }
            ],
            "key": "AMtAZ2/a/AeZjxORv2Y1a18MltXmG79efsCqAyDdj5s=" <-- this is bytes
        }
    ]
}

@broody broody force-pushed the grpc-entities-endpoint branch 2 times, most recently from 199b111 to 2cb0bf5 Compare November 20, 2023 22:05
@broody broody force-pushed the grpc-entities-endpoint branch from 2cb0bf5 to 7f4d902 Compare November 20, 2023 22:06
@broody broody force-pushed the grpc-entities-endpoint branch from 1dc6544 to beb6801 Compare November 22, 2023 01:44
@broody broody marked this pull request as ready for review November 22, 2023 01:45
schemas: RwLock<HashMap<ModelName, Ty>>,
}

impl ModelCache {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will eventually need a way to invalidate a cache entry when we index an update to an existing model. Lets create an issue to track it

global_tables.extend(tables);
}

// TODO: Fallback to subqueries, SQLite has a max limit of 64 on 'table 'JOIN'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is fine to return an error in this case. 64 joins is a reasonable limit

@@ -31,7 +33,7 @@ pub struct KeysClause {
#[derive(Debug, Serialize, Deserialize, PartialEq, Hash, Eq, Clone)]
pub struct AttributeClause {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also rename this to MemberClause in a future PR

@tarrencev tarrencev merged commit b3f4997 into main Nov 22, 2023
9 checks passed
@tarrencev tarrencev deleted the grpc-entities-endpoint branch November 22, 2023 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants