-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
QueryBuilder: 'contains' operator not implemented for SQLite storage backends #6494
Comments
Here I put my investigations on how to possibly implement this. First attempt (with
|
Important note on feature parity with PostgeSQLI am not sure neither the docs of SQLAlchemy on the PSQL dialect, nor our QueryBuilder docs, really explain what For a list, it should return "all lists where all elements are included". For elements, I don't know, needs to be checked. Here some quick examples (not to forget), but I suggest to do proper testing before implementing the more complex query with Data generationI first generate some simple data (in a PSQL backend) - note this should be done only once, or you will get multiple copies of the results! node = orm.Int(1).store()
node.base.extras.set_many({'sub': {'a': 1, 'b': 2, 'c': 3}})
node = orm.Int(2).store()
node.base.extras.set_many({'sub': {'b': 2, 'c': 3}})
node = orm.Int(3).store()
node.base.extras.set_many({'sub': ['a', 'b', 'c']})
node = orm.Int(4).store()
node.base.extras.set_many({'sub': ['b', 'c']})
node = orm.Int(5).store()
node.base.extras.set_many({'sub': 1}) Queries"Contains" with a listqbuild = orm.QueryBuilder()
filters = {"extras.sub": {"contains": ["a"]}}
qbuild.append(
orm.Int,
project=['attributes.value', 'extras.sub'],
filters=filters,
)
print(qbuild.all()) returns
"Does not contain" with a listqbuild = orm.QueryBuilder()
filters = {"extras.sub": {"!contains": ["a"]}}
qbuild.append(
orm.Int,
project=['attributes.value', 'extras.sub'],
filters=filters,
)
qbuild.all() returns
"Contains" with a dictionary, key exists, values does not matchqbuild = orm.QueryBuilder()
filters = {"extras.sub": {"contains": {"a": 'whatever'}}}
qbuild.append(
orm.Int,
project=['attributes.value', 'extras.sub'],
filters=filters,
)
qbuild.all() returns
"Contains" with a dictionary, key-value pair existsqbuild = orm.QueryBuilder()
filters = {"extras.sub": {"contains": {"a": 1}}}
qbuild.append(
orm.Int,
project=['attributes.value', 'extras.sub'],
filters=filters,
)
qbuild.all() returns
|
Possible solution for AiiDAAfter some testing (and the if operator == 'contains':
from sqlalchemy import select
if isinstance(value, (list, set)):
subq = select(database_entity).where(func.json_each(database_entity).table_valued('value', joins_implicitly=True).c.value.in_(value)).correlate_except()
subsubq = select(func.count()).select_from(subq).scalar_subquery()
return subsubq == len(value)
elif isinstance(value, dict):
raise NotImplementedError
else:
raise TypeError("contains filters can only have as a parameter a list (when matching against lists) or dictionaries (when matching against dictionaries)") If useful, here is an example script I was using for testing: from aiida import orm, load_profile
from aiida.storage.sqlite_temp import SqliteTempBackend
temp_profile = SqliteTempBackend.create_profile('temp-profile')
load_profile(temp_profile, allow_switch=True)
node = orm.Int(1).store()
node.base.extras.set_many({'sub': {'a': 1, 'b': 2, 'c': 3}})
node = orm.Int(2).store()
node.base.extras.set_many({'sub': {'b': 2, 'c': 3}})
node = orm.Int(3).store()
node.base.extras.set_many({'sub': {'a': 1, 'b': 2}})
node = orm.Int(4).store()
node.base.extras.set_many({'sub': "a"})
node = orm.Int(5).store()
node.base.extras.set_many({'sub': 1})
node = orm.Int(6).store()
node.base.extras.set_many({'sub': ["a", "b"]})
node = orm.Int(7).store()
node.base.extras.set_many({'sub': ["c", "d"]})
node = orm.Int(8).store()
node.base.extras.set_many({'sub': ["a", "c"]})
node = orm.Int(9).store()
node.base.extras.set_many({'sub': ["b", "d", "a"]})
qbuild = orm.QueryBuilder()
filters = {"extras.sub": {"~contains": ["a", "b"]}}
#filters = {"extras.sub": {"contains": ["c"]}}
qbuild.append(
orm.Int,
project=['attributes.value'],
filters=filters,
)
print(qbuild.all())
print(qbuild.as_sql(True)) Still TODOs (for whoever wants to pick this up):
|
aiida-core/src/aiida/storage/sqlite_zip/orm.py
Line 287 in 04926fe
Some discussion about implementation:
sqlalchemy/sqlalchemy#7836
The text was updated successfully, but these errors were encountered: