Searching a catalog for all but one value?

[r-ford]

Hi Intake team,

I'm searching the Pangeo CMIP6 catalog with the following:

query = dict(experiment_id=['1pctCO2', 'piControl'], table_id='Omon', variable_id='hfds', grid_label='gn', member_id='r1i1p1f1', require_all_on=['source_id'])

But for regridding reasons, there is a model I don't want to include (source_id='AWI-CM-1-1-MR'). Is it possible to search a catalog to exclude a certain value instead of just manually listing all but one (something like source_id!='AWI-CM-1-1-MR')?

[andersy005]

thank you for your patience, @r-ford.

Is it possible to search a catalog to exclude a certain value instead of just manually listing all but one (something like source_id!='AWI-CM-1-1-MR')?

One way to do this is via a regular expression

In [35]: url
Out[35]: 'https://raw.githubusercontent.com/intake/intake-esm/main/tutorial-catalogs/GOOGLE-CMIP6.json'

In [36]: cat = intake.open_esm_datastore(url)


In [43]: cat.df.source_id.unique()
Out[43]: array(['IPSL-CM6A-LR', 'CanESM5'], dtype=object)

In [42]: cat.search(source_id='^((?!IPSL-CM6A-LR).)*$').df.source_id.unique()
Out[42]: array(['CanESM5'], dtype=object)

'^((?!IPSL-CM6A-LR).)*$' excludes all rows containing IPSL-CM6A-LR in source_id from search results. I hope this helps.

[r-ford]

That worked, thanks @andersy005!