Partial matching of strings in generateScientificExpression

[linupi]

Port of SciCatProject/backend-v3#680 to the new backend!

Description

This PR adds an additional CONTAINS_STRING condition to the generateScientificExpression function to filter within strings in scientific metadata.

Motivation

When defining our meta data schemas we realized that it would be very helpful for us to also filter on partially matching strings in the scientific metadata. However, so far there is only EQUAL_TO_STRING for text based entries in scientific metadata so we propose an additional CONTAINS_STRING which should work on simple strings as well as on lists. Here is an example:

"scientificMetadata": {
        "localContact": "John Doe",
        "experimentalists": [
                "John Doe",
                "Max Mustermann"
         ]

The idea of the proposed CONTAINS_STRING is that it would match e.g. John in both cases (as localContact as well as in experimentalists)

Changes:

• additinal condition for generateScientificExpression in mongo-queryable.js
• the related frontend changes can be found in Additinal CONTAINS_STRING option for characteristic filter frontend#974

Tests included/Docs Updated?

• Included for each change/fix?
• Passing? (Merge will not be approved unless this is checked)
• Docs updated?
• New packages used/requires npm install?
• Toggle added for new features?

[linupi]

@henrikjohansson712 : compared to the old backend the generation of the ScientificExpression looks much simpler.

Does the new backend impose to always have value as item for every entry in the scientific metadata, or did I get a wrong impression there? (a.k.a are simple entries without value/unit no longer supported through the new backend?)

[stephan271]

I dont know what is currently implemented, but it must be possible to define data without the value/unit Syntax. If its not supported, it must be added.

[nitrosx]

@stephan271 can you please provide an example of a scientificMetadata without value/unit?

[stephan271]

[nitrosx]

At ESS, we always have value and unit.
Regarding this PR, we should expect both cases.
Separately, we should open a new issue to check the scientific metadata that should allow both cases

[nguyenlinhlinh]

At MAXIV we also only have unit and value for physical quantities. Others will only have value.

[nitrosx]

@linupi are you able to update the code to include the case when the scientific metadata entry is a string and not a object with fields value and unit?

[linupi]

@linupi are you able to update the code to include the case when the scientific metadata entry is a string and not a object with fields value and unit?

I could for sure adopt the code proposed in here, but I am wondering if it is not a more general issue since the existence of a .value entry in the scientific metadata is assumed at different places in the code.
e.g.

src/common/utils.ts:20:    return { valueSI: Number(quantity.value), unitSI: quantity.unit };
src/common/utils.ts:92:    const matchKeyMeasurement = `scientificMetadata.${lhs}.valueSI`;
src/common/utils.ts:97:        scientificFilterQuery[`${matchKeyGeneric}.value`] = { $eq: rhs };
src/common/utils.ts:102:        scientificFilterQuery[`${matchKeyGeneric}.value`] = { $eq: escapedRhs };
src/common/utils.ts:111:          scientificFilterQuery[`${matchKeyGeneric}.value`] = { $eq: rhs };
src/common/utils.ts:121:          scientificFilterQuery[`${matchKeyGeneric}.value`] = { $gt: rhs };
src/common/utils.ts:131:          scientificFilterQuery[`${matchKeyGeneric}.value`] = { $lt: rhs };
src/datasets/interceptors/fullquery.interceptor.ts:34:                `${lhs}.value`,
src/datasets/interceptors/fullquery.interceptor.ts:46:                  `${lhs}.value`,

wouldn't it make sense to provide a more generic helper to deal with the different options of having .value, .value+.unit or none of them? ... in that case I wouldn't be the best person to implement that I think.

[nitrosx]

@linupi should I merge this branch in to master?

[linupi]

@linupi should I merge this branch in to master?

@nitrosx no not yet. First we have to solve the issue discussed above (#138) and after that I will have to adopt this code again. Help with #138 would be much appreciated.

[nitrosx]

We should close this PR as I included the changes in #1293.