[Azure Search] Remove field value from index when not present in data source

[Choc13]

Query/Question
We are using Azure Search to index blobs and their metadata. Some of the blob metadata is optional and occasionally users of our application would like to remove these metadata values. However, even though the blob is re-indexed the value is not removed from the corresponding search index document. I understand that by default the indexer works in a create of update mode, whereby it will only add or update fields that are present in the data source, rather than removing those that are missing. Is there a way to alter this behaviour so that it will always ensure that the document in the search index exactly matches the data in the data source?

Note that this is further compounded by the fact that the Blob Store .NET SDK throws an exception when a metadata field is set to empty string or null as highlighted in this issue. Also, I would like to avoid any solutions that involve having to call specific index operations on a per document basis to deal with this edge case.

Why is this not a Bug or a feature Request?
As mentioned above, I understand that this is the default indexer behaviour and I hope that there is already a way to work around this without having to request new features.

Setup (please complete the following information if applicable):

• OS: Windows
• IDE : Visual Studio
• Version of the Library used: 9.0.1.

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [ x] Query Added
• [ x] Setup information Added

[arv100kri]

I understand that by default the indexer works in a create of update mode, whereby it will only add or update fields that are present in the data source, rather than removing those that are missing.

"mergeOrUpload" is the current behavior that indexers operate by, which is also something that you've already highlighted previously. We currently do not have a mechanism for indexers to change that mode and I wouldn't be able to provide a reasonable alternative at this time.

Also, I would like to avoid any solutions that involve having to call specific index operations on a per document basis to deal with this edge case.

Unfortunately, that's the only possible work around at this current moment - if you are able to identify the documents by their key, you can use the API to modify the specific document.

Is there a way to alter this behaviour so that it will always ensure that the document in the search index exactly matches the data in the data source

I will look into finding if there is something I can do to provide a different work around, but I do not believe it will be possible in the near future. I am going to keep this bug open however, to help us track a future possible option for indexers to try to achieve that.

[jongio]

The Azure Search team requested that we move this to User Voice. They have also created an internal tracking work item for this.

https://feedback.azure.com/forums/263029-azure-search/suggestions/42514306-allow-for-metadata-that-disappeared-from-the-origi