Index is created without a timestamp, when created by scout model observer

[m-lotze]

Describe the bug
If you are not using scout:import before you index the first model, the index is created without a timestamp. This leads to an invalid alias name error, if you then want to use scout:import:

{"error":{"root_cause":[{"type":"invalid_alias_name_exception","reason":"Invalid alias name [...]: an index or data stream exists with the same name as the alias"}],"type":"invalid_alias_name_exception","reason":"Invalid alias name [...]: an index or data stream exists with the same name as the alias"},"status":400} {"exception":"[object] (Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"invalid_alias_name_exception\",\"reason\":\"Invalid alias name [...]: an index or data stream exists with the same name as the alias\"}],\"type\":\"invalid_alias_name_exception\",\"reason\":\"Invalid alias name [...]: an index or data stream exists with the same name as the alias\"},\"status\":400}

To Reproduce
Steps to reproduce the behavior:

1. Create a searchable model e.g. with a model factory and save it, so the scout observer gets triggered and the index is created
2. list elasticsearch indexes with localhost:9200/_cat/indices
3. there should be an index with the searchableAs name without an appended timestamp
4. try to scout:import App\Model
5. the error above should be logged

Expected behavior
The index should always be created with a timestamp appended.

Version
Laravel: 9.13.0
Scout: 9.4.9
Matchish\Elasticsearch: 5.0.2

[hkulekci]

Thinking aloud;

I checked the how that part is working, and I see that import command using the ImportStages class is creating an index name with Index::fromSource method as below :

    public static function fromSource(ImportSource $source): Index
    {
        $name = $source->searchableAs().'_'.time();
        $settingsConfigKey = "elasticsearch.indices.settings.{$source->searchableAs()}";
        $mappingsConfigKey = "elasticsearch.indices.mappings.{$source->searchableAs()}";
        $defaultSettings = [
            'number_of_shards' => 1,
            'number_of_replicas' => 0,

        ];
        $settings = config($settingsConfigKey, config('elasticsearch.indices.settings.default', $defaultSettings));
        $mappings = config($mappingsConfigKey, config('elasticsearch.indices.mappings.default'));

        return new static($name, $settings, $mappings);
    }

So index name getting a suffix in this method is expected.
But the following method is used when I try to index data by updating the object of the model or creating something :

    public function update($models)
    {
        $params = new Bulk();
        $params->index($models);
        $response = $this->elasticsearch->bulk($params->toArray())->asArray();
        if (array_key_exists('errors', $response) && $response['errors']) {
            $error = new ServerResponseException(json_encode($response, JSON_PRETTY_PRINT));
            throw new \Exception('Bulk update error', $error->getCode(), $error);
        }
    }

Inside this method, inside the Bulk class, the index name using as products directly. So, Elasticsaerch creates the index with model name instantly without suffix and alias when we are trying to use model create or update events. I thought that we could check whether or not we have an index or alias related to this model, and we can create an index before doing any update operation with the previous method. This makes every update or creation event a little bit slower. What do you think? Do you have any other suggestions?

[matchish]

Hey) Could you create a test that reproduce the issue? I think it can be possible when we do update, and before we didn't do an import by using the command. I would like to not do extra checks before update, but let's think what we can do. It would be great if we can put a request to a bulk that will create an index with alias if it's not exists

[hkulekci]

Sure, I will write a test for the issue.

[hkulekci]

@matchish you can check the problem with the test.

[matchish]

https://stackoverflow.com/questions/54690343/elasticsearch-when-inserting-a-record-to-index-i-dont-want-to-create-an-index

What do you think if we add this parameter as default settings to a cluster? It will prevent the bug in default configuration. There is another case when user don't have access to the cluster. But until we don't have a feedback that devs actually have this case, I think we can postpone implementing more sophisticated solution. At least for now I don't see what we can do without adding extra check.

[hkulekci]

For now, I could not find any other solution without adding the extra check, too. Maybe, we need to sleep on this a little bit more :D

[Firman95]

Any update about this issue? I still got the error using version 7.6. Is there a workaround that we can do?

[matchish]

@Firman95 would it solve the issue for you?
https://stackoverflow.com/q/54690343