From fe45fd89fa2adf17c6dd352b29339defd3d189a6 Mon Sep 17 00:00:00 2001 From: Alvaro Vega Date: Wed, 29 Nov 2023 10:21:25 +0100 Subject: [PATCH 1/3] add a note about index recreation --- doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md b/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md index 1bd8299e4..dfa178eae 100644 --- a/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md +++ b/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md @@ -298,6 +298,8 @@ Since version 3.0.0 of cygnus an index named `cyg_raw_opt` is created in this wa Note that attempting to use any other datamodel different from these ones will cause a "Unknown data model" ERROR in log traces. In addition, note that default datamodel for NGSIMongoSink is dm-by-entity as all others sinks which uses datamodel. +When datamodel changes Cygnus tries to recreate index (delete current and create a new one) depending on keys related to datamodel. This procedure could takes much time depending on data volume and existence of mongo cluster deploy. In order to reduce this time, an expected index could be created in advance when cygnus and mongo loads allow it. + ## Administration guide ### Configuration `NGSIMongoSink` is configured through the following parameters: From 06faa9bdf780a8b9822c0349205a3021d742469e Mon Sep 17 00:00:00 2001 From: Alvaro Vega Date: Wed, 29 Nov 2023 13:17:12 +0100 Subject: [PATCH 2/3] Update doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Fermín Galán Márquez --- doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md b/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md index dfa178eae..6f99b5948 100644 --- a/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md +++ b/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_mongo_sink.md @@ -298,7 +298,7 @@ Since version 3.0.0 of cygnus an index named `cyg_raw_opt` is created in this wa Note that attempting to use any other datamodel different from these ones will cause a "Unknown data model" ERROR in log traces. In addition, note that default datamodel for NGSIMongoSink is dm-by-entity as all others sinks which uses datamodel. -When datamodel changes Cygnus tries to recreate index (delete current and create a new one) depending on keys related to datamodel. This procedure could takes much time depending on data volume and existence of mongo cluster deploy. In order to reduce this time, an expected index could be created in advance when cygnus and mongo loads allow it. +When datamodel changes Cygnus tries to recreate index (delete current and create a new one) depending on keys related to datamodel. This procedure could takes much time depending on data volume and existence of mongo cluster deploy. In order to reduce this time, an expected index could be created in advance (i.e. before doing the datamodel change) when cygnus and mongo loads allow it. ## Administration guide ### Configuration From 78758d7f224171b5de86e1b8ae72822c619be7c6 Mon Sep 17 00:00:00 2001 From: Alvaro Vega Date: Wed, 29 Nov 2023 13:18:25 +0100 Subject: [PATCH 3/3] Update ngsi_sth_sink.md --- doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_sth_sink.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_sth_sink.md b/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_sth_sink.md index 346763010..8a54b3043 100644 --- a/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_sth_sink.md +++ b/doc/cygnus-ngsi/flume_extensions_catalogue/ngsi_sth_sink.md @@ -302,6 +302,8 @@ Since version 3.0.0 of cygnus index named `cyg_agg_opt` is created in this way b Note that attempting to use any other datamodel different from these ones will cause a "Unknown data model" ERROR in log traces. In addition, note that default datamodel for NGSISTHSink is dm-by-entity as all others sinks which uses datamodel. +When datamodel changes Cygnus tries to recreate index (delete current and create a new one) depending on keys related to datamodel. This procedure could takes much time depending on data volume and existence of mongo cluster deploy. In order to reduce this time, an expected index could be created in advance (i.e. before doing the datamodel change) when cygnus and mongo loads allow it. + ## Administration guide ### Configuration `NGSISTHSink` is configured through the following parameters: