-
Notifications
You must be signed in to change notification settings - Fork 140
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] KNN filtering + constant_keyword + multiple-index search leads to sporadic and incorrect results #2373
Comments
Can someone from Opensearch core move this issue to k-NN repo. @opensearch-project/admin |
To help make my example more minimal, the following also runs into the same problem (sporadically returning either GET http://localhost:9200/index-1,index-2/_search
Content-Type: application/json
{
"query": {
"knn": {
"embedding": {
"filter": {
"term": {
"info.category": "BAR"
}
},
"k": 30,
"vector": [
0.2,
0.3,
0.4
]
}
}
}
} |
0 hits response (with `profile` enabled)GET http://localhost:9200/index-1,index-2/_search?explain=true
HTTP/1.1 200 OK
TASK_RESOURCE_USAGE: {"action":"indices:data/read/search[phase/query]","taskId":1857,"parentTaskId":1847,"nodeId":"UCeVNJKHT9yggFJSwa_Atw","taskResourceUsage":{"cpu_time_in_nanos":1003083,"memory_in_bytes":23296}}
TASK_RESOURCE_USAGE: {"action":"indices:data/read/search[phase/query]","taskId":1848,"parentTaskId":1847,"nodeId":"UCeVNJKHT9yggFJSwa_Atw","taskResourceUsage":{"cpu_time_in_nanos":5204458,"memory_in_bytes":21312}}
content-type: application/json; charset=UTF-8
content-encoding: gzip
content-length: 1001
{
"took": 32,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"profile": {
"shards": [
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][0]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 1250,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1250
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 792,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 792
}
}
],
"rewrite_time": 754193902759,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 875
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][1]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 1417,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1417
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1250,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1250
}
}
],
"rewrite_time": 754193881467,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 61750
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][2]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 500,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 500
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 667,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 667
}
}
],
"rewrite_time": 754193854635,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 458
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][3]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 22624,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 21666,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 958
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 2166,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 1249,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 917
}
}
],
"rewrite_time": 754199264301,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 2165
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][4]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 56000,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 55333,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 667
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1457,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 1166,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 291
}
}
],
"rewrite_time": 754199294800,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 1877
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][0]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 1292,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1292
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 292,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 292
}
}
],
"rewrite_time": 754193896384,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 957
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][1]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 625,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 625
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 541,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 541
}
}
],
"rewrite_time": 754193826426,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 10709
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][2]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 833,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 833
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1084,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1084
}
}
],
"rewrite_time": 754199175260,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 1083
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][3]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 23583,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 23166,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 417
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1292,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 917,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 375
}
}
],
"rewrite_time": 754199263676,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 2542
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][4]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"User requested \"match_none\" query.\")",
"time_in_nanos": 29249,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 28958,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 291
}
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1791,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 1125,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 2,
"create_weight": 666
}
}
],
"rewrite_time": 754199268009,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 2374
}
]
}
],
"aggregations": []
}
]
}
}
Response file saved.
> 2025-01-10T173218.200.json
Response code: 200 (OK); Time: 78ms (78 ms); Content length: 12694 bytes (12.69 kB) 4 hits response (with `profile` enabled)GET http://localhost:9200/index-1,index-2/_search?explain=true
HTTP/1.1 200 OK
TASK_RESOURCE_USAGE: {"action":"indices:data/read/search[phase/fetch/id]","taskId":1752,"parentTaskId":1740,"nodeId":"UCeVNJKHT9yggFJSwa_Atw","taskResourceUsage":{"cpu_time_in_nanos":2070791,"memory_in_bytes":27320}}
TASK_RESOURCE_USAGE: {"action":"indices:data/read/search[phase/query]","taskId":1747,"parentTaskId":1740,"nodeId":"UCeVNJKHT9yggFJSwa_Atw","taskResourceUsage":{"cpu_time_in_nanos":6382376,"memory_in_bytes":39600}}
TASK_RESOURCE_USAGE: {"action":"indices:data/read/search[phase/query]","taskId":1742,"parentTaskId":1740,"nodeId":"UCeVNJKHT9yggFJSwa_Atw","taskResourceUsage":{"cpu_time_in_nanos":26501416,"memory_in_bytes":56472}}
content-type: application/json; charset=UTF-8
content-encoding: gzip
content-length: 1632
{
"took": 179,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_shard": "[index-1][3]",
"_node": "UCeVNJKHT9yggFJSwa_Atw",
"_index": "index-1",
"_id": "2",
"_score": 1.0,
"_source": {
"embedding": [
0.2,
0.3,
0.4
],
"ids": [
4,
5,
6
]
},
"_explanation": {
"value": 1.0,
"description": "within top 30",
"details": []
}
},
{
"_shard": "[index-1][4]",
"_node": "UCeVNJKHT9yggFJSwa_Atw",
"_index": "index-1",
"_id": "1",
"_score": 0.97087383,
"_source": {
"embedding": [
0.1,
0.2,
0.3
],
"ids": [
1,
2,
3
]
},
"_explanation": {
"value": 0.97087383,
"description": "within top 30",
"details": []
}
},
{
"_shard": "[index-2][4]",
"_node": "UCeVNJKHT9yggFJSwa_Atw",
"_index": "index-2",
"_id": "1",
"_score": 0.97087383,
"_source": {
"embedding": [
0.3,
0.4,
0.5
],
"ids": [
7,
8,
9
]
},
"_explanation": {
"value": 0.97087383,
"description": "within top 30",
"details": []
}
},
{
"_shard": "[index-2][3]",
"_node": "UCeVNJKHT9yggFJSwa_Atw",
"_index": "index-2",
"_id": "2",
"_score": 0.89285713,
"_source": {
"embedding": [
0.4,
0.5,
0.6
],
"ids": [
10,
11,
12
]
},
"_explanation": {
"value": 0.89285713,
"description": "within top 30",
"details": []
}
}
]
},
"profile": {
"shards": [
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][0]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 1516875,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1516875
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 958,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 958
}
}
]
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1625,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1625
}
}
],
"rewrite_time": 706793748656,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 6084
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][1]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 1609375,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1609375
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 1667,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1667
}
}
]
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 916,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 916
}
}
],
"rewrite_time": 706793794196,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 1542
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][2]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 1610541,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1610541
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 29500,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 29500
}
}
]
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1625,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1625
}
}
],
"rewrite_time": 706793822280,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 1375
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][3]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 1960456,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 9958,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 1936332,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 14166
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 22333,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 7959,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 11541,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 2833
}
}
]
},
{
"type": "DocAndScoreQuery",
"description": "DocAndScore[30]",
"time_in_nanos": 8792,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 1417,
"score_count": 1,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 1542,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 4375,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 2,
"compute_max_score": 0,
"build_scorer_count": 3,
"create_weight": 1458
}
}
],
"rewrite_time": 706828599614,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 19458
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-1][4]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 2073914,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 10874,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 2059499,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 3541
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 71250,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 8792,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 61792,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 666
}
}
]
},
{
"type": "DocAndScoreQuery",
"description": "DocAndScore[30]",
"time_in_nanos": 8291,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 1166,
"score_count": 1,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 584,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 5208,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 2,
"compute_max_score": 0,
"build_scorer_count": 3,
"create_weight": 1333
}
}
],
"rewrite_time": 706828572197,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 6126
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][0]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 1590542,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1590542
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 23958,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 23958
}
}
]
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 833,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 833
}
}
],
"rewrite_time": 706793764279,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 2666
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][1]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 1587625,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1587625
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 29750,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 29750
}
}
]
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 958,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 958
}
}
],
"rewrite_time": 706793740071,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 792
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][2]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 10667,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 10667
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 2041,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 2041
}
}
]
},
{
"type": "MatchNoDocsQuery",
"description": "MatchNoDocsQuery(\"\")",
"time_in_nanos": 1167,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 0,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 0,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 0,
"compute_max_score": 0,
"build_scorer_count": 0,
"create_weight": 1167
}
}
],
"rewrite_time": 706821589114,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 2042
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][3]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 2125710,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 11792,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 2107960,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 5958
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 50250,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 8875,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 40500,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 875
}
}
]
},
{
"type": "DocAndScoreQuery",
"description": "DocAndScore[30]",
"time_in_nanos": 6916,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 791,
"score_count": 1,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 333,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 3209,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 2,
"compute_max_score": 0,
"build_scorer_count": 3,
"create_weight": 2583
}
}
],
"rewrite_time": 706828550738,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 5166
}
]
}
],
"aggregations": []
},
{
"id": "[UCeVNJKHT9yggFJSwa_Atw][index-2][4]",
"inbound_network_time_in_millis": 0,
"outbound_network_time_in_millis": 0,
"searches": [
{
"query": [
{
"type": "BoostQuery",
"description": "(ConstantScore(*:*))^0.0",
"time_in_nanos": 2080833,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 11708,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 2060458,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 8667
},
"children": [
{
"type": "MatchAllDocsQuery",
"description": "*:*",
"time_in_nanos": 50167,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 9376,
"score_count": 0,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 0,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 39333,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 4,
"compute_max_score": 0,
"build_scorer_count": 4,
"create_weight": 1458
}
}
]
},
{
"type": "DocAndScoreQuery",
"description": "DocAndScore[30]",
"time_in_nanos": 24543,
"breakdown": {
"set_min_competitive_score_count": 0,
"match_count": 0,
"shallow_advance_count": 0,
"next_doc": 1167,
"score_count": 1,
"compute_max_score_count": 0,
"advance": 0,
"advance_count": 0,
"score": 1542,
"shallow_advance": 0,
"create_weight_count": 1,
"build_scorer": 19626,
"set_min_competitive_score": 0,
"match": 0,
"next_doc_count": 2,
"compute_max_score": 0,
"build_scorer_count": 3,
"create_weight": 2208
}
}
],
"rewrite_time": 706828482655,
"collector": [
{
"name": "SimpleTopScoreDocCollector",
"reason": "search_top_hits",
"time_in_nanos": 18292
}
]
}
],
"aggregations": []
}
]
}
}
Response file saved.
> 2025-01-10T173131.200.json
Response code: 200 (OK); Time: 217ms (217 ms); Content length: 17974 bytes (17.97 kB) |
Unable to replicate when running against main branch of k-NN repo. I think it may have been fixed by this PR. |
@jackh-ncl thanks for update. @buddharajusahil thanks for fixing multiple bugs with 1 PR. Closing this GH issue. |
Describe the bug
Filtering on a constant keyword isn't working as expected within a KNN filter clause. It sporadically gives different responses if you keep firing the request. Steps to reproduce below.
Related component
Search
To Reproduce
Index setup
bulk-index.ndjson
Search query
Results
Result 1:
Result 2:
Expected behavior
If I recreate the indices without specifying
constant_keyword
, I get the results I would expect (with 2 hits) as below:Reindexed data (with category as a plain `keyword` type)
Results
Additional Details
Plugins
n/a
Screenshots
n/a
Host/Environment (please complete the following information):
AWS OpenSearch (v2.17) and I was able to replicate the above against a v2.18 Docker container.
docker-compose service
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: