[BUG] AttributeError: 'StaticDiskIndex' object has no attribute 'query'

[ucas010]

Expected Behavior

Tell us what should happen

Actual Behavior

Tell us what happens instead

Example Code

refer the python version code

my_query_vector = np.random.random((1,768)).astype(np.float32)
internal_indices, distances = index.query(
    query=my_query_vector,
    k_neighbors=25,
    complexity=50,  # must be as big or bigger than `k_neighbors`
)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'StaticDiskIndex' object has no attribute 'query'
>>> index
<diskannpy._static_disk_index.StaticDiskIndex object at 0x7f1d9d1fceb0>
>>> index.
index.batch_search(  index.search(  

Dataset Description

Please tell us about the shape and datatype of your data, (e.g. 128 dimensions, 12.3 billion points, floats)

• Dimensions: 768
• Number of Points: 10000
• Data type: np.float32

Error

Paste the full error, with any sensitive information minimally redacted and marked $$REDACTED$$
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'StaticDiskIndex' object has no attribute 'query'

Your Environment

• Operating system , Ubuntu 20.04.1 LTS)
• DiskANN version (or commit built from pip install ,v=0.7.0

Additional Details

Any other contextual information you might feel is important.

[ucas010]

急缺一个能够参考落地的example，不然还是束之高阁。
有没有大佬给打个样？

[ucas010]

100万数据，768维度，
done.
Loaded PQ Pivots: #ctrs: 256, #dims: 768, #chunks: 512
Loaded PQ centroids and in-memory compressed vectors. #points: 1000000 #dim: 768 #aligned_dim: 768 #chunks: 512
Disk-Index File Meta-data: # nodes per sector: 1, max node len (bytes): 3332, max node degree: 64
Opened file : /tmp/my_index/ann_disk.index
Setting up thread-specific contexts for nthreads: 44
allocating ctx: 0x7f03a4a27000 to thread-id:139655923397248
allocating ctx: 0x7f03a4a16000 to thread-id:139652512578816
allocating ctx: 0x7f03a4a05000 to thread-id:139653733693440
allocating ctx: 0x7f03a49f4000 to thread-id:139650643506944
allocating ctx: 0x7f03a49e3000 to thread-id:139653725296768
allocating ctx: 0x7f03a49d2000 to thread-id:139653370743552
allocating ctx: 0x7f03a49c1000 to thread-id:139652495785472
allocating ctx: 0x7f03a49b0000 to thread-id:139653742090112
allocating ctx: 0x7f03a499f000 to thread-id:139651734040320
allocating ctx: 0x7f03a498e000 to thread-id:139653353950208
allocating ctx: 0x7f03a497d000 to thread-id:139650626705408
allocating ctx: 0x7f03a496c000 to thread-id:139652504182144
allocating ctx: 0x7f03a495b000 to thread-id:139650635106176
allocating ctx: 0x7f03a494a000 to thread-id:139652520975488
allocating ctx: 0x7f03a4939000 to thread-id:139650618304640
allocating ctx: 0x7f03a4928000 to thread-id:139653362346880
allocating ctx: 0x7f03a4917000 to thread-id:139651717246976
allocating ctx: 0x7f03a4906000 to thread-id:139653716900096
allocating ctx: 0x7f03a48f5000 to thread-id:139652487388800
allocating ctx: 0x7f03a48e4000 to thread-id:139651708850304
io_setup() failed with EAGAIN: Consider increasing /proc/sys/fs/aio-max-nr
Aborted (core dumped)
报错了，采用的上述参考代码。有大佬解释下么？

[daxpryce]

This is my fault - the docs should say index.search | index.batch_search not index.query | index.batch_query. I'll try to get these cleaned up asap for everyone!

[daxpryce]

100万数据，768维度， done. Loaded PQ Pivots: #ctrs: 256, #dims: 768, #chunks: 512 Loaded PQ centroids and in-memory compressed vectors. #points: 1000000 #dim: 768 #aligned_dim: 768 #chunks: 512 Disk-Index File Meta-data: # nodes per sector: 1, max node len (bytes): 3332, max node degree: 64 Opened file : /tmp/my_index/ann_disk.index Setting up thread-specific contexts for nthreads: 44 allocating ctx: 0x7f03a4a27000 to thread-id:139655923397248 allocating ctx: 0x7f03a4a16000 to thread-id:139652512578816 allocating ctx: 0x7f03a4a05000 to thread-id:139653733693440 allocating ctx: 0x7f03a49f4000 to thread-id:139650643506944 allocating ctx: 0x7f03a49e3000 to thread-id:139653725296768 allocating ctx: 0x7f03a49d2000 to thread-id:139653370743552 allocating ctx: 0x7f03a49c1000 to thread-id:139652495785472 allocating ctx: 0x7f03a49b0000 to thread-id:139653742090112 allocating ctx: 0x7f03a499f000 to thread-id:139651734040320 allocating ctx: 0x7f03a498e000 to thread-id:139653353950208 allocating ctx: 0x7f03a497d000 to thread-id:139650626705408 allocating ctx: 0x7f03a496c000 to thread-id:139652504182144 allocating ctx: 0x7f03a495b000 to thread-id:139650635106176 allocating ctx: 0x7f03a494a000 to thread-id:139652520975488 allocating ctx: 0x7f03a4939000 to thread-id:139650618304640 allocating ctx: 0x7f03a4928000 to thread-id:139653362346880 allocating ctx: 0x7f03a4917000 to thread-id:139651717246976 allocating ctx: 0x7f03a4906000 to thread-id:139653716900096 allocating ctx: 0x7f03a48f5000 to thread-id:139652487388800 allocating ctx: 0x7f03a48e4000 to thread-id:139651708850304 io_setup() failed with EAGAIN: Consider increasing /proc/sys/fs/aio-max-nr Aborted (core dumped) 报错了，采用的上述参考代码。有大佬解释下么？

Regarding the error you're getting - check to see what your aio-max-nr is set to by default. According to the internet, 2^16 is likely the default. That said, you're allocating 44 threads for this - and I would be absolutely shocked if you were using 2^16 for just 44 threads. So I'm not sure why this is happening unless you have a ton of open indices at the same time.

sudo su -
cat /proc/sys/fs/aio-max-nr # this will tell you what it's currently set to
echo 655360 > /proc/sys/fs/aio-max-nr # this will increase the possible amount of asynchronous AIO requests

I would try to track down why so many are being used - and I'm not entirely sure how to do that. Increasing the total amount your system will allow may make it so that you can finish your test, but you will probably want to try to track down why so many async requests are being opened at the same time. This stack overflow may be of utility: https://stackoverflow.com/questions/28999765/how-does-the-linux-kernel-handle-asynchronous-i-o-aio-requests - specifically using iostat -x -d 1 and looking for the avgqu-sz and util % columns. To be honest I can't make out what I'm really looking at on my own machine, so I don't know how helpful it actually is.