Internal Server Error while searching by text

[vishaal27]

Hi,

I am trying to search the hosted clip-retrieval backend index by text and end up getting an internal server error.
Here is the code I used:

from clip_retrieval.clip_client import ClipClient, Modality
client = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.TEXT)
results = client.query(text="an image of a cat")
print(results)

Output:

{'message': 'Internal Server Error'}

However, when I search by image it seems to be working fine:

from clip_retrieval.clip_client import ClipClient, Modality
client = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.IMAGE)
results = client.query(text="an image of a cat")
print(results[0])

Output:

{'caption': 'orange cat with supicious look stock photo', 'url': 'https://media.istockphoto.com/photos/orange-cat-with-supicious-look-picture-id907595140?k=6&m=907595140&s=612x612&w=0&h=4CTvSxNvv4sxSCPxViryha4kAjuxDbrXM5vy4VPOuzk=', 'id': 518836491, 'similarity': 0.5591729879379272}

Please let me know if I am doing something incorrect, or if direct text-to-text similarity search is not enabled by default in clip-retrieval's client? Thanks :)

[rom1504]

Laion5B text index is not computed indeed but you can use laion400m one it you want What's your use case ?

…

On Mon, Sep 11, 2023, 12:11 Vishaal Udandarao ***@***.***> wrote: Hi, I am trying to search the hosted clip-retrieval backend index by text and end up getting an internal server error. Here is the code I used: from clip_retrieval.clip_client import ClipClient, Modalityclient = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.TEXT)results = client.query(text="an image of a cat")print(results) Output: {'message': 'Internal Server Error'} However, when I search by image it seems to be working fine: from clip_retrieval.clip_client import ClipClient, Modalityclient = ClipClient(url="https://knn.laion.ai/knn-service", indice_name="laion5B-L-14", modality=Modality.IMAGE)results = client.query(text="an image of a cat")print(results[0]) Output: {'caption': 'orange cat with supicious look stock photo', 'url': 'https://media.istockphoto.com/photos/orange-cat-with-supicious-look-picture-id907595140?k=6&amp;m=907595140&amp;s=612x612&amp;w=0&amp;h=4CTvSxNvv4sxSCPxViryha4kAjuxDbrXM5vy4VPOuzk=', 'id': 518836491, 'similarity': 0.5591729879379272} Please let me know if I am doing something incorrect, or if direct text-to-text similarity search is not enabled by default in clip-retrieval's client? Thanks :) — Reply to this email directly, view it on GitHub <#311>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR437RHNIBH4NT5HCC6ITTXZ3IV7ANCNFSM6AAAAAA4S7ZMDY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[vishaal27]

My main usecase: I want to do a text search for certain keywords in all the captions of the LAION-5B dataset, what would you recommend is the fastest way to do this (without locally downloading the parquet files)? I realise the clip index does not do direct text matching but rather cosine similarities. I would still be interested in pure text-to-text matching on the LAION-5B index.
For example, I want to search "gaussian noise" as keyword and check for exact matches / most similar text captions in the LAION-5B dataset.