This repository was implemented based on the DeLF paper, and it was mainly focused on the parts that was specified in the paper but not implemented in the tensorflow/delf repository. We recommend you to follow those paper and repository if you are not familiar with DeLF.
You can implement your own CBIR(Content-based image retrieval) system based on the DeLF model with this repository. This repository provides two cases depending on whether you are learning the model yourself or not. To build a custom retrieval system specific to your custom dataset, you will need to train your own model along with following instructions. However, if the tensorflow hub model learned in the Landmark dataset is sufficient to your system, it is not necessary to learn (Then, you will need to tune some parameters to use tensorflow hub model).
(you can also get this content in section 4 of the paper)
- Descriptor Fine-tuning (Section 4.1)
- Attention-based Training (Section 4.2)
- Retrieval System (Section 4.4)
By importing DelfV1 class from tensorflow/delf repository, we implemented training code for the case of using custom dataset. You can follow above 2 trianing steps (Descriptor Fine-tuning & Attention-based Training) by following this repository for your custom model.
When you are planning to implement an image retrieval system, Matching image one by one is not enough. Thus, as specified in the paper, we implemented Product Quantization to aggregate and retrieve images from a large number of features. It could be possible through facebook's faiss library.
- Large-Scale Image Retrieval with Attentive Deep Local Features
- tensorflow/models/research/delf
- facebookresearch/faiss
- tfhub/delf
- Hyeonseo Nam / @gud2great
- Yunsoo Jung / @yunsu3042