Efficient image retrieval using multi neural hash codes and bloom filters

  • Sourin Chakrabarti
Keywords: Neural hash codes, Bloom filters, Convolutional neural networks.

Abstract

This paper aims to deliver an efficient and modified approach for image retrieval using multiple neural hash codes and limiting the number of queries using bloom filters by identifying false positives beforehand. Traditional approaches involving neural networks for image retrieval tasks tend to use higher layers for feature extraction. But it has been seen that the activations of lower layers have proven to be more effective in a number of scenarios. In our approach, we have leveraged the use of local deep convolutional neural networks which combines the powers of both the features of lower and higher layers for creating feature maps which are then compressed using PCA and fed to a bloom filter after binary sequencing using a modified multi k-means approach. The feature maps obtained are further used in the image retrieval process in a hierarchical coarse-to-fine manner by first comparing the images in the higher layers for semantically similar images and then gradually moving towards the lower layers searching for structural similarities. While searching, the neural hashes for the query image are again calculated and queried in the bloom filter which tells us whether the query image is absent in the set or maybe present. If the bloom filter doesn't necessarily rule out the query, then it goes into the image retrieval process. This approach can be particularly helpful in cases where the image store is distributed since the approach supports parallel querying.

References

[1]alvi, A., Ercoli, S., Bertini, M. and Del Bimbo, A., 2016, December. Bloom filters and compact hash codes for efficient and distributed image retrieval. In 2016 IEEE International Symposium on Multimedia (ISM) (pp. 515-520). IEEE.
[2]Babenko, A., Slesarev, A., Chigorin, A. and Lempitsky, V., 2014, September. Neural codes for image retrieval. In European conference on computer vision (pp. 584-599). Springer, Cham.
[3]Yu, W., Sun, X., Yang, K., Rui, Y. and Yao, H., 2018. Hierarchical semantic image matching using CNN feature pyramid. Computer Vision and Image Understanding, 169, pp.40-51.
[4]Yue-Hei Ng, J., Yang, F. and Davis, L.S., 2015. Exploiting local features from deep networks for image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp.53-61).
[5]Ercoli, S., Bertini, M. and Del Bimbo, A., 2015, June. Compact hash codes and data structures for efficient mobile visual search. In 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW) (pp. 1-6). IEEE.
[6]Babenko, A. and Lempitsky, V., 2015. Aggregating deep convolutional features for image retrieval. arXiv preprint arXiv:1510.07493.
[7]Radenovi ́c, F., Tolias, G. and Chum, O., 2018. Fine-tuning CNN image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence, 41(7), pp.1655-1668.
[8]Rae, J.W., Bartunov, S. and Lillicrap, T.P., 2019. Meta-learning neural Bloom filters. arXiv preprint arXiv:1906.04304.
[9]Araujo, A., Chaves, J., Lakshman, H., Angst, R. and Girod, B., 2016. Large-scale query-by-image video retrieval using bloom filters. arXivpreprint arXiv:1604.07939.
[10]Xia, Z., Feng, X., Lin, J. and Hadid, A., 2017. Deep convolutional hashing using pairwise multi-label supervision for large-scale visualsearch. Signal Processing: Image Communication, 59, pp.109-116.
[11]Uricchio, T., Bertini, M., Seidenari, L. and Bimbo, A., 2015. Fisher encoded convolutional bag-of-windows for efficient image retrieval and social image tagging. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 9-15).
[12]J ́egou, H. and Zisserman, A., 2014. Triangulation embedding and democratic aggregation for image search. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3310-3317).
[13]Sharif Razavian, A., Azizpour, H., Sullivan, J. and Carlsson, S., 2014. CNN features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 806-813).
[14]Inoue, K. and Kise, K., 2009, September. Compressed representation of feature vectors using a Bloomier filter and its application to specific object recognition. In 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops (pp. 2133-2140).
[15]Bloom, B.H., 1970. Space/time trade-offs in hash coding with allowable errors. Communications of the ACM, 13(7), pp.422-426.
[16]Arandjelovic, R. and Zisserman, A., 2013. All about VLAD. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 1578-1585).
[17]Lowe, D.G., 2004. Distinctive image features from scale-invariant key-points. International journal of computer vision, 60(2), pp.91-110.
[18]Philbin, J., Chum, O., Isard, M., Sivic, J. and Zisserman,A., 2007, June. Object retrieval with large vocabularies and fast spatial matching. In 2007 IEEE conference on computer vision and pattern recognition (pp. 1-8). IEEE.
[19]Philbin, J., Chum, O., Isard, M., Sivic, J. and Zisserman,A., 2008, June. Lost in quantization: Improving particular object retrieval in large scale image databases. In 2008 IEEE conference on computer vision and pattern recognition (pp. 1-8). IEEE.
[20]Jegou, H., Douze, M. and Schmid, C., 2008, October. Hammingem-bedding and weak geometric consistency for large scale image search. In European conference on computer vision (pp. 304-317). Springer, Berlin, Heidelberg.
[21]LeCun, Y., Boser, B.E., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W.E. and Jackel, L.D., 1990. Handwritten digit recognition with a back-propagation network. In Advances in neural information processing systems (pp. 396-404).
[22]Jogin, M., Madhulika, M.S., Divya, G.D., Meghana, R.K.and Apoorva, S., 2018, May. Feature extraction using Convolution Neural Networks (CNN) and Deep Learning. In 2018 3rd IEEE International Conferenceon Recent Trends in Electronics, Information & Communication Technology (RTEICT) (pp. 2319-2323). IEEE.
[23]Ng, S.C., 2017. Principal component analysis to reduce dimension on digital image. Procedia computer science, 111, pp.113-119.
[24]S ́anchez, J., Perronnin, F., Mensink, T. and Verbeek, J., 2013. Image Classification with the fisher vector: Theory and practice. International Journal of computer vision, 105(3), pp.222-245.
[25]Yang, L. and Jin, R., 2006. Distance metric learning: A comprehensive survey. Michigan State University, 2(2), p.4.
[26]Gao, L., Song, J., Zou, F., Zhang, D. and Shao, J., 2015, October. Scalable multimedia retrieval by deep learning hashing with relative similarity learning. In Proceedings of the 23rd ACM international conference on Multimedia (pp. 903-906).
[27]Lin, K., Yang, H.F., Hsiao, J.H. and Chen, C.S., 2015. Deep learning binary hash codes for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp.27-35).
[28]Sun, J., Shum, H.Y. and Zheng, N.N., 2002, May. Stereo matching using belief propagation. In European Conference on Computer Vision (pp.510-524). Springer, Berlin, Heidelberg.
[29]Boykov, Y., Veksler, O. and Zabih, R., 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on pattern analysis and machine intelligence, 23(11), pp.1222-1239.
[30]Kanazawa, A., Jacobs, D.W. and Chandraker, M., 2016. Warpnet: Weakly supervised matching for single-view reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3253-3261).
[31]Barnes, C., Shechtman, E., Goldman, D.B. and Finkelstein, A., 2010, September. The generalized patchmatch correspondence algorithm. In European Conference on Computer Vision (pp. 29-43). Springer, Berlin, Heidelberg.
[32]Danielsson, O., 2015, June. Category-sensitive hashing and Bloom filter based descriptors for online keypoint recognition. In Scandinavian Conference on Image Analysis (pp. 329-340). Springer, Cham.
[33]Appleby, A., 2011. Murmur3 hash function.
[34]Berg, A. and Deng, J., 2010. and L Fei-Fei. Large scale visual recognition challenge(ILSVRC).
[35]Babenko, A. and Lempitsky, V., 2014. The inverted multi-index. IEEE transactions on pattern analysis and machine intelligence, 37(6), pp.1247-1260
Published
2020-12-15
How to Cite
Chakrabarti, S. (2020). Efficient image retrieval using multi neural hash codes and bloom filters. Asian Journal For Convergence In Technology (AJCT) ISSN -2350-1146, 6(3), 16-21. https://doi.org/10.33130/AJCT.2020v06i03.004

Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.