Abstract

In this work we target the problem of estimating accurately localised correspondences between a pair of images. We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems and propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than $10\times$, with equivalent results. This is achieved by sparsifying the correlation tensor containing tentative matches, and its subsequent processing with a 4D CNN using submanifold sparse convolutions. Localisation accuracy is significantly improved by processing the input images in higher resolution, which is possible due to the reduced memory footprint, and by a novel two-stage correspondence relocalisation module. The proposed Sparse-NCNet method obtains state-of-the-art results on the HPatches Sequences and InLoc visual localisation benchmarks, and competitive results in the Aachen Day-Night benchmark.

Paper

		I. Rocco, R. Arandjelović and J. Sivic Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions European Conference on Computer Vision, 2020 [Paper on arXiv]

BibTeX

@inproceedings{Rocco20,
        author       = "Rocco, I. and Arandjelovi\'c, R. and Sivic, J.",
        title        = "Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions",
        booktitle    = "European Conference on Computer Vision",
        year         = 2020,
        }

Code

Please see the Github repo here: https://github.com/ignacio-rocco/sparse-ncnet.

Acknowledgements

This work was partially supported by ERC grant LEAP No. 336845, the European Regional Development Fund under project IMPACT (reg. no. CZ.02.1.01/0.0/0.0/15 003/0000468), Louis Vuitton ENS Chair on Artificial Intelligence, and the French government under management of Agence Nationale de la Recherche as part of the "Investissements d'avenir" program, reference ANR-19-P3IA-0001 (PRAIRIE 3IA Institute).

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright.