Figure 1. An example of region-based semantic flow and dense flow field.

Paper

B. Ham, M. Cho, C. Schmid, J. Ponce
Proposal Flow
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
PDF | arXiv | Abstract | BibTeX

Abstract

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that proposal flow can effectively be transformed into a conventional dense flow field. We introduce a new dataset that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use this benchmark to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

BibTeX

@InProceedings{ham2016,
author = {Bumsub Ham and Minsu Cho and Cordelia Schmid and Jean Ponce},
title = {Proposal Flow},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year = {2016}
}
					

B. Ham, M. Cho, C. Schmid, J. Ponce
Proposal Flow: Semantic Correspondences from Object Proposals
IEEE Trans. on Pattern Analysis and Machine Intelligence (2017)
PDF | arXiv | Abstract | BibTeX | More results (21MB)

Abstract

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

BibTeX

@InProceedings{ham2016,
author = {Bumsub Ham and Minsu Cho and Cordelia Schmid and Jean Ponce},
title = {Proposal Flow: Semantic Correspondences from Object Proposals},
booktitle = {arXiv:1703.07144},
year = {2017}
}
					

Data & Code





Acknowledgements

This work was supported by the ERC grants VideoWorld and Allegro, and the Institut Universitaire de France.