Skip to content

Latest commit

 

History

History
74 lines (55 loc) · 16.9 KB

README.md

File metadata and controls

74 lines (55 loc) · 16.9 KB
Raw

UPerNet

Unified Perceptual Parsing for Scene Understanding

Introduction

Official Repo

Code Snippet

Abstract

Humans recognize the visual world at multiple levels: we effortlessly categorize scenes and detect objects inside, while also identifying the textures and surfaces of the objects along with their different compositional parts. In this paper, we study a new task called Unified Perceptual Parsing, which requires the machine vision systems to recognize as many visual concepts as possible from a given image. A multi-task framework called UPerNet and a training strategy are developed to learn from heterogeneous image annotations. We benchmark our framework on Unified Perceptual Parsing and show that it is able to effectively segment a wide range of concepts from images. The trained networks are further applied to discover visual knowledge in natural scenes. Models are available at this https URL.

Citation

@inproceedings{xiao2018unified,
  title={Unified perceptual parsing for scene understanding},
  author={Xiao, Tete and Liu, Yingcheng and Zhou, Bolei and Jiang, Yuning and Sun, Jian},
  booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
  pages={418--434},
  year={2018}
}

Results and models

Cityscapes

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) mIoU mIoU(ms+flip) config download
UPerNet R-18 512x1024 40000 4.8 4.47 75.39 77.0 config model |log
UPerNet R-50 512x1024 40000 6.4 4.25 77.10 78.37 config model | log
UPerNet R-101 512x1024 40000 7.4 3.79 78.69 80.11 config model | log
UPerNet R-50 769x769 40000 7.2 1.76 77.98 79.70 config model | log
UPerNet R-101 769x769 40000 8.4 1.56 79.03 80.77 config model | log
UPerNet R-18 512x1024 80000 - - 76.02 77.38 config model | log
UPerNet R-50 512x1024 80000 - - 78.19 79.19 config model | log
UPerNet R-101 512x1024 80000 - - 79.40 80.46 config model | log
UPerNet R-50 769x769 80000 - - 79.39 80.92 config model | log
UPerNet R-101 769x769 80000 - - 80.10 81.49 config model | log

ADE20K

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) mIoU mIoU(ms+flip) config download
UPerNet R-18 512x512 80000 6.6 24.76 38.76 39.81 config model | log
UPerNet R-50 512x512 80000 8.1 23.40 40.70 41.81 config model | log
UPerNet R-101 512x512 80000 9.1 20.34 42.91 43.96 config model | log
UPerNet R-18 512x512 160000 - - 39.23 39.97 config model | log
UPerNet R-50 512x512 160000 - - 42.05 42.78 config model | log
UPerNet R-101 512x512 160000 - - 43.82 44.85 config model | log

Pascal VOC 2012 + Aug

Method Backbone Crop Size Lr schd Mem (GB) Inf time (fps) mIoU mIoU(ms+flip) config download
UPerNet R-18 512x512 20000 4.8 25.80 72.9 74.71 config model | log
UPerNet R-50 512x512 20000 6.4 23.17 74.82 76.35 config model | log
UPerNet R-101 512x512 20000 7.5 19.98 77.10 78.29 config model | log
UPerNet R-18 512x512 40000 - - 73.71 74.61 config model | log
UPerNet R-50 512x512 40000 - - 75.92 77.44 config model | log
UPerNet R-101 512x512 40000 - - 77.43 78.56 config model | log