Awesome Dynamic Inference

A curated list for Dynamic Inference research in deep learning, including: dynamic routing networks, early prediction networks and conditional computation networks.

Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, awesome-AutoDL and awesome-architecture-search.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

• Awesome Blogs

• Deep Learning-based NAS and HPO

  • 2020 Venues
  • 2019 Venues
  • 2018 Venues
  • 2017 Venues
  • Previous Venues

• Awesome Surveys

Awesome Blogs

• AutoML info and AutoML Freiburg-Hannover
• What’s the deal with Neural Architecture Search?
• Google Could AutoML and PocketFlow
• AutoML Challenge and AutoDL Challenge

Related Awesome Libraies

• Keras Tuner
• NNI
• AutoGluon
• Auto-PyTorch
• AutoDL-Projects
• awesome-AutoDL

Dynamic Inference Network methods

Type	Dynamic Routing (R)	Eraly Prediction (E)	Condictional Computation (C)	Others (O)
Explanation	Skip unnecessary units	Predict with intermediate features		other types

2020

Title	Venue	Type	Code
Learning Dynamic Routing for Semantic Segmentation	CVPR	O	-
ScopeFlow: Dynamic Scene Scoping for Optical Flow	CVPR	C	-
Dynamic Convolution: Attention over Convolution Kernels	CVPR	C	-
Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference	CVPR	C	-
S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search	CVPR	O	-
Dynamic Graph Message Passing Networks	CVPR	C	-
Conditional Channel Gated Networks for Task-Aware Continual Learning	CVPR	R	-
Resolution Adaptive Networks for Efficient Inference	CVPR	O	-
Dynamic Representations Toward Efficient Inference on Deep Neural Networks by Decision Gates	CVPRW	E	-
BATCH-SHAPING FOR LEARNING CONDITIONAL CHANNEL GATED NETWORKS	ICLR	R	-
MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution	ECCV	O	-
CoDiNet: Path Distribution Modeling with Consistency and Diversity for Dynamic Routing	UnKnown	R	-
Fully Dynamic Inference with Deep Neural Networks	UnKnown	R	-
Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach	UnKnown	C	-
Dynamic ReLU	UnKnown	C	-
Computation on sparse neural networks: an inspiration for future hardware	UnKnown	R	-
Spatially adaptive inference with stochastic feature sampling and interpolation	UnKnown	R	-
Multi-scale dense networks for resource efficient image classification	ICLR	E	-
Dynamic group convolution for accelerating convolutional neural networks		R	-
Discriminative layer pruning for convolutional neural networks		R	-
Dynamic region aware convolution		R	-

2019

Title	Venue	Type	Code
BlockDrop: Dynamic Inference Paths in Residual Networks	CVPR	R	-
Improved Techniques for Training Adaptive Deep Networks	ICCV	E	-
Universally slimmable networks and improved training techniques	ICCV	R	-

2018

Title	Venue	Type	Code
SkipNet: Learning Dynamic Routing in Convolutional Networks	ECCV	R	-
Online Multi-Object Tracking with Instance-Aware Tracker and Dynamic Model Refreshment	ECCV	C	-
Convolutional Networks with Adaptive Inference Graphs	ECCV	R	-
Slimmable neural networks		R	-
Multi-scale dense networks for resource efficient image classification	ICLR	E	-
Hydranets: Specialized dynamic architectures for efficient inference	CVPR	E	-

2012-2017

Title	Venue	Type	Code
Branchynet: Fast inference via early exiting from deep neural networks	ICPR	E	-
Adaptive computation time for recurrent neural networks		E	-
Spatially adaptive computation time for residual networks	CVPR	C	-
Recurrent segmentation for variable computational budgets	CVPR	E	-
Improved techniques for training adaptive deep networks	ICCV	E	-
Dynamic channel pruning: Feature boosting and suppression	ICLR	C	-

Surveys

Title	Venue	Year