Skip to content

Latest commit

 

History

History
111 lines (81 loc) · 7.87 KB

practical_structural_pruning.md

File metadata and controls

111 lines (81 loc) · 7.87 KB
Raw

Practical Structural Pruning

Welcome to the resource list for structural pruning, which is continually being updated. If you have any useful resources, papers, or repositories related to structural pruning and would like to share them with other users, please feel free to open a new pull request or issue.


0. Projects/Papers that use Torch-Pruning

Title URL
LLaMA-Pruning: Structural Pruning for LLaMA project
Performance-aware Approximation of Global Channel Pruning for Multitask CNNs project, TPAMI'23
LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning project, NeurIPS'22
mobile-yolov5-pruning-distillation project
Pruning-for-YOLOX project
YOLOX-pruning project
Vision Model Compression (ResNet-18, VGG-16) project
LAASP project
Magnitude and Similarity based Variable Rate Filter Pruning for Efficient Convolution Neural Networks project
Resnet50-pruning projects
yolov5-pruning project
Model-Pruning (MobileNetv1/v2/v3, VGG16, and EfficientNet-b0) project
Melanoma detection CNN using PyTorch project
KamalEngine (Model Reuse) project
Modified-Yolov4Tiny-RaspberryPi project
model_pruning (ResNet18+CIFAR-10) project

1. Benchmark & Survey & Awesome List

Title Authors Venue Code
Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning Huan Wang Preprint pytorch
Structured Pruning for Deep Convolutional Neural Networks Yang He Preprint Awesome List

2. General

Title Authors Venue Code
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks Zhonghui You NeurIPS'19 pytorch
Neural Pruning via Growing Regularization Huan Wang ICLR'21 pytorch
Group Fisher Pruning for Practical Network Compression Liyang Liu ICML'21 pytorch
Only Train Once: A One-Shot Neural Network Training And Pruning Framework Tianyi Chen NeurIPS'21 pytorch
OTOV2: AUTOMATIC, GENERIC, USER-FRIENDLY Tianyi Chen ICLR'23 pytorch
DepGraph: Towards Any Structural Pruning Gongfan Fang CVPR'23 pytorch

3. YOLO

Type Title Authors Venue Code
YOLOv5 Performance-aware Approximation of Global Channel Pruning for Multitask CNNs Hancheng Ye TPAMI'23 pytorch
YOLOv7 Torch-Pruning: benchmarks/prunability/yolov7 - - pytorch

4. GANs

Title Authors Venue Code
GAN Compression: Efficient Architectures for Interactive Conditional GANs Muyang Li CVPR'20 pytorch

5. Vision Transformers

Title Authors Venue Code
Vision Transformer Pruning Mingjian Zhu Preprint N/A
SAViT: Structure-Aware Vision Transformer Pruning via Collaborative Optimization Zheng Chuanyang NeurIPS'22 N/A
Width & Depth Pruning for Vision Transformers Fang Yu AAAI'22 N/A
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen NeurIPS'21 pytorch
CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Prediction Zhuoran Song Preprint pytorch
DepGraph: Towards Any Structural Pruning Gongfan Fang CVPR'23 pytorch

6. NLP

Title Authors Venue Code
Structured Pruning Learns Compact and Accurate Models Mengzhou Xia ACL'22 pytorch

7. GNNs

Title Authors Venue Code
DepGraph: Towards Any Structural Pruning Gongfan Fang CVPR'23 pytorch

8. Misc

Title Source
Issue - Does the tool support to prune the yolov5/v7/v8? Torch-Pruning
Issue - How can we transfer schedular and optimizer from original model to pruned model Torch-Pruning
Discussion - How to get the index of the pruned channel(s) when using High-level Pruners? Torch-Pruning
Issue - How to get all groups of a network Torch-Pruning
Issue - 论文中图3(b)的剪枝方案是怎么推出的 (in Chinese) Torch-Pruning
Model Compression - the Pruning Approaches Lecture Notes (Princeton)
CVPR 2023 | DepGraph：任意架构的结构化剪枝，CNN、Transformer、GNN等都适用！(In Chinese) CVer
[CVPR 2023 DepGraph 通用结构化剪枝 (In Chinese)](https://zhuanlan.zhihu.com/p/619146631?)