Skip to content

Latest commit

 

History

History
58 lines (50 loc) · 1.78 KB

benchmark.md

File metadata and controls

58 lines (50 loc) · 1.78 KB
Raw

Benchmarking

  1. Introduction
  2. Benchmark Support Matrix
  3. Get Started with Benchmark
  4. Examples

Introduction

The benchmarking feature of Neural Compressor is used to measure the model performance with the objective settings. Users can get the performance of the float32 model and the optimized low precision model in the same scenarios.

Benchmark Support Matrix

Environment Category
Operating System linux
windows
Architecture x86_64
aarch64
gpu

Get Started with Benchmark API

Benchmark provide capability to automatically run with multiple instance through cores_per_instance and num_of_instance config (CPU only). And please make sure cores_per_instance * num_of_instance must be less than CPU physical core numbers. benchmark.fit accept b_dataloader or b_func as input. b_func is customized benchmark function. If user passes the b_dataloader, then b_func is not required.

from neural_compressor.config import BenchmarkConfig
from neural_compressor.benchmark import fit

conf = BenchmarkConfig(warmup=10, iteration=100, cores_per_instance=4, num_of_instance=7)
fit(model="./int8.pb", conf=conf, b_dataloader=eval_dataloader)

Examples

Refer to the Benchmark example.