Release Test Plan History

Goal as of 9/30/2021

Meet 9/30/2021 release plan: https://github.com/CentaurusInfra/arktos/wiki/Computing-930-2021-Tracks

Perf-tests

• scale-out: 25K nodes (1TP, 1RP); 50K nodes (2TP, 2RP)
• scale-up: 1K nodes (1 apiserver)

Perf/Scalability Test Plan：50K

Main SLIS/SLOs

Status	SLI	SLO
Official	PodStartupLatency	99th percentile per cluster <= 6s
Official	SaturationPodStartupLatency	99th percentile per cluster <= 10s
Official	SchedulingThroughput	99th percentile per cluste >= 100
WIP	Latency of mutating API calls for single objects	99th percentile per cluste <= 1s
WIP	Latency of non-streaming read-only API calls	99th percentile per cluste (a) <= 1s if scope=resource (b) <= 5s if scope=namespace (c) <= 30s if scope=cluster

Test Scenarios/Plan

Scale-out: TP + RP

Node Num	Test Setup	Test Scope	Test Setup	Cluster QPS/Latency pod QPS	ETA
500	Kubemark Hollow-node	Density&Load	1TP + 1RP	100/5	9/20/2021 - DONE
1K	Kubemark Hollow-node	Load Only	1TP + 1RP	100/5	9/20/2021 - DONE
25K	Kubemark Hollow-node	Density	1TP + 1RP	100/25	9/29/2021 - DONE
50K	Kubemark Hollow-node	Density	2TP + 2RP	100/25	9/30/2021 - DONE

Scale-up: 1apiserver

Node Num	Test Setup	Test Scope	ETA
1K	Kubemark Hollow-node	Load/Density	9/20/2021 - DONE

Test Results

https://github.com/CentaurusInfra/arktos/wiki/2021-930-Release-Tests-Results

Goal as of 2/5/2021

Meet 2/5/2021 release plan:

E2E tests

• Multi-tenancy - Qian Chen
• Autoscaling - Vinay
• VM - Vinay
• ...

Perf-tests

• scale-out: 10K nodes (1TP, 1RP); 20K nodes (2TP, 1RP)
• scale-up: 8K nodes (1 apiserver)

E2E Test Plan

Multi-tenancy

Test Setup

Test Request/Plan

Autoscaling

Test Setup

Test Request/PLan

VM

Test Setup

Test Request/PLan

...

Perf/Scalability Test Plan：20K

Main SLIS/SLOs

Status	SLI	SLO
Official	PodStartupLatency	99th percentile per cluster-day1 <= 5s
Official	SaturationPodStartupLatency	99th percentile per cluster-day1 <= 5s
Official	SchedulingThroughput	99th percentile per cluster-day1 >= 20
WIP	Latency of mutating API calls for single objects	99th percentile per cluster-day1 <= 1s
WIP	Latency of non-streaming read-only API calls	99th percentile per cluster-day1 (a) <= 1s if scope=resource (b) <= 5s if scope=namespace (c) <= 30s if scope=cluster

Test Scenarios/Plan

Scale-out: TP + RP

Node Num	Test Setup	Test Scope	Test Setup	ETA
500	Kubemark Hollow-node	Density&Load	1TP + 1RP	1/31/2021
10K	Kubemark Hollow-node	Density&Load	1TP + 1RP	2/5/2021
20K	Kubemark Hollow-node	Density	2TP + 1RP	2/5/2021

Scale-up: 1apiserver

Node Num	Test Setup	Test Scope	ETA
500	Kubemark Hollow-node	Density&Load	1/31/2021
8K	Kubemark Hollow-node	Density&Load	2/5/2021

Test Results

Scale-up

https://github.com/CentaurusInfra/arktos/wiki/020521-Release-Tests-Result---Scale-up

Scale-out