Skip to content

Latest commit

 

History

History
233 lines (184 loc) · 20.4 KB

gpu_memory_appendix.md

File metadata and controls

233 lines (184 loc) · 20.4 KB

GPU Memory Appendix

On this page, we will display the required memory and execution time under different combinations of memory optimization parameters. We would also like to thank the community for their support in helping us enrich the table data.

Please note that due to variations in the running environment, the content in the table may differ from actual conditions and is for reference only.

RTX 3090 Results

Results from bmgjet

  • Environments
CPU Ram Disk GPU
7950X 64GB 6200mhz, 30,32,32 NVME 990 Pro 3090 (Water Cooled 500W PL, +1000mhz Vram) Sits about 2.1ghz on core

Memory usage reported from GPUz logs.

Torch was not compiled with flash attention.

Windows 11 Using Python Environment from Forge (webui_forge_cu124_torch24.7z) Feb 5 2024.

  • Resolution of 512
Num frames Parameters GPU memory Time
24 frames normal 0 18117mb 1:32s - 3.80s/it
48 frames normal 0 19995mb 4:06s - 9.98s/it
72 frames normal 0 22133mb 9:57s - 23.88s/it
96 frames normal 10 22200mb 12:51s - 30.84s/it
120 frames normal 7 23195mb 15:57s - 38.26s/it
  • Resolution of 768
Num frames Parameters GPU memory Time
24 frames normal 0 22105mb 4:07s - 9.94s/it
48 frames normal 7 22195mb 14:34s - 34.98s/it
72 frames lowmem 0 17998mb 38:29s - 91.99s/it
96 frames lowmem 0 23000mb 1:04:16s - 153.74s/it
120 frames lowmem 0 28195mb 5:29:54s - 797.28s/it
(froze system during vram overflows for a few sec each pass)

RTX 4090

Results from CreateAI

The GPU memory reported below is the max_memory_allocated() value. The values read from nvidia-smi may be higher than the reported values because CUDA occupies some GPU memory (usually between 500 - 800 MiB), and PyTorch's caching mechanism also requests additional GPU memory.

  • Resolution of 384
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
15387MiB
00:42s
15052MiB
00:50s
14941MiB
00:50s
14867MiB
00:57s
14718MiB
00:58s
48
frames
16686MiB
01:15s
16018MiB
01:33s
15795MiB
01:35s
15647MiB
01:38s
15350MiB
01:40s
72
frames
17996MiB
01:51s
16993MiB
02:12s
16659MiB
02:21s
16436MiB
02:22s
15990MiB
02:31s
96
frames
19297MiB
02:31s
17959MiB
02:58s
17514MiB
03:09s
17216MiB
03:13s
16621MiB
03:27s
120
frames
20599MiB
03:17s
18928MiB
03:50s
18371MiB
04:09s
17999MiB
04:14s
17256MiB
04:29s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
4096MiB
02:51s
4096MiB
03:25s
4096MiB
03:35s
4096MiB
02:45s
4095MiB
03:13s
48
frames
4148MiB
03:37s
4148MiB
04:01s
4148MiB
04:19s
4148MiB
03:43s
4147MiB
04:52s
72
frames
4200MiB
05:01s
4200MiB
05:40s
4200MiB
07:51s
4200MiB
05:01s
4200MiB
06:18s
96
frames
5248MiB
05:59s
4249MiB
06:49s
4249MiB
08:43s
4249MiB
06:03s
4249MiB
07:09s
120
frames
6549MiB
07:24s
4876MiB
08:12s
4321MiB
10:45s
4299MiB
08:09s
4299MiB
08:32s
  • Resolution of 512
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
16366MiB
01:18s
15805MiB
01:26s
15607MiB
01:37s
15475MiB
01:36s
15211MiB
01:39s
48
frames
18720MiB
02:21s
17532MiB
02:49s
17136MiB
02:55s
16872MiB
02:58s
16344MiB
03:01s
72
frames
21036MiB
03:41s
19254MiB
04:25s
18660MiB
04:34s
18264MiB
04:36s
17472MiB
04:51s
96
frames
-----MiB
--:--s
20972MiB
06:18s
20180MiB
06:24s
19652MiB
06:36s
18596MiB
06:56s
120
frames
-----MiB
--:--s
-----MiB
--:--s
21704MiB
08:50s
21044MiB
08:53s
19724MiB
09:08s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
4212MiB
04:31s
4212MiB
04:44s
4211MiB
05:12s
4211MiB
04:51s
4212MiB
04:14s
48
frames
4666MiB
05:17s
4401MiB
06:11s
4400MiB
06:33s
4399MiB
08:29s
4401MiB
06:18s
72
frames
6981MiB
07:27s
5199MiB
08:00s
4605MiB
08:57s
4598MiB
11:02s
4598MiB
08:46s
96
frames
9298MiB
09:33s
6922MiB
10:13s
6130MiB
11:31s
5602MiB
13:46s
4794MiB
11:29s
120
frames
11613MiB
11:33s
8643MiB
13:11s
7653MiB
14:18s
6993MiB
14:23s
5673MiB
14:16s
  • Resolution of 640
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
17671MiB
01:47s
16739MiB
02:09s
16429MiB
02:17s
16234MiB
02:23s
15850MiB
02:28s
48
frames
21324MiB
03:49s
19468MiB
04:23s
18849MiB
04:55s
18437MiB
04:59s
17612MiB
05:14s
72
frames
-----MiB
--:--s
-----MiB
--:--s
21230MiB
07:57s
20611MiB
08:13s
19373MiB
08:25s
96
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
21134MiB
11:57s
120
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
4301MiB
03:49s
4301MiB
04:31s
4301MiB
04:55s
4301MiB
06:33s
4301MiB
04:52s
48
frames
7271MiB
07:05s
5416MiB
08:04s
4798MiB
08:07s
4589MiB
09:42s
4589MiB
09:18s
72
frames
10889MiB
10:38s
8106MiB
11:54s
7179MiB
11:58s
6560MiB
12:52s
5322MiB
13:13s
96
frames
14509MiB
14:13s
10795MiB
15:50s
9557MiB
16:10s
8732MiB
17:08s
7082MiB
17:39s
120
frames
18128MiB
18:26s
13487MiB
19:57s
11942MiB
21:12s
10910MiB
21:55s
8847MiB
26:20s
  • Resolution of 768
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
19223MiB
02:38s
17900MiB
03:06s
17448MiB
03:18s
17153MiB
03:23s
16624MiB
03:34s
48
frames
-----MiB
--:--s
-----MiB
--:--s
20946MiB
07:28s
20352MiB
07:35s
19164MiB
08:04s
72
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
96
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
120
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
5251MiB
05:19s
4398MiB
05:55s
4398MiB
06:52s
4398MiB
07:24s
4399MiB
07:35s
48
frames
10457MiB
09:38s
7786MiB
11:09s
6896MiB
12:43s
6304MiB
12:52s
5114MiB
13:20s
72
frames
15671MiB
15:14s
11661MiB
17:50s
10325MiB
18:22s
9433MiB
19:32s
7652MiB
19:19s
96
frames
-----MiB
--:--s
15534MiB
24:30s
13752MiB
24:28s
12564MiB
25:54s
10188MiB
29:19s
120
frames
-----MiB
--:--s
19406MiB
33:04s
17179MiB
33:14s
15694MiB
35:01s
12724MiB
39:11s
  • Resolution of 896
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
21085MiB
03:46s
19265MiB
04:25s
18659MiB
04:39s
18248MiB
04:43s
17543MiB
04:59s
48
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
20994MiB
11:34s
72
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
96
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
120
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
7128MiB
07:26s
5309MiB
07:45s
4703MiB
07:50s
4517MiB
08:29s
4517MiB
09:31s
48
frames
14220MiB
14:05s
10582MiB
15:01s
9370MiB
15:43s
8562MiB
16:34s
6946MiB
17:36s
72
frames
-----MiB
--:--s
15857MiB
24:53s
14038MiB
25:06s
12826MiB
26:49s
10401MiB
27:30s
96
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
17087MiB
38:52s
13853MiB
45:19s
120
frames
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
-----MiB
--:--s
17310MiB
55:50s

A100

Results from CreateAI

The GPU memory reported below is the max_memory_allocated() value. The values read from nvidia-smi may be higher than the reported values because CUDA occupies some GPU memory (usually between 500 - 800 MiB), and PyTorch's caching mechanism also requests additional GPU memory.

  • Resolution of 384
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
15387MiB
00:36s
15052MiB
00:43s
14941MiB
00:50s
14867MiB
00:50s
14717MiB
00:56s
48
frames
16686MiB
00:57s
16018MiB
01:10s
15795MiB
01:28s
15647MiB
01:16s
15350MiB
01:21s
72
frames
17996MiB
01:29s
16993MiB
01:43s
16659MiB
01:53s
16436MiB
01:53s
15990MiB
01:57s
96
frames
19297MiB
02:01s
17959MiB
02:21s
17514MiB
02:29s
17216MiB
02:33s
16621MiB
02:54s
120
frames
20599MiB
02:33s
18928MiB
03:04s
18371MiB
03:12s
17999MiB
03:16s
17256MiB
03:25s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
4096MiB
03:56s
4096MiB
04:24s
4096MiB
04:16s
4096MiB
04:14s
4095MiB
04:41s
48
frames
4148MiB
04:45s
4148MiB
05:27s
4148MiB
05:29s
4148MiB
05:32s
4147MiB
06:06s
72
frames
4200MiB
06:10s
4200MiB
07:10s
4200MiB
07:13s
4200MiB
07:19s
4200MiB
07:24s
96
frames
5248MiB
07:53s
4249MiB
08:16s
4249MiB
08:30s
4249MiB
08:28s
4249MiB
09:05s
120
frames
6547MiB
08:43s
4876MiB
09:54s
4321MiB
10:16s
4299MiB
10:20s
4299MiB
11:26s
  • Resolution of 512
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
16399MiB
00:59s
15805MiB
01:07s
15607MiB
01:13s
15475MiB
01:18s
15211MiB
01:16s
48
frames
18720MiB
01:47s
17532MiB
02:10s
17136MiB
02:25s
16872MiB
02:22s
16344MiB
02:24s
72
frames
21036MiB
02:55s
19254MiB
03:27s
18660MiB
03:36s
18264MiB
03:41s
17472MiB
03:51s
96
frames
23348MiB
04:08s
20972MiB
04:46s
20180MiB
05:05s
19652MiB
05:12s
18596MiB
05:38s
120
frames
25664MiB
05:42s
22694MiB
06:28s
21704MiB
06:47s
21044MiB
06:50s
19724MiB
07:06s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
4212MiB
05:03s
4213MiB
05:31s
4212MiB
05:23s
4212MiB
05:50s
4212MiB
05:51s
48
frames
4666MiB
06:47s
4401MiB
07:13s
4401MiB
07:34s
4400MiB
08:21s
4400MiB
08:19s
72
frames
6981MiB
08:41s
5199MiB
09:44s
4605MiB
10:10s
4598MiB
11:13s
4598MiB
12:00s
96
frames
9298MiB
10:59s
6922MiB
12:10s
6130MiB
12:16s
5602MiB
13:27s
4794MiB
15:17s
120
frames
11613MiB
13:27s
8643MiB
14:55s
7653MiB
15:22s
6993MiB
16:04s
5673MiB
18:04s
  • Resolution of 640
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
17707MiB
01:26s
16778MiB
01:45s
16470MiB
01:48s
16263MiB
01:51s
15850MiB
01:53s
48
frames
21324MiB
03:09s
19468MiB
03:36s
18849MiB
03:53s
18437MiB
03:53s
17612MiB
04:03s
72
frames
24943MiB
05:06s
22158MiB
05:54s
21230MiB
06:12s
20612MiB
06:18s
19374MiB
06:35s
96
frames
28562MiB
07:35s
24848MiB
08:36s
23611MiB
09:04s
22785MiB
09:07s
21134MiB
09:35s
120
frames
32182MiB
10:32s
27541MiB
11:54s
25994MiB
12:20s
24962MiB
12:37s
22899MiB
13:05s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
4301MiB
05:40s
4301MiB
06:07s
4301MiB
06:27s
4301MiB
06:33s
4301MiB
06:57s
48
frames
7271MiB
09:18s
5416MiB
09:51s
4798MiB
10:26s
4589MiB
10:36s
4589MiB
11:52s
72
frames
10889MiB
13:03s
8106MiB
14:07s
7179MiB
14:35s
6561MiB
17:30s
5322MiB
16:33s
96
frames
14509MiB
16:57s
10795MiB
18:12s
9557MiB
18:29s
8732MiB
18:55s
7082MiB
21:10s
120
frames
18128MiB
22:05s
13487MiB
22:49s
11942MiB
23:47s
10910MiB
23:50s
8847MiB
32:46s
  • Resolution of 768
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
19300MiB
02:09s
17962MiB
02:32s
17517MiB
02:37s
17219MiB
02:39s
16624MiB
02:46s
48
frames
24510MiB
04:46s
21837MiB
05:32s
20946MiB
05:51s
20352MiB
05:59s
19164MiB
06:10s
72
frames
29725MiB
08:23s
25715MiB
09:35s
24378MiB
09:56s
23486MiB
10:05s
21703MiB
10:40s
96
frames
34925MiB
12:52s
29579MiB
14:33s
27797MiB
15:01s
26609MiB
15:13s
24233MiB
15:53s
120
frames
40134MiB
18:29s
33451MiB
20:39s
31223MiB
21:31s
29737MiB
21:42s
26766MiB
22:51s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
5251MiB
08:04s
4398MiB
08:15s
4398MiB
08:21s
4398MiB
08:48s
4399MiB
09:14s
48
frames
10457MiB
12:42s
7786MiB
13:54s
6896MiB
14:17s
6304MiB
14:37s
5114MiB
16:21s
72
frames
15671MiB
18:18s
11661MiB
19:44s
10325MiB
20:53s
9435MiB
20:47s
7652MiB
23:42s
96
frames
20880MiB
24:55s
15534MiB
27:07s
13752MiB
28:13s
12564MiB
28:44s
10188MiB
30:30s
120
frames
26092MiB
31:21s
19406MiB
34:09s
17179MiB
35:13s
15694MiB
35:15s
12724MiB
32:55s
  • Resolution of 896
Num
frames
normal_mode + 0 steps normal_mode + 10 steps normal_mode + 7 steps normal_mode + 5 steps normal_mode + 1 steps
24
frames
21181MiB
02:58s
19362MiB
03:32s
18755MiB
03:48s
18351MiB
03:47s
17543MiB
03:54s
48
frames
28271MiB
07:18s
24633MiB
08:24s
23420MiB
08:49s
22611MiB
08:53s
20994MiB
09:10s
72
frames
35360MiB
13:22s
29902MiB
14:57s
28083MiB
15:33s
26870MiB
15:46s
24445MiB
16:13s
96
frames
42447MiB
21:08s
35169MiB
23:48s
32744MiB
24:41s
31126MiB
25:08s
27891MiB
26:33s
120
frames
49538MiB
30:55s
40442MiB
34:03s
37410MiB
35:11s
35388MiB
35:31s
31345MiB
36:40s
Num
frames
low_gpu_mode + 0 steps low_gpu_mode + 10 steps low_gpu_mode + 7 steps low_gpu_mode + 5 steps low_gpu_mode + 1 steps
24
frames
7128MiB
08:52s
5309MiB
09:30s
4702MiB
10:37s
4517MiB
10:55s
4517MiB
12:09s
48
frames
14220MiB
15:56s
10582MiB
17:04s
9370MiB
18:49s
8562MiB
19:08s
6946MiB
21:50s
72
frames
21315MiB
24:58s
15857MiB
26:54s
14038MiB
29:10s
12826MiB
29:42s
10401MiB
30:58s
96
frames
28407MiB
31:16s
21130MiB
34:22s
18704MiB
36:45s
17087MiB
36:34s
13853MiB
43:48s
120
frames
35504MiB
45:00s
26405MiB
48:05s
23373MiB
49:27s
21352MiB
50:34s
17310MiB
59:38s