Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: CUDA error: out of memory #44

Open
BigPig117 opened this issue Apr 7, 2022 · 0 comments
Open

RuntimeError: CUDA error: out of memory #44

BigPig117 opened this issue Apr 7, 2022 · 0 comments

Comments

@BigPig117
Copy link

hello, this is my error when i try to train the centerpoint.yaml

cfg.OPTIMIZATION = edict()
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.BATCH_SIZE_PER_GPU: 4
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.NUM_EPOCHS: 80
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.OPTIMIZER: adam_onecycle
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.LR: 0.003
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.WEIGHT_DECAY: 0.01
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.MOMENTUM: 0.9
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.MOMS: [0.95, 0.85]
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.PCT_START: 0.4
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.DIV_FACTOR: 10
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.DECAY_STEP_LIST: [35, 45]
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.LR_DECAY: 0.1
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.LR_CLIP: 1e-07
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.LR_WARMUP: False
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.WARMUP_EPOCH: 1
2022-04-07 11:02:52,498 INFO cfg.OPTIMIZATION.GRAD_NORM_CLIP: 10
2022-04-07 11:02:52,498 INFO cfg.TAG: centerpoint
2022-04-07 11:02:52,498 INFO cfg.EXP_GROUP_PATH: kitti_models
2022-04-07 11:02:52,570 INFO Database filter by min points Car: 14357 => 13532
2022-04-07 11:02:52,571 INFO Database filter by min points Pedestrian: 2207 => 2168
2022-04-07 11:02:52,571 INFO Database filter by min points Cyclist: 734 => 705
2022-04-07 11:02:52,582 INFO Database filter by difficulty Car: 13532 => 10759
2022-04-07 11:02:52,584 INFO Database filter by difficulty Pedestrian: 2168 => 2075
2022-04-07 11:02:52,585 INFO Database filter by difficulty Cyclist: 705 => 581
2022-04-07 11:02:52,588 INFO Loading KITTI dataset
2022-04-07 11:02:52,646 INFO Total samples for KITTI dataset: 3712
Traceback (most recent call last):
File "train.py", line 202, in
main()
File "train.py", line 116, in main
model = build_network(model_cfg=cfg.MODEL, num_class=len(cfg.CLASS_NAMES), dataset=train_set)
File "../pcdet/models/init.py", line 18, in build_network
model_cfg=model_cfg, num_class=num_class, dataset=dataset
File "../pcdet/models/detectors/init.py", line 30, in build_detector
model_cfg=model_cfg, num_class=num_class, dataset=dataset
File "../pcdet/models/detectors/centerpoint.py", line 7, in init
self.module_list = self.build_networks()
File "../pcdet/models/detectors/detector3d_template.py", line 47, in build_networks
model_info_dict=model_info_dict
File "../pcdet/models/detectors/detector3d_template.py", line 136, in build_dense_head
voxel_size=model_info_dict.get('voxel_size', False)
File "../pcdet/models/dense_heads/center_head.py", line 66, in init
[self.class_names.index(x) for x in cur_class_names if x in class_names]
RuntimeError: CUDA error: out of memory

base on GTX3070 cuda11.1 pytorch1.8.2 , can u give me some suggestions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant