Check failed: error == cudaSuccess (2 vs. 0) out of memory

[Hub-Tian]

Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
@ 0x7f6d6f7dce6d (unknown)
@ 0x7f6d6f7deced (unknown)
@ 0x7f6d6f7dca5c (unknown)
@ 0x7f6d6f7df63e (unknown)
@ 0x7f6d7699a2c1 caffe::SyncedMemory::to_gpu()
@ 0x7f6d76999629 caffe::SyncedMemory::mutable_gpu_data()
@ 0x7f6d7680e1a2 caffe::Blob<>::mutable_gpu_data()
@ 0x7f6d76842448 caffe::BaseConvolutionLayer<>::forward_gpu_gemm()
@ 0x7f6d769ce826 caffe::ConvolutionLayer<>::Forward_gpu()
@ 0x7f6d76963215 caffe::Net<>::ForwardFromTo()
@ 0x7f6d76963587 caffe::Net<>::Forward()
@ 0x7f6d76981b67 caffe::Solver<>::Step()
@ 0x7f6d76982429 caffe::Solver<>::Solve()
@ 0x408adb train()
@ 0x405f8c main
@ 0x7f6d6b59baf5 __libc_start_main
@ 0x4067fd (unknown)
I0221 13:19:16.887039 26191 caffe.cpp:217] Using GPUs 0
I0221 13:19:17.773277 26191 caffe.cpp:222] GPU 0: Tesla K80
I0221 13:19:18.569195 26191 solver.cpp:48] Initializing solver from parameters:

I am using the original code downloaded from this repository to train a car detector on KITTI. My GPU is Tesla K80. I ran train_mscnn.sh under /kitti_car/mscnn-7s-576-2x/ but the error message suggested that the memory is not enough. And it can not be solved by using more GPUs, for example 4. I wonder what I should change to handle this problem.

[zhaoweicai]

A single K80 is enough for all model training. Please see #7