Final singlefile checkpoint saves one folder up

[daviswer]

Addresses #126

Adjusts the save location of final single-file checkpoint to be one folder up (directly in the specified ckp directory, rather than under the 'checkpoints' subfolder). That way the dataloader checkpointer only has to deal with distributed checkpoint folders.

[lchu6]

@daviswer one convention I have been using is:

ckpt_root_folder/checkpoints/step_xxx: for dcp
ckpt_root_folder/pth/step_xxx: for pth
ckpt_root_folder/hf/step_xxx: for converted hf format

would using this convention fit the need here as well? if so, we can use this convention.

[daviswer]

oh yeah that's a good idea, I'll set that up

[lchu6]

@daviswer is the final pth file a single file or multiple?

[lchu6]

I think we should make it ckpt_root_folder/pth/step_xxx/consolidated.00.pth.

This will make it easier for future consumption/conversion, e.g. https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py#L231

to summarize, our folders should look like:
ckpt_root_folder/checkpoints/step_xxx/: many dcp files
ckpt_root_folder/pth/step_xxx/: single consolidated.00.pth file
ckpt_root_folder/hf/step_xxx/: many safetensor files

[daviswer]

Sure I can do that. It is always a single file.