Could you provide tokenized continue-pretraining dataset for reproduction?

[gywlssww]

Could you provide tokenized continue-pretraining dataset for reproduction like pruning dataset?
Is tokenizer.model you provided exactly the same tokenizer as Llama-2?

[xiamengzhou]

Yes, we use the same tokenizer as llama-2. We'd love to share the data, but due to the shear amount of it, I am not sure what is the best way to serve it. Let me know if you have any idea!

[gywlssww]

Does the size of the dataset exceed the limit of Google Drive, One Drive or dropbox,,?

[vmasrani]

+1! Would be very helpful to have the finetuning/continue-pretraining dataset as well to be able to reproduce paper results.