From b31038aae90c296f108dde777037d223a1e5bfff Mon Sep 17 00:00:00 2001 From: NanoCode012 Date: Wed, 3 Jan 2024 11:56:19 +0900 Subject: [PATCH] chore(readme): update instruction to set config to load from cache (#1030) --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 4dd80339a4..53dd46aacf 100644 --- a/README.md +++ b/README.md @@ -935,8 +935,9 @@ accelerate launch -m axolotl.cli.train your_config.yml You can optionally pre-tokenize dataset with the following before finetuning. This is recommended for large datasets. -- Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface. -- Use `--debug` to see preprocessed examples. +- Set `dataset_prepared_path:` to a local folder for saving and loading pre-tokenized dataset. +- (Optional): Set `push_dataset_to_hub: hf_user/repo` to push it to Huggingface. +- (Optional): Use `--debug` to see preprocessed examples. ```bash python -m axolotl.cli.preprocess your_config.yml