We execute
python data_processing
to extract the required parts of the jsonl file into a new csv file.
- The run_optuna function is run in the main.py file and is used to find the right combination of hyperparameters.
- Record the final combination of hyperparameters output in the function for the next step of training.
Run the main_run function in the main.py file. Train the model on the training set and also generate a file of the prediction results for the test set.
Do not run both the run_optuna and main_run functions in the main.py file.