Feature Request: Prompt List Input as a YAML File

[SamuelDevdas]

## Feature Request: Prompt List Input as a YAML File

**Description:**
I would like to suggest a feature that allows users to input a list of prompts from a YAML file. This would streamline the process of adding multiple prompts for grid search or A/B testing, making the tool more efficient and user-friendly.

**Feature Details:**
- **Functionality:** Users can upload or link a YAML file containing a list of prompts.
- **File Format:** The YAML file should have a specific structure that the application can easily parse. For example:
  ```yaml
  prompts:
    - "Write a short sentence!"
    - "Describe a sunset."
    - "Explain the theory of relativity in simple terms."
  ```
- **Integration:** A new button or link on the interface to upload or specify the path to the YAML file.
- **Validation:** Basic validation to ensure the uploaded file is in the correct YAML format and contains valid prompts.
- **Error Handling:** Clear error messages if the YAML file is improperly formatted or if there are issues reading the prompts.

**Benefits:**
- **Efficiency:** Users can quickly add multiple prompts without manually entering them one by one.
- **Convenience:** Simplifies the process of setting up experiments, especially for users who frequently run tests with multiple prompts.
- **Scalability:** Facilitates batch processing and experimentation.

**Thank you for considering this feature request. I believe it will greatly enhance the usability of the Ollama Grid Search tool.**

[dezoito]

@SamuelDevdas ,

Thank you! I like the idea, and I think it overlaps a bit with a feature I've wanted to introduce (the possibility of re-running experiments).

Can we brainstorm this a little?

1- I was thinking that since we can export experiments as JSON files, we could (perhaps should), use the same format and file extension to import prompts. The import routine should try to import all prompts in the file, create the appropriate input elements. If it finds additional parameters (system_prompt, a list of values for temperature or repeat_penalty, etc), those should also be used. If the JSON contains just prompts, use the defaults.

Would that work for you?

2- Does the YML format provide any advantages other than readability, compared to JSON? I have the impression that it could be tricky to express multi-line/complex prompts, but please correct me on that if I am wrong.

3- A selling point for this feature (regardless of format), is that one could easily transfer experiments/prompts from on machine to another, by exporting/importing files, and even keep those in source control.

Ler me know what you think.

[SamuelDevdas]

https://www.promptfoo.dev/docs/configuration/parameters/

Thanks for your quick reply and inputs. For answers to some of your questions, here is some inspiration we could use.

I have been using back and forth ollama-grid-search and promptfoo, and hands down the prior is the winner for user friendliness, simplicity and quick model comparisons etc. but holds huge potential to be the standalone alternative to promptoo, hence the suggestion.

[dezoito]

As an update, I just pushed a new release that allows re-running experiments - basically, it resets the main form with the parameters used on a selected past experiment, and let's the user change params or re-run it straight away.

If that works well It's going to be a lot easier to import experiment files (so users can copy stuff from different installations), and I plan to make so that this should work if the file contains only prompt variables as well.