Skip to content

Commit

Permalink
feat(docs): add some (not shown) content
Browse files Browse the repository at this point in the history
  • Loading branch information
ThePyProgrammer committed Aug 3, 2024
1 parent f0ae274 commit dee89fd
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 2 deletions.
15 changes: 14 additions & 1 deletion docs/components/dataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,10 @@ cse[0]

## Loading a List

Of course, this is not the only way to load data. If you have any other inputs, one of the easiest ways to load it into our system is to simply read it into a **list of dictionaries**. Here is an example:
Of course, this is not the only way to load data. If you have any other inputs, one of the easiest ways to load it into our system is to simply read it into a **list of dictionaries** and **list of strings**. Here is an example:

```python linenums="45"
# load list of dictionaries
raw_data = [
{"prompt": "what's up"},
{"prompt": "hey!"},
Expand All @@ -117,6 +118,18 @@ dummy_dataset = HuggingFaceDataset[Prompt].from_list("dummy", raw_data)
dummy_dataset.name
# "dummy"

dummy_dataset[0]
# Prompt(prompt="what's up")

# load list of strings
raw_data = [
"what's up",
"hey!",
"why is the world so curle"
]

dummy_dataset = HuggingFaceDataset[Prompt].from_list("dummy", raw_data)

dummy_dataset[0]
# Prompt(prompt="what's up")
```
Expand Down
1 change: 1 addition & 0 deletions docs/components/judges.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Judges
6 changes: 6 additions & 0 deletions docs/components/templates.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Prompt Templates

WalledEval strives to provide a method to build easy-to-use templates for researchers and testers alike to use for all kinds of different tasks. Whether it be a template to [automatically mutate prompts](auto-red-teaming.md) or to [prompt LLMs to perform like judges](quickstart/judges.md), prompt templates take up a major portion of


Beyond just loading data, our libray provides methods to load adversarial Prompt Templates like [DAN](https://github.com/verazuo/jailbreak_llms) and [DeepInception](https://github.com/tmlr-group/DeepInception). The team of WalledEval has compiled an extensive dataset of Prompt Templates from several papers, datasets and codebases, with more to come. We hope to use this to standardise a potential practice of keeping strings out of the codebase.
2 changes: 1 addition & 1 deletion docs/quickstart/templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Beyond general

With the advent of LLMs being used for mutation, inference and judges, prompt templates have become a recurring feature in various parts of the general evaluation framework. Sadly, since [`promptsource`](https://github.com/bigscience-workshop/promptsource), there hasn't been much effort made in compiling a large variety of prompt templates in one centralised system. This is where our **prompt database** comes in! WalledEval compiles prompts from many, many papers in prior literature to consolidate a huge database to choose from.

WalledEval strives to provide a method to build easy-to-use templates for researchers and testers alike to use for all kinds of different tasks. Whether it be a template to [automatically mutate prompts](auto-red-teaming.md) or to [prompt LLMs to perform like judges](judges.md), prompt templates take up a major portion of
WalledEval strives to provide a method to build easy-to-use templates for researchers and testers alike to use for all kinds of different tasks. Whether it be a template to [automatically mutate prompts](auto-red-teaming.md) or to [prompt LLMs to perform like judges](quickstart/judges.md), prompt templates take up a major portion of


Beyond just loading data, our libray provides methods to load adversarial Prompt Templates like [DAN](https://github.com/verazuo/jailbreak_llms) and [DeepInception](https://github.com/tmlr-group/DeepInception). The team of WalledEval has compiled an extensive dataset of Prompt Templates from several papers, datasets and codebases, with more to come. We hope to use this to standardise a potential practice of keeping strings out of the codebase.

0 comments on commit dee89fd

Please sign in to comment.