Skip to content

Commit

Permalink
Merge pull request #16 from fiddler-labs/release/v0.0.1
Browse files Browse the repository at this point in the history
Bumping up RC to release
  • Loading branch information
iterix authored May 25, 2023
2 parents 3a41549 + 045ee64 commit 4ccef5d
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 9 deletions.
34 changes: 27 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# <img src="https://github.com/fiddler-labs/fiddler-auditor/blob/main/docs/source/images/fiddler-auditor-logo.png" width="60%" alt="Fiddler Auditor">
# <img src="https://github.com/fiddler-labs/fiddler-auditor/blob/main/docs/source/images/fiddler-auditor-logo.png?raw=true" width="60%" alt="Fiddler Auditor">

Auditing Large Language Models made easy!

Expand All @@ -18,30 +18,50 @@ However, it’s been known that LLMs can hallucinate, generate adversarial respo
The Fiddler Auditor enables you to test LLMs and NLP models, identify weaknesses in the models, and mitigate potential adversarial outcomes before deploying them to production.

## Features and Capabilities

<p>
<div align="left">
<img src="https://github.com/fiddler-labs/fiddler-auditor/blob/main/examples/images/fiddler-auditor-flow.png?raw=true"
alt="Fiddler Auditor Flow"/>
</div>
</p>

Fiddler Auditor supports

- Red-teaming LLMs for your use-case with prompt perturbation
- Integration with LangChain
- Custom evaluation metrics
- Generative and Discriminative NLP models
- Comparison of LLMs (Upcoming)
- Comparison of LLMs

<p>
<div align="left">
<img src="https://github.com/fiddler-labs/fiddler-auditor/blob/main/docs/source/images/fiddler-auditor-prompt-evaluation.png?raw=true"
alt="Example Report"/>
<em> An example report generated by the Fiddler Auditor for text-davinci-003. </em>
</div>
</p>


## Installation
Auditor is available on pypi.

`pip install fiddler-auditor`
### Fom PyPI
Auditor is available on pypi and we test on Python 3.8 and above. Install using
the following command

```bash
pip install fiddler-auditor
```

or you can install from source after cloning the repo using the following command
### From source
You can install from source after cloning this repo using the following command

`pip install .`
```bash
pip install .
```

## Quick-start guides
- [Evaluate LLM Correctness and Robustness](https://github.com/fiddler-labs/fiddler-auditor/blob/main/examples/LLM_Evaluation.ipynb)
- [Evaluate LLM Correctness and Robustness](https://github.com/fiddler-labs/fiddler-auditor/blob/main/examples/LLM_Evaluation.ipynb)
- [Evaluate LLMs with custom metrics](https://github.com/fiddler-labs/fiddler-auditor/blob/main/examples/Custom_Evaluation.ipynb)


Expand Down
2 changes: 1 addition & 1 deletion auditor/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""module entrypoint"""
__version__ = '0.0.1.rc3'
__version__ = '0.0.1'
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "fiddler-auditor"
version = "0.0.1.rc3"
version = "0.0.1"
authors = [
{ name="Fiddler Labs", email="[email protected]" },
]
Expand Down

0 comments on commit 4ccef5d

Please sign in to comment.