diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..1d1dd29 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,38 @@ +# Guardrails Validator Template + +## How to create a Guardrails Validator +- On the top right of the page, click "Use this template", select "create a new repository" and set a name for the package. See [Naming Conventions](#naming-conventions) below. +- Clone down the new repository. +- Modify the class in [validator/main.py](validator/main.py) with source code for the new validator + - Make sure that the class still inherits from `Validator` and has the `register_validator` annotation. + - Set the `name` in the `register_validator` to the name of the repo prefixed with your org as a namespace and set the appropriate data type. +- Change [validator/__init__.py](validator/__init__.py) to your new Validator classname instead of ValidatorTemplate +- Perform a self install with `make dev` or `pip install -e ".[dev]"` +- Locally test the validator with the [test instructions below](#testing-and-using-your-validator) +- Modify the README and follow the Validator Card format; you can find an example [here](https://github.com/guardrails-ai/lowercase/blob/main/README.md) + +* Note: This package uses a pyproject.toml file, on first run, run `make dev` to pull down and install all dependencies + +### Naming Conventions +1. Avoid using `is` and `bug` +2. Use snake_case: i.e. `_` to separate words. e.g. valid_address +3. For the description of the repo, write one sentence that says what the validator does; should be the same as the description in the pydoc string. +4. When annotating the class use the `{namespace}/{validator_name}` pattern: e.g. `@register_validator(name=“guardrails/valid_address”)` + +### Testing and using your validator +- Open [test/test-validator.py](test/test-validator.py) to test your new validator +- Import your new validator and modify `ValidatorTestObject` accordingly +- Modify the TEST_OUTPUT and TEST_FAIL_OUTPUT accordingly +- Run `python test/test-validator.py` via terminal, make sure the returned output reflects the input object +- Write advanced tests for failures, etc. + +## Upload your validator to the validator hub +- Update the [pyproject.toml](pyproject.toml) file and make necessary changes as follows: + - Update the `name` field to the name of your validator + - Update the `description` field to a short description of your validator + - Update the `authors` field to your name and email + - Add/update the `dependencies` field to include all dependencies your validator needs. +- If there are are any post-installation steps such as downloading tokenizers, logging into huggingface etc., update the [post-install.py](validator/post-install.py) file accordingly. +- You can add additional files to the [validator](validator) directory, but don't rename any existing files/directories. + - e.g. Add any environment variables (without the values, just the keys) to the [.env](.env) file. +- Ensure that there are no other dependencies or any additional steps required to run your validator. diff --git a/LICENSE.txt b/LICENSE.txt new file mode 100644 index 0000000..261eeb9 --- /dev/null +++ b/LICENSE.txt @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/Makefile.txt b/Makefile.txt new file mode 100644 index 0000000..414e178 --- /dev/null +++ b/Makefile.txt @@ -0,0 +1,16 @@ +dev: + pip install -e ".[dev]" + +lint: + ruff check . + +test: + pytest ./tests + +type: + pyright validator + +qa: + make lint + make type + make tests \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..5da02bf --- /dev/null +++ b/README.md @@ -0,0 +1,110 @@ +# Overview + +| Developed by | Guardrails AI | +| --- | --- | +| Date of development | Feb 15, 2024 | +| Validator type | Format | +| Blog | | +| License | Apache 2 | +| Input/Output | Output | + +## Description + +### Intended Use +This validator is a template for creating other validators, but for demonstrative purposes it ensures that a generated output is the literal `pass`. + +### Requirements + +* Dependencies: + - guardrails-ai>=0.4.0 + +* Foundation model access keys: + - OPENAI_API_KEY + +## Installation + +```bash +$ guardrails hub install hub://guardrails/validator_template +``` + +## Usage Examples + +### Validating string output via Python + +In this example, we apply the validator to a string output generated by an LLM. + +```python +# Import Guard and Validator +from guardrails.hub import ValidatorTemplate +from guardrails import Guard + +# Setup Guard +guard = Guard().use( + ValidatorTemplate +) + +guard.validate("pass") # Validator passes +guard.validate("fail") # Validator fails +``` + +### Validating JSON output via Python + +In this example, we apply the validator to a string field of a JSON output generated by an LLM. + +```python +# Import Guard and Validator +from pydantic import BaseModel, Field +from guardrails.hub import ValidatorTemplate +from guardrails import Guard + +# Initialize Validator +val = ValidatorTemplate() + +# Create Pydantic BaseModel +class Process(BaseModel): + process_name: str + status: str = Field(validators=[val]) + +# Create a Guard to check for valid Pydantic output +guard = Guard.from_pydantic(output_class=Process) + +# Run LLM output generating JSON through guard +guard.parse(""" +{ + "process_name": "templating", + "status": "pass" +} +""") +``` + +# API Reference + +**`__init__(self, on_fail="noop")`** + +
+ +**`validate(self, value, metadata) -> ValidationResult`** + diff --git a/env.txt b/env.txt new file mode 100644 index 0000000..e69de29 diff --git a/gitignore.txt b/gitignore.txt new file mode 100644 index 0000000..c8398b0 --- /dev/null +++ b/gitignore.txt @@ -0,0 +1,9 @@ +.python-version +__pycache__/ +build +*.egg-info +.venv +.pytest_cache +.ruff_cache +.vscode +.idea \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..d3574eb --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,30 @@ +[project] +name = "validator-template" +version = "0.0.0" +description = "Template repo for Guardrails Hub validators." +authors = [ + {name = "Guardrails AI", email = "contact@guardrailsai.com"} +] +license = {file = "LICENSE"} +readme = "README.md" +requires-python = ">= 3.8.1" +dependencies = [ + "guardrails-ai>=0.4.0" +] + +[project.optional-dependencies] +dev = [ + "pyright", + "pytest", + "ruff" +] + +[tool.pytest.ini_options] +minversion = "6.0" +addopts = "-rP" +testpaths = [ + "tests" +] + +[tool.pyright] +include = ["validator"] \ No newline at end of file diff --git a/tests/__init__.py b/tests/__init__.py new file mode 100644 index 0000000..72a2623 --- /dev/null +++ b/tests/__init__.py @@ -0,0 +1,3 @@ +from .main import ValidatorTemplate + +__all__ = ["ValidatorTemplate"] diff --git a/tests/test_validator.py b/tests/test_validator.py new file mode 100644 index 0000000..77afb8a --- /dev/null +++ b/tests/test_validator.py @@ -0,0 +1,27 @@ +# to run these, run +# make tests + +from guardrails import Guard +import pytest +from validator import ValidatorTemplate + +# We use 'exception' as the validator's fail action, +# so we expect failures to always raise an Exception +# Learn more about corrective actions here: +# https://www.guardrailsai.com/docs/concepts/output/#%EF%B8%8F-specifying-corrective-actions +guard = Guard.from_string(validators=[ValidatorTemplate(arg_1="arg_1", arg_2="arg_2", on_fail="exception")]) + +def test_pass(): + test_output = "pass" + result = guard.parse(test_output) + + assert result.validation_passed is True + assert result.validated_output == test_output + +def test_fail(): + with pytest.raises(Exception) as exc_info: + test_output = "fail" + guard.parse(test_output) + + # Assert the exception has your error_message + assert str(exc_info.value) == "Validation failed for field with errors: {A descriptive but concise error message about why validation failed}" diff --git a/validator/__init__.py b/validator/__init__.py new file mode 100644 index 0000000..72a2623 --- /dev/null +++ b/validator/__init__.py @@ -0,0 +1,3 @@ +from .main import ValidatorTemplate + +__all__ = ["ValidatorTemplate"] diff --git a/benchmark_context_relevancy_prompt.py b/validator/benchmark_context_relevancy_prompt.py similarity index 100% rename from benchmark_context_relevancy_prompt.py rename to validator/benchmark_context_relevancy_prompt.py diff --git a/benchmark_hallucination_prompt.py b/validator/benchmark_hallucination_prompt.py similarity index 100% rename from benchmark_hallucination_prompt.py rename to validator/benchmark_hallucination_prompt.py diff --git a/benchmark_qa_correctness_prompt.py b/validator/benchmark_qa_correctness_prompt.py similarity index 100% rename from benchmark_qa_correctness_prompt.py rename to validator/benchmark_qa_correctness_prompt.py diff --git a/validator/tutorial_arize_hallucination_evaluator_with_rag.ipynb b/validator/tutorial_arize_hallucination_evaluator_with_rag.ipynb new file mode 100644 index 0000000..fd108b5 --- /dev/null +++ b/validator/tutorial_arize_hallucination_evaluator_with_rag.ipynb @@ -0,0 +1,257 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "vAuY_sNgr4OQ" + }, + "source": [ + "# Install Dependencies\n", + "Various installations are required for OTL, LlamaIndex and Open AI." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "HY97cJFAk7Sc" + }, + "outputs": [], + "source": [ + "!pip install -qq 'openinference-instrumentation-llama-index>=0.1.6' 'openinference-instrumentation-llama-index>=0.1.6' llama-index-llms-openai opentelemetry-exporter-otlp llama-index>=0.10.3 \"llama-index-callbacks-arize-phoenix>=0.1.2\" arize-otel\n", + "\n", + "import os\n", + "from getpass import getpass\n", + "\n", + "openai_api_key = getpass(\"🔑 Enter your OpenAI API key: \")\n", + "os.environ[\"OPENAI_API_KEY\"] = openai_api_key" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OZvseSFHsD7M" + }, + "source": [ + "# Initialize Arize Phoenix\n", + "Set up OTL tracer for the `LlamaIndexInstrumentor`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "VCDUdapRrqpd" + }, + "outputs": [], + "source": [ + "from openinference.instrumentation.llama_index import LlamaIndexInstrumentor\n", + "from arize_otel import register_otel, Endpoints\n", + "\n", + "# Setup OTEL via our convenience function\n", + "register_otel(\n", + " endpoints = Endpoints.ARIZE,\n", + " space_key = getpass(\"🔑 Enter your Arize space key in the space settings page of the Arize UI: \"),\n", + " api_key = getpass(\"🔑 Enter your Arize API key in the space settings page of the Arize UI: \"),\n", + " model_id = \"test-guard-july10-6:07pm\", # name this to whatever you would like\n", + ")\n", + "LlamaIndexInstrumentor().instrument()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0z-3-Reas5TW" + }, + "source": [ + "# Instrument Guardrails AI\n", + "Install and instrument Guardrails AI. Import `ArizeDatasetEmbeddings` Guard." + ] + }, + { + "cell_type": "code", + "source": [ + "!pip install -qq guardrails-ai litellm" + ], + "metadata": { + "collapsed": true, + "id": "Q8nwDMRXfmjy" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "!pip install --no-cache-dir -qq git+https://github.com/Arize-ai/rag-llm-prompt-evaluator-guard\n", + "!guardrails hub install hub://arize-ai/llm_rag_evaluator\n", + "\n", + "from guardrails.hub import LlmRagEvaluator" + ], + "metadata": { + "id": "TK22jVYzBTJT" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "E5GV4hwGUxxB" + }, + "outputs": [], + "source": [ + "from guardrails import Guard\n", + "\n", + "guard = Guard.from_string(\n", + " validators=[\n", + " LlmRagEvaluator(\n", + " eval_llm_prompt_generator=HallucinationPrompt(prompt_name=\"hallucination_judge_llm\"),\n", + " llm_evaluator_fail_response=\"hallucinated\",\n", + " llm_evaluator_pass_response=\"factual\",\n", + " llm_callable=\"gpt-4o-mini\",\n", + " on_fail=\"exception\",\n", + " on=\"prompt\")\n", + " ],\n", + " )\n", + "guard._disable_tracer = True" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "MpwPFAtDsrcf" + }, + "outputs": [], + "source": [ + "import openai\n", + "from typing import Optional, List, Mapping, Any\n", + "\n", + "from llama_index.core import SimpleDirectoryReader, SummaryIndex\n", + "from llama_index.core.callbacks import CallbackManager\n", + "from llama_index.core.llms import (\n", + " CustomLLM,\n", + " CompletionResponse,\n", + " CompletionResponseGen,\n", + " LLMMetadata,\n", + ")\n", + "from llama_index.core.llms.callbacks import llm_completion_callback\n", + "from llama_index.core import Settings\n", + "\n", + "from llama_index.llms.openai import OpenAI\n", + "\n", + "def monkey_completion(prompt, **kwargs):\n", + " _, _, context_component_of_prompt = prompt.partition(\"Context information is below.\")\n", + " _, _, query_component_of_prompt = prompt.partition(\"Query: \")\n", + " return guard(\n", + " llm_api=openai.chat.completions.create,\n", + " prompt=prompt,\n", + " model=\"gpt-3.5-turbo\",\n", + " max_tokens=1024,\n", + " temperature=0.5,\n", + " metadata={\n", + " \"user_message\": query_component_of_prompt,\n", + " \"context\": context_component_of_prompt,\n", + " }\n", + " )\n", + "\n", + "outerOpenAI = OpenAI()\n", + "\n", + "class GuardedLLM(CustomLLM):\n", + " context_window: int = 3900\n", + " num_output: int = 256\n", + " model_name: str = \"custom\"\n", + " dummy_response: str = \"My response\"\n", + " openai_llm: Any = None\n", + "\n", + " @property\n", + " def metadata(self) -> LLMMetadata:\n", + " \"\"\"Get LLM metadata.\"\"\"\n", + " return outerOpenAI.metadata\n", + "\n", + " @llm_completion_callback()\n", + " def complete(self, prompt: str, **kwargs: Any) -> CompletionResponse:\n", + " validated_response = monkey_completion(prompt, **kwargs)\n", + " return CompletionResponse(text=validated_response.raw_llm_output)\n", + "\n", + " @llm_completion_callback()\n", + " def stream_complete(\n", + " self, prompt: str, **kwargs: Any\n", + " ) -> CompletionResponseGen:\n", + " response = \"\"\n", + " for token in self.dummy_response:\n", + " response += token\n", + " yield CompletionResponse(text=response, delta=token)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "u2nKIt4duKAU" + }, + "source": [ + "# Set Up RAG Application\n", + "Create a LlamaIndex VectorStore to create a classic RAG application over Paul Graham essays." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "p-g6O8tvsxKd" + }, + "outputs": [], + "source": [ + "!mkdir -p 'data/paul_graham/'\n", + "!curl 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' > 'data/paul_graham/paul_graham_essay.txt'\n", + "import logging\n", + "import sys\n", + "\n", + "logging.basicConfig(stream=sys.stdout, level=logging.INFO)\n", + "logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))\n", + "\n", + "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n", + "# load documents\n", + "documents = SimpleDirectoryReader(\"./data/paul_graham/\").load_data()\n", + "index = VectorStoreIndex.from_documents(documents, chunk_size=512)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "d1Rzk87ZwDo8" + }, + "outputs": [], + "source": [ + "from guardrails.errors import ValidationError\n", + "\n", + "query_engine = index.as_query_engine(\n", + " llm=GuardedLLM(),\n", + ")\n", + "\n", + "# Take jailbreak prompt from public dataset https://github.com/verazuo/jailbreak_llms/blob/main/data/prompts/jailbreak_prompts_2023_05_07.csv\n", + "sample_prompt = \"\"\"What are the 3 ingredients to great work?\"\"\"\n", + "\n", + "response = query_engine.query(sample_prompt)\n", + "print(response)" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file