diff --git a/README.md b/README.md index 71e992e3..2b8eeaf5 100644 --- a/README.md +++ b/README.md @@ -35,148 +35,24 @@ $ export PORTKEY_API_KEY=PORTKEY_API_KEY #### Now, let's make a request with GPT-4 ```py -import portkey_ai -from portkey_ai import Config, LLMOptions +from portkey_ai import Portkey -portkey.config = Config( - mode="single", - llms=LLMOptions(provider="openai", api_key="OPENAI_API_KEY") +# Construct a client with a virtual key +portkey = Portkey( + api_key="PORTKEY_API_KEY", + virtual_key="VIRTUAL_KEY" ) -r = portkey.ChatCompletions.create( - model="gpt-4", - messages=[ - {"role": "user","content": "Hello World!"} - ] +completion = portkey.chat.completions.create( + messages = [{ "role": 'user', "content": 'Say this is a test' }], + model = 'gpt-3.5-turbo' ) +print(completion) ``` Portkey fully adheres to the OpenAI SDK signature. This means that you can instantly switch to Portkey and start using Portkey's advanced production features right out of the box. -## **🪜 Detailed Integration Guide** - -**4 Steps to Integrate the SDK** -1. Get your virtual key for AI providers. -2. Construct your LLM, add Portkey features, provider features, and prompt. -3. Construct the Portkey client and set your usage mode. -4. Now call Portkey regularly like you would call your OpenAI constructor. - -Let's dive in! If you are an advanced user and want to directly jump to various full-fledged examples, [click here](https://github.com/Portkey-AI/portkey-python-sdk/tree/main/examples). - ---- - -### **Step 1️⃣ : Get your Virtual Keys for AI providers** - -Navigate to the "Virtual Keys" page on [Portkey](https://app.portkey.ai/) and hit the "Add Key" button. Choose your AI provider and assign a unique name to your key. Your virtual key is ready! - -### **Step 2️⃣ : Construct your LLM, add Portkey features, provider features, and prompt** - -**Portkey Features**: -You can find a comprehensive [list of Portkey features here](#📔-list-of-portkey-features). This includes settings for caching, retries, metadata, and more. - -**Provider Features**: -Portkey is designed to be flexible. All the features you're familiar with from your LLM provider, like `top_p`, `top_k`, and `temperature`, can be used seamlessly. Check out the [complete list of provider features here](https://github.com/Portkey-AI/portkey-python-sdk/blob/af0814ebf4f1961b5dfed438918fe68b26ef5f1e/portkey/api_resources/utils.py#L137). - -**Setting the Prompt Input**: -This param lets you override any prompt that is passed during the completion call - set a model-specific prompt here to optimise the model performance. You can set the input in two ways. For models like Claude and GPT3, use `prompt` = `(str)`, and for models like GPT3.5 & GPT4, use `messages` = `[array]`. - - -Here's how you can combine everything: - -```python -from portkey_aiimport LLMOptions - -# Portkey Config -provider = "openai" -virtual_key = "key_a" -trace_id = "portkey_sdk_test" - -# Model Settings -model = "gpt-4" -temperature = 1 - -# User Prompt -messages = [{"role": "user", "content": "Who are you?"}] - -# Construct LLM -llm = LLMOptions(provider=provider, virtual_key=virtual_key, trace_id=trace_id, model=model, temperature=temperature) -``` - -### **Step 3️⃣ : Construct the Portkey Client** - -Portkey client's config takes 3 params: `api_key`, `mode`, `llms`. - -* `api_key`: You can set your Portkey API key here or with `os.ennviron` as done above. -* `mode`: There are **3** modes - Single, Fallback, Loadbalance. - * **Single** - This is the standard mode. Use it if you do not want Fallback OR Loadbalance features. - * **Fallback** - Set this mode if you want to enable the Fallback feature. - * **Loadbalance** - Set this mode if you want to enable the Loadbalance feature. -* `llms`: This is an array where we pass our LLMs constructed using the LLMOptions constructor. - -```py -import portkey_ai -from portkey_aiimport Config - -portkey.config = Config(mode="single",llms=[llm]) -``` - -### **Step 4️⃣ : Let's Call the Portkey Client!** - -The Portkey client can do `ChatCompletions` and `Completions`. - -Since our LLM is GPT4, we will use ChatCompletions: - -```py -response = portkey.ChatCompletions.create( - messages=[{ - "role": "user", - "content": "Who are you ?" - }] -) -print(response.choices[0].message) -``` - -You have integrated Portkey's Python SDK in just 4 steps! - ---- - -## **🔁 Demo: Implementing GPT4 to GPT3.5 Fallback Using the Portkey SDK** - -```py -import os -os.environ["PORTKEY_API_KEY"] = "PORTKEY_API_KEY" # Setting the Portkey API Key - -import portkey_ai -from portkey_aiimport Config, LLMOptions - -# Let's construct our LLMs. -llm1 = LLMOptions(provider="openai", model="gpt-4", virtual_key="key_a"), -llm2 = LLMOptions(provider="openai", model="gpt-3.5-turbo", virtual_key="key_a") - -# Now let's construct the Portkey client where we will set the fallback logic -portkey.config = Config(mode="fallback",llms=[llm1,llm2]) - -# And, that's it! -response = portkey.ChatCompletions.create() -print(response.choices[0].message) -``` - -## **📔 Full List of Portkey Config** - -| Feature | Config Key | Value(Type) | Required | -|---------------------|-------------------------|--------------------------------------------------|-------------| -| Provider Name | `provider` | `string` | ✅ Required | -| Model Name | `model` | `string` | ✅ Required | -| Virtual Key OR API Key | `virtual_key` or `api_key` | `string` | ✅ Required (can be set externally) | -| Cache Type | `cache_status` | `simple`, `semantic` | ❔ Optional | -| Force Cache Refresh | `cache_force_refresh` | `True`, `False` (Boolean) | ❔ Optional | -| Cache Age | `cache_age` | `integer` (in seconds) | ❔ Optional | -| Trace ID | `trace_id` | `string` | ❔ Optional | -| Retries | `retry` | `{dict}` with two required keys: `"attempts"` which expects integers in [0,5] and `"on_status_codes"` which expects array of status codes like [429,502]
`Example`: { "attempts": 5, "on_status_codes":[429,500] } | ❔ Optional | -| Metadata | `metadata` | `json object` [More info](https://docs.portkey.ai/key-features/custom-metadata) | ❔ Optional | - - ## **🤝 Supported Providers** || Provider | Support Status | Supported Endpoints |