Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix generate script after new snippets #1490

Merged
merged 9 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
311 changes: 256 additions & 55 deletions docs/api-inference/tasks/chat-completion.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,12 @@ This is a subtask of [`text-generation`](https://huggingface.co/docs/api-inferen
- [google/gemma-2-2b-it](https://huggingface.co/google/gemma-2-2b-it): A text-generation model trained to follow instructions.
- [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct): Very powerful text generation model trained to follow instructions.
- [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct): Small yet powerful text generation model.
- [HuggingFaceH4/starchat2-15b-v0.1](https://huggingface.co/HuggingFaceH4/starchat2-15b-v0.1): Strong coding assistant model.
- [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407): Very strong open-source large language model.
- [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct): Strong text generation model to follow instructions.

#### Conversational Vision-Language Models (VLMs)

- [meta-llama/Llama-3.2-11B-Vision-Instruct](https://huggingface.co/meta-llama/Llama-3.2-11B-Vision-Instruct): Powerful vision language model with great visual understanding and reasoning capabilities.
- [microsoft/Phi-3.5-vision-instruct](https://huggingface.co/microsoft/Phi-3.5-vision-instruct): Strong image-text-to-text model.
- [Qwen/Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct): Strong image-text-to-text model.

### API Playground

Expand Down Expand Up @@ -65,46 +64,133 @@ The API supports:
curl 'https://api-inference.huggingface.co/models/google/gemma-2-2b-it/v1/chat/completions' \
-H "Authorization: Bearer hf_***" \
-H 'Content-Type: application/json' \
-d '{
"model": "google/gemma-2-2b-it",
"messages": [{"role": "user", "content": "What is the capital of France?"}],
"max_tokens": 500,
"stream": false
--data '{
"model": "google/gemma-2-2b-it",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
],
"max_tokens": 500,
"stream": true
}'

```
</curl>

<python>
With huggingface_hub client:
```py
from huggingface_hub import InferenceClient

client = InferenceClient(api_key="hf_***")

for message in client.chat_completion(
model="google/gemma-2-2b-it",
messages=[{"role": "user", "content": "What is the capital of France?"}],
messages = [
{
"role": "user",
"content": "What is the capital of France?"
}
]

stream = client.chat.completions.create(
model="google/gemma-2-2b-it",
messages=messages,
max_tokens=500,
stream=True,
):
print(message.choices[0].delta.content, end="")
stream=True
)

for chunk in stream:
print(chunk.choices[0].delta.content, end="")
```

With openai client:
```py
from openai import OpenAI

client = OpenAI(
base_url="https://api-inference.huggingface.co/v1/",
api_key="hf_***"
)

messages = [
{
"role": "user",
"content": "What is the capital of France?"
}
]

stream = client.chat.completions.create(
model="google/gemma-2-2b-it",
messages=messages,
max_tokens=500,
stream=True
)

for chunk in stream:
print(chunk.choices[0].delta.content, end="")
```

To use the Python client, see `huggingface_hub`'s [package reference](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client#huggingface_hub.InferenceClient.chat_completion).
</python>

<js>
With huggingface_hub client:
```js
import { HfInference } from "@huggingface/inference";
import { HfInference } from "@huggingface/inference"

const inference = new HfInference("hf_***");
const client = new HfInference("hf_***")

for await (const chunk of inference.chatCompletionStream({
let out = "";

const stream = client.chatCompletionStream({
model: "google/gemma-2-2b-it",
messages: [{ role: "user", content: "What is the capital of France?" }],
messages: [
{
role: "user",
content: "What is the capital of France?"
}
],
max_tokens: 500
});

for await (const chunk of stream) {
if (chunk.choices && chunk.choices.length > 0) {
const newContent = chunk.choices[0].delta.content;
out += newContent;
console.log(newContent);
}
}
```

With openai client:
```js
import { OpenAI } from "openai"

const client = new OpenAI({
baseURL: "https://api-inference.huggingface.co/v1/",
apiKey: "hf_***"
})

let out = "";

const stream = await client.chat.completions.create({
model: "google/gemma-2-2b-it",
messages: [
{
role: "user",
content: "What is the capital of France?"
}
],
max_tokens: 500,
})) {
process.stdout.write(chunk.choices[0]?.delta?.content || "");
stream: true,
});

for await (const chunk of stream) {
if (chunk.choices && chunk.choices.length > 0) {
const newContent = chunk.choices[0].delta.content;
out += newContent;
console.log(newContent);
}
}
```

Expand All @@ -125,73 +211,188 @@ To use the JavaScript client, see `huggingface.js`'s [package reference](https:/
curl 'https://api-inference.huggingface.co/models/meta-llama/Llama-3.2-11B-Vision-Instruct/v1/chat/completions' \
-H "Authorization: Bearer hf_***" \
-H 'Content-Type: application/json' \
-d '{
"model": "meta-llama/Llama-3.2-11B-Vision-Instruct",
"messages": [
--data '{
"model": "meta-llama/Llama-3.2-11B-Vision-Instruct",
"messages": [
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}},
{"type": "text", "text": "Describe this image in one sentence."}
{
"type": "text",
"text": "Describe this image in one sentence."
},
{
"type": "image_url",
"image_url": {
"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
}
}
]
}
],
"max_tokens": 500,
"stream": false
"max_tokens": 500,
"stream": true
}'

```
</curl>

<python>
With huggingface_hub client:
```py
from huggingface_hub import InferenceClient

client = InferenceClient(api_key="hf_***")

image_url = "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
messages = [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe this image in one sentence."
},
{
"type": "image_url",
"image_url": {
"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
}
}
]
}
]

stream = client.chat.completions.create(
model="meta-llama/Llama-3.2-11B-Vision-Instruct",
messages=messages,
max_tokens=500,
stream=True
)

for message in client.chat_completion(
model="meta-llama/Llama-3.2-11B-Vision-Instruct",
messages=[
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": image_url}},
{"type": "text", "text": "Describe this image in one sentence."},
],
}
],
for chunk in stream:
print(chunk.choices[0].delta.content, end="")
```

With openai client:
```py
from openai import OpenAI

client = OpenAI(
base_url="https://api-inference.huggingface.co/v1/",
api_key="hf_***"
)

messages = [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe this image in one sentence."
},
{
"type": "image_url",
"image_url": {
"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
}
}
]
}
]

stream = client.chat.completions.create(
model="meta-llama/Llama-3.2-11B-Vision-Instruct",
messages=messages,
max_tokens=500,
stream=True,
):
print(message.choices[0].delta.content, end="")
stream=True
)

for chunk in stream:
print(chunk.choices[0].delta.content, end="")
```

To use the Python client, see `huggingface_hub`'s [package reference](https://huggingface.co/docs/huggingface_hub/package_reference/inference_client#huggingface_hub.InferenceClient.chat_completion).
</python>

<js>
With huggingface_hub client:
```js
import { HfInference } from "@huggingface/inference";
import { HfInference } from "@huggingface/inference"

const inference = new HfInference("hf_***");
const imageUrl = "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg";
const client = new HfInference("hf_***")

for await (const chunk of inference.chatCompletionStream({
let out = "";

const stream = client.chatCompletionStream({
model: "meta-llama/Llama-3.2-11B-Vision-Instruct",
messages: [
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": imageUrl}},
{"type": "text", "text": "Describe this image in one sentence."},
],
role: "user",
content: [
{
type: "text",
text: "Describe this image in one sentence."
},
{
type: "image_url",
image_url: {
url: "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
}
}
]
}
],
max_tokens: 500
});

for await (const chunk of stream) {
if (chunk.choices && chunk.choices.length > 0) {
const newContent = chunk.choices[0].delta.content;
out += newContent;
console.log(newContent);
}
}
```

With openai client:
```js
import { OpenAI } from "openai"

const client = new OpenAI({
baseURL: "https://api-inference.huggingface.co/v1/",
apiKey: "hf_***"
})

let out = "";

const stream = await client.chat.completions.create({
model: "meta-llama/Llama-3.2-11B-Vision-Instruct",
messages: [
{
role: "user",
content: [
{
type: "text",
text: "Describe this image in one sentence."
},
{
type: "image_url",
image_url: {
url: "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
}
}
]
}
],
max_tokens: 500,
})) {
process.stdout.write(chunk.choices[0]?.delta?.content || "");
stream: true,
});

for await (const chunk of stream) {
if (chunk.choices && chunk.choices.length > 0) {
const newContent = chunk.choices[0].delta.content;
out += newContent;
console.log(newContent);
}
}
```

Expand Down
1 change: 0 additions & 1 deletion docs/api-inference/tasks/image-classification.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ For more details about the `image-classification` task, check out its [dedicated
### Recommended models

- [google/vit-base-patch16-224](https://huggingface.co/google/vit-base-patch16-224): A strong image classification model.
- [facebook/deit-base-distilled-patch16-224](https://huggingface.co/facebook/deit-base-distilled-patch16-224): A robust image classification model.

Explore all available models and find the one that suits you best [here](https://huggingface.co/models?inference=warm&pipeline_tag=image-classification&sort=trending).

Expand Down
Loading
Loading