Skip to content

Commit

Permalink
Merge branch 'main' into feature/api-playground-demo
Browse files Browse the repository at this point in the history
  • Loading branch information
Wauplin authored Nov 2, 2023
2 parents c55e7a7 + d58bfa1 commit 8d99fb3
Show file tree
Hide file tree
Showing 11 changed files with 42 additions and 36 deletions.
18 changes: 6 additions & 12 deletions docs/hub/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ Try it out now on our [Playground](https://huggingface.co/spaces/enzostvs/hub-ap
<img class="w-full object-contain" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/hub-api-playground.png"/>
</div>

## Search API
## Repo listing API

The following endpoints help get information about models, datasets, Spaces, and metrics stored on the Hub.

### GET /api/models

Get information from all models in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the following pages. You can specify additional parameters to have more specific results.
Get information from all models in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the next pages. You can specify additional parameters to have more specific results.
- `search`: Filter based on substrings for repos and their usernames, such as `resnet` or `microsoft`
- `author`: Filter models by an author or organization, such as `huggingface` or `microsoft`
- `filter`: Filter based on tags, such as `text-classification` or `spacy`.
Expand Down Expand Up @@ -60,7 +60,7 @@ This is equivalent to `huggingface_hub.get_model_tags()`.

### GET /api/datasets

Get information from all datasets in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the following pages. You can specify additional parameters to have more specific results.
Get information from all datasets in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the next pages. You can specify additional parameters to have more specific results.
- `search`: Filter based on substrings for repos and their usernames, such as `pets` or `microsoft`
- `author`: Filter datasets by an author or organization, such as `huggingface` or `microsoft`
- `filter`: Filter based on tags, such as `task_categories:text-classification` or `languages:en`.
Expand Down Expand Up @@ -116,15 +116,14 @@ This is equivalent to `huggingface_hub.get_dataset_tags()`.

### GET /api/spaces

Get information from all Spaces in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the following pages. You can specify additional parameters to have more specific results.
Get information from all Spaces in the Hub. The response is paginated, use the [`Link` header](https://docs.github.com/en/rest/guides/using-pagination-in-the-rest-api?apiVersion=2022-11-28#link-header) to get the next pages. You can specify additional parameters to have more specific results.
- `search`: Filter based on substrings for repos and their usernames, such as `resnet` or `microsoft`
- `author`: Filter models by an author or organization, such as `huggingface` or `microsoft`
- `filter`: Filter based on tags, such as `text-classification` or `spacy`.
- `sort`: Property to use when sorting, such as `downloads` or `author`.
- `direction`: Direction in which to sort, such as `-1` for descending, and anything else for ascending.
- `limit`: Limit the number of models fetched.
- `full`: Whether to fetch most model data, such as all tags, the files, etc.
- `config`: Whether to also fetch the repo config.

Payload:

Expand All @@ -148,11 +147,6 @@ Get all information for a specific model.

This is equivalent to `huggingface_hub.space_info(repo_id, revision)`.

### GET /api/metrics

Get information from all metrics in the Hub.

This is equivalent to `huggingface_hub.list_metrics()`.

## Repo API

Expand Down Expand Up @@ -246,7 +240,7 @@ headers = { "authorization" : "Bearer $token" }

This is equivalent to `huggingface_hub.whoami()`.

## Collection API
## Collections API

Use Collections to group repositories from the Hub (Models, Datasets, Spaces and Papers) on a dedicated page.

Expand Down Expand Up @@ -341,4 +335,4 @@ This is equivalent to `huggingface_hub.update_collection_item()`.

Remove an item from a collection. You must know the item object id which is different from the repo_id/paper_id provided when adding the item to the collection. The `item_id` can be retrieved by fetching the collection.

This is equivalent to `huggingface_hub.delete_collection_item()`.
This is equivalent to `huggingface_hub.delete_collection_item()`.
2 changes: 1 addition & 1 deletion docs/hub/doi.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ If ever there’s a new version of a model or dataset, a new DOI can easily be a

You just need to click on "Generate new DOI" and tadaam!🎉 a new DOI is assigned for the current revision of your model or dataset.

## Why is there 'locked by DOI' message on delete, rename and change visibility action on my model or dataset?
## Why is there a 'locked by DOI' message on delete, rename and change visibility action on my model or dataset?

DOIs make finding information about a model or dataset easier and sharing them with the world via a permanent link that will never expire or change. As such, datasets/models with DOIs are intended to persist perpetually and may only be deleted, renamed and changed their visibility upon filing a request with our support (website at huggingface.co)

Expand Down
4 changes: 1 addition & 3 deletions docs/hub/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Hugging Face Hub documentation

The Hugging Face Hub is a platform with over 120k models, 20k datasets, and 50k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together. The Hub works as a central place where anyone can explore, experiment, collaborate, and build technology with Machine Learning. Are you ready to join the path towards open source Machine Learning? 🤗
The Hugging Face Hub is a platform with over 350k models, 75k datasets, and 150k demo apps (Spaces), all open source and publicly available, in an online platform where people can easily collaborate and build ML together. The Hub works as a central place where anyone can explore, experiment, collaborate, and build technology with Machine Learning. Are you ready to join the path towards open source Machine Learning? 🤗

<div class="grid grid-cols-1 gap-4 sm:grid-cols-2 lg:grid-cols-3 md:mt-10">

Expand All @@ -25,8 +25,6 @@ The Hugging Face Hub is a platform with over 120k models, 20k datasets, and 50k
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-the-hub">The Model Hub</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./model-cards">Model Cards</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-gated">Gated Models</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-libraries">Libraries</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-tasks">Tasks</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-uploading">Uploading Models</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-downloading">Downloading Models</a>
<a class="!no-underline hover:opacity-60 transform transition-colors hover:translate-x-px" href="./models-libraries">Libraries</a>
Expand Down
2 changes: 1 addition & 1 deletion docs/hub/models-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Releasing an update to a model that you've already published can be done by push

## What if I have a different checkpoint of the model trained on a different dataset?

By convention, each model repo should contain a single checkpoint trained on a particular dataset. You should upload any new checkpoints trained on different datasets to the Hub in a new model repo. You can link the models together by using a tag specified in the `tags` key in your [model card's metadata](./model-cards), by using [Collections](./collections) to group distinct related repositories together or by linking to them in the model cards. The [akiyamasho/AnimeBackgroundGAN-Shinkai](https://huggingface.co/akiyamasho/AnimeBackgroundGAN-Shinkai#other-pre-trained-model-versions) model, for example, references other checkpoints in the model card under *"Other pre-trained model versions"*.
By convention, each model repo should contain a single checkpoint. You should upload any new checkpoints trained on different datasets to the Hub in a new model repo. You can link the models together by using a tag specified in the `tags` key in your [model card's metadata](./model-cards), by using [Collections](./collections) to group distinct related repositories together or by linking to them in the model cards. The [akiyamasho/AnimeBackgroundGAN-Shinkai](https://huggingface.co/akiyamasho/AnimeBackgroundGAN-Shinkai#other-pre-trained-model-versions) model, for example, references other checkpoints in the model card under *"Other pre-trained model versions"*.

## Can I link my model to a paper on arXiv?

Expand Down
15 changes: 14 additions & 1 deletion docs/hub/models-widgets.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,11 @@ widget:
text: "Hello my name is Julien"
```

<div class="flex justify-center">
<img class="block dark:hidden" width="450" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-textgen-light.png"/>
<img class="hidden dark:block" width="450" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-textgen-dark.png"/>
</div>

The `output` property should be a YAML dictionary that represents the Inference API output.

For a model that outputs text, see the example above.
Expand All @@ -120,6 +125,11 @@ widget:
score: 0.2
```

<div class="flex justify-center">
<img class="block dark:hidden" width="450" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-audiocls-light.png"/>
<img class="hidden dark:block" width="450" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-audiocls-dark.png"/>
</div>

Finally, for a model that outputs an image, audio, or any other kind of asset, the output should include a `url` property linking to either a file name or path inside the repo or a remote URL. For example, for a text-to-image model:

```yaml
Expand All @@ -129,7 +139,10 @@ widget:
url: images/tiger.jpg
```

<!-- todo(add a screenshot) -->
<div class="flex justify-center">
<img class="block dark:hidden" width="450" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-text2img-light.png"/>
<img class="hidden dark:block" width="450" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/infrence-examples-text2img-dark.png"/>
</div>

We can also surface the example outputs in the Hugging Face UI, for instance, for a text-to-image model to display a gallery of cool image generations.

Expand Down
2 changes: 1 addition & 1 deletion docs/hub/notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ By default, changes made to notifications will only apply to the selected notifi

By default, you'll be watching all the organizations you are a member of and will be notified of any new activity on those.

You can also choose to get notified on arbitrary users or organizations. To do so, use the "Watch" button on their HF profiles. Note that you can also quickly watch/unwatch users and organizations directly from your [notifications settings](#notifications-settings).
You can also choose to get notified on arbitrary users or organizations. To do so, use the "Watch repos" button on their HF profiles. Note that you can also quickly watch/unwatch users and organizations directly from your [notifications settings](#notifications-settings).

_Unlike Github or similar services, you cannot watch a specific repository. You must watch users/organizations to get notified about any new activity on any of their repositories. The goal is to simplify this functionality for users as much as possible and to make sure you don't miss anything you might be interested in._

Expand Down
4 changes: 2 additions & 2 deletions docs/hub/paper-pages.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ If the repository card (`README.md`) includes a link to a paper on arXiv, the Hu

## Claiming authorship to a Paper

The Hub will automatically match paper to users based on their email.
The Hub will attempt to automatically match paper to users based on their email.

<div class="flex justify-center">
<img class="block dark:hidden" width="300" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/papers-authors.png"/>
Expand All @@ -40,7 +40,7 @@ If your paper is not linked to your account, you can click in your name in the c

### Can I control which Paper pages show in my profile?

Yes! You can visit your Papers in [settings](https://huggingface.co/settings/papers), where you will see a list of verified paper. There, you can click the "Show on profile" checkbox to hide/show it in your profile.
Yes! You can visit your Papers in [settings](https://huggingface.co/settings/papers), where you will see a list of verified papers. There, you can click the "Show on profile" checkbox to hide/show it in your profile.

### Do you support ACL anthology?

Expand Down
4 changes: 4 additions & 0 deletions js/src/lib/components/InferenceWidget/shared/WidgetExample.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,7 @@ export type WidgetExample<TOutput = WidgetExampleOutput> =
| WidgetExampleTableDataInput<TOutput>
| WidgetExampleZeroShotTextInput<TOutput>
| WidgetExampleSentenceSimilarityInput<TOutput>;

type KeysOfUnion<T> = T extends any ? keyof T : never;

export type WidgetExampleAttribute = KeysOfUnion<WidgetExample>;
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
<script lang="ts">
import type { WidgetProps, ModelLoadInfo, ExampleRunOpts } from "../types";
import type { WidgetExample } from "../WidgetExample";
import type { QueryParam } from "../../shared/helpers";
import type { WidgetExample, WidgetExampleAttribute } from "../WidgetExample";
type TWidgetExample = $$Generic<WidgetExample>;
Expand Down Expand Up @@ -33,7 +32,7 @@
export let outputJson: string;
export let applyInputSample: (sample: TWidgetExample, opts?: ExampleRunOpts) => void = () => {};
export let validateExample: (sample: WidgetExample) => sample is TWidgetExample;
export let exampleQueryParams: QueryParam[] = [];
export let exampleQueryParams: WidgetExampleAttribute[] = [];
let isDisabled = model.inference !== InferenceDisplayability.Yes && model.pipeline_tag !== "reinforcement-learning";
let isMaximized = false;
Expand Down
12 changes: 5 additions & 7 deletions js/src/lib/components/InferenceWidget/shared/helpers.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
import type { ModelData } from "../../../interfaces/Types";
import { randomItem, parseJSON } from "../../../utils/ViewUtils";
import type { WidgetExample } from "./WidgetExample";
import type { WidgetExample, WidgetExampleAttribute } from "./WidgetExample";
import type { ModelLoadInfo, TableData } from "./types";
import { LoadState } from "./types";

type KeysOfUnion<T> = T extends any ? keyof T : never;
export type QueryParam = KeysOfUnion<WidgetExample>;
const KEYS_TEXT: WidgetExampleAttribute[] = ["text", "context", "candidate_labels"];
const KEYS_TABLE: WidgetExampleAttribute[] = ["table", "structured_data"];
type QueryParamVal = string | null | boolean | (string | number)[][];
const KEYS_TEXT: QueryParam[] = ["text", "context", "candidate_labels"];
const KEYS_TABLE: QueryParam[] = ["table", "structured_data"];

export function getQueryParamVal(key: QueryParam): QueryParamVal {
export function getQueryParamVal(key: WidgetExampleAttribute): QueryParamVal {
const searchParams = new URL(window.location.href).searchParams;
const value = searchParams.get(key);
if (KEYS_TEXT.includes(key)) {
Expand All @@ -35,7 +33,7 @@ export function getWidgetExample<TWidgetExample extends WidgetExample>(
}

// Update current url search params, keeping existing keys intact.
export function updateUrl(obj: Partial<Record<QueryParam, string | undefined>>): void {
export function updateUrl(obj: Partial<Record<WidgetExampleAttribute, string | undefined>>): void {
if (!window) {
return;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,21 +43,21 @@
let scrollTableToRight: () => Promise<void>;
let tableWithOutput: (string | number)[][];
$: {
const strucuredData = convertTableToData(table);
const structuredData = convertTableToData(table);
if (output?.length) {
strucuredData.Prediction = output;
const lastColIndex = Object.keys(strucuredData).length - 1;
structuredData.Prediction = output;
const lastColIndex = Object.keys(structuredData).length - 1;
highlighted = highlightOutput(output, lastColIndex);
scrollTableToRight();
} else {
delete strucuredData.Prediction;
delete structuredData.Prediction;
highlighted = {};
if (highlightErrorKey) {
highlighted[highlightErrorKey] = "bg-red-100 border-red-100 dark:bg-red-800 dark:border-red-800";
highlightErrorKey = "";
}
}
tableWithOutput = convertDataToTable(strucuredData);
tableWithOutput = convertDataToTable(structuredData);
}
const COLORS = ["blue", "green", "yellow", "purple", "red"] as const;
Expand Down

0 comments on commit 8d99fb3

Please sign in to comment.