Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding manifest generation as part of plugin generation #4677

Merged
merged 6 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 68 additions & 3 deletions specs/cli/plugin-add.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,18 @@ Every time a plugin is added, a copy of the OpenAPI document file will be stored

An [API Manifest][def] file named `apimanifest.json` will be generated (if non existing) or updated (if already existing) in the root folder `./kiota` next to `workspace.json`. API Manifest represents a snapshot of API dependencies and permissions required to access those APIs. This file will represent a concatenated surface of all APIs used across plugins and clients. Both files, `apimanifest.json` and `workspace.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI document location and the properties of the manifest will be generated and would trigger an update to the [API Manifest][def].

Developers can generate `openai` and `apimanifest` type of plugins. By generating `openai` or `apimanifest`, two outputs will be generated: a\) the plugin type you have chosen and b\) a sliced OpenAPI document named `{plugin-name}-openapi.json|yaml` with only the endpoints that matches `--include-path` and `--exclude-path`, if provided.
Developers can generate `name_to_be_defined`, `openai` and `apimanifest` type of plugins. By generating plugins, three outputs will be generated: 1\) a sliced OpenAPI document named `{plugin-name}-openapi.json|yaml`, 2\) the plugin type you have chosen and 3\) an [app manifest](https://learn.microsoft.com/en-us/microsoftteams/platform/resources/schema/manifest-schema) file named `manifest.json` which conforms with the schema https://learn.microsoft.com/en-us/microsoftteams/platform/resources/schema/manifest-schema.
> [!NOTE]
> In one's solution, there might be two different [API Manifests][def]. The `apimanifest.json` in the `./kiota` folder represents a single artifact surface of all APIs and it will always be generated. The second one, specific to each plugin when providing `--type apimanifest`, will be named `{plugin-name}-apimanifest.json` and saved in the chosen output directory.

Once the `workspace.json` file is generated and the OpenAPI document file is saved locally, the generation will be executed and the plugin and the sliced OpenAPI document will become available.
Once the `workspace.json` file is generated and the OpenAPI document file is saved locally, the generation will be executed and the plugin, the sliced OpenAPI document and the `manifest.json` will become available.

### Sliced OpenAPI document
The generated sliced OpenAPI document
will include only the endpoints that matches `--include-path` and `--exclude-path`, if provided, along with their referenced components. All unused components will be removed as well as all OpenAPI extensions (starts with x-) that don't match with our internal documentation of supported extensions in Kiota.

### Plugins
For `name_to_be_defined` plugins, the generated plugin should follow the internal document dedicate to it that explains naming, schema and required fields.

For `openai` plugins, the generated plugin will be named `openai-plugins.json` and the mapping should follow [Hidi logic to generate OpenAI Plugin](https://github.com/microsoft/OpenAPI.NET/blob/vnext/src/Microsoft.OpenApi.Hidi/OpenApiService.cs#L748). Requiring fields default as the following:

Expand All @@ -38,6 +45,63 @@ For `apimanifest`, the generated file will be named `{plugin-name}-apimanifest.j
| publisherEmail | Defaults to the contact email from the OpenAPI document. If the contact email is not available, it defaults to '[email protected]'. |
| | |

### App manifest
The app manifest file describes how one's plugin integrates into Microsoft 365 and it's not related to plugin types. App manifests are required for [publishing apps to Microsoft 365 app stores](https://learn.microsoft.com/en-us/partner-center/marketplace/checklist#step-3-check-that-your-manifest-is-compliant).
For `manifest.json` file, we will:
1. Add a plugin node to the `manifest.json` file if a `manifest.json` file already exists in the output directory.

```json
"copilotExtensions": {
"plugins": [
{
"id": "{plugin-name}",
"file": "<generated_plugin_file>.json"
}
]
},
```
2. If a `plugins` node already exists and there is no plugin with the same `id`, add the new plugin information. If the `id` already exists, replace the current content.
3. If there is no `manifest.json` file, we should create a basic manifest with only required information as the following example:

```jsonc
{
"$schema": "https://developer.microsoft.com/json-schemas/teams/vDevPreview/MicrosoftTeams.schema.json",
"manifestVersion": "devPreview",
"version": "1.0.0",
"id": "<generated_GUID>",
"developer": {
"name": "<Defaults to `contact.name` from the OpenAPI document. If the `contact.name` is not available, it defaults to `Kiota Generator, Inc.`>",
"websiteUrl": "<Defaults to `contact.url` from the OpenAPI document. If the `contact.url` is not available, it defaults to `https://www.example.com/contact/`>",
"privacyUrl": "<Defaults to `x-privacy-policy-url` extension from the OpenAPI document. If the `x-privacy-policy-url` is not available, it defaults to `https://www.example.com/privacy/`>",
"termsOfUseUrl": "<Defaults to `termsOfService` from the OpenAPI document. If the `termsOfService` is not available, it defaults to `https://www.example.com/terms/`>"
},
"packageName": "com.microsoft.kiota.plugin.<pluginame>",
"name": {
"short": "<plugin_name>",
"full": "API Plugin <plugin_name> for <OpenAPI document title>"
},
"description": {
"short": "API Plugin for <description from the OpenAPI document>. If the description is not available, it defaults to `API Plugin for <OpenAPI document title>`",
"full": "API Plugin for <description from the OpenAPI document>. If the description is not available, it defaults to `API Plugin for <OpenAPI document title>`"
},
"icons": {
"color": "color.png", //we could default it to a color version of Kiota logo 192x192 pixels where the icon symbol is 96x96 pixels or just use the same as [Teams Toolkit](https://github.com/OfficeDev/TeamsFx/blob/dev/templates/js/workflow/appPackage/color.png)
"outline": "outline.png" //we could default it to an outline Kiota icon 32x32 pixels or just use the same as [Teams Toolkit](https://github.com/OfficeDev/TeamsFx/blob/dev/templates/js/workflow/appPackage/outline.png).
},
"accentColor": "#FFFFFF", //always white
"copilotExtensions": {
"plugins": [
{
"id": {plugin-name}
"file": "<generated_plugin_file>.json"
}
]
}
}
```

4. [Validate](https://learn.microsoft.com/en-us/office/dev/add-ins/testing/troubleshoot-manifest) the generated `manifest.json` file.

## Parameters

| Parameters | Required | Example | Description | Telemetry |
Expand All @@ -46,7 +110,7 @@ For `apimanifest`, the generated file will be named `{plugin-name}-apimanifest.j
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI document in JSON or YAML format to use to generate the plugin. Accepts a URL or a local directory. | No |
| `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value |
| `--exclude-path \| -e` | No | /advisories | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value |
| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`.| Yes |
| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `name_to_be_defined`, `openai` and `apimanifest`.| Yes |
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes |
| `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the location of `workspace.json`. Defaults to `./output`. | Yes, without its value |
maisarissi marked this conversation as resolved.
Show resolved Hide resolved

Expand Down Expand Up @@ -177,6 +241,7 @@ _The resulting API Manifest named `apimanifest.json` in the `./kiota` folder (co
└─generated
└─plugins
└─github
└─manifest.json # App manifest
└─github-apimanifest.json # Specific API Manifest
└─openai-plugins.json #OpenAI Plugin
└─github-openapi.json # Sliced and augmented OpenAPI document
Expand Down
4 changes: 2 additions & 2 deletions specs/cli/plugin-edit.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Once the `workspace.json` file and the API Manifest are updated, the code genera
| `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI document in JSON or YAML format to use to generate the plugin. Accepts a URL or a local directory. | Yes, without its value |
| `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value |
| `--exclude-path \| -e` | No | /repos/{owner}/{repo}#DELETE | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value |
| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`.| Yes |
| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `name_to_be_defined`, `openai` and `apimanifest`.| Yes |
| `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes |
| `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the current working directory. Defaults to `./output`. | Yes, without its value |

Expand Down Expand Up @@ -146,4 +146,4 @@ _The resulting API Manifest named `apimanifest.json` in the `./kiota` folder (co
└─github-openapi.json # Sliced and augmented OpenAPI document
```

[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html
[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html
8 changes: 4 additions & 4 deletions specs/scenarios/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ Kiota generates client code for an API. With the advancement of the AI, API clie

Today's AI models can easily generate messages and images for users. While this is helpful when building a simple chat app, it is not enough to build fully automated AI agents that can automate business processe and needs specific to one's company and empower users to achieve more. To do so, users need to combine AI models with other sources, such as APIs.

OpenAI has defined [OpenAI plugins](https://platform.openai.com/docs/plugins/introduction), one way to enable GPT to interact with APIs, allowing it to perform several actions. To build a plugin, it's necessary to create a [plugin manifest file](https://platform.openai.com/docs/plugins/getting-started/plugin-manifest) that defines relevant metadata information that allows GPT to call an API.
OpenAI has defined [OpenAI plugins](https://platform.openai.com/docs/plugins/introduction), which used to be the way to enable GPT to interact with APIs. OpenAI plugins are now deprecated, and OpenAI has moved to [`actions in GPTs`](https://platform.openai.com/docs/actions/introduction).

In addition to OpenAI plugin manifest, [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html) is another way to declare dependencies of APIs and their characteristics. API Manifest addresses a limitation present in both the OpenAI plugin manifest and OpenAPI document, it can references one or more OpenAPI documents as dependencies.
In addition to OpenAI plugin manifest and actions in GPTs, [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html) is another way to declare dependencies of APIs and their characteristics. API Manifest addresses a limitation present in both OpenAI plugin manifest and OpenAPI document, it can references one or more OpenAPI documents as dependencies.
For developers using [Semantic Kernel](
https://learn.microsoft.com/en-us/semantic-kernel/overview/) as their AI orchestractor, [API Manifest is supported as a input format](https://github.com/microsoft/semantic-kernel/pull/4961), in preview state, for plugin generation.
https://learn.microsoft.com/en-us/semantic-kernel/overview/) as their AI orchestractor, [API Manifest is also supported as a input format](https://github.com/microsoft/semantic-kernel/pull/4961), in preview state, for plugin generation.


## Current Challenges
Expand All @@ -18,7 +18,7 @@ https://learn.microsoft.com/en-us/semantic-kernel/overview/) as their AI orchest

## Goals

- Enable developers to customize Copilot to be more helpful in their daily lives, at specific tasks, at work, at home by providing tools to output OpenAI plugin manifests.
- Enable developers to customize Copilot to be more helpful in their daily lives, at specific tasks, at work, at home by providing tools to output API plugins.
- Enable developers to generate API Manifest that can be converted into Semantic Kernel API Manifest Plugins.

## Proposal
Expand Down
Loading