From 902eff5cf99dc4016f3085e56fa6f559cb650fe6 Mon Sep 17 00:00:00 2001 From: Maisa Rissi Date: Fri, 17 May 2024 16:41:40 -0300 Subject: [PATCH 1/4] Adding manifest generation as part of plugin generation --- specs/cli/plugin-add.md | 69 +++++++++++++++++++++++++++++++++++--- specs/cli/plugin-edit.md | 4 +-- specs/scenarios/plugins.md | 8 ++--- 3 files changed, 71 insertions(+), 10 deletions(-) diff --git a/specs/cli/plugin-add.md b/specs/cli/plugin-add.md index 7059d3658b..faeeea3e30 100644 --- a/specs/cli/plugin-add.md +++ b/specs/cli/plugin-add.md @@ -10,11 +10,18 @@ Every time a plugin is added, a copy of the OpenAPI document file will be stored An [API Manifest][def] file named `apimanifest.json` will be generated (if non existing) or updated (if already existing) in the root folder `./kiota` next to `workspace.json`. API Manifest represents a snapshot of API dependencies and permissions required to access those APIs. This file will represent a concatenated surface of all APIs used across plugins and clients. Both files, `apimanifest.json` and `workspace.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI document location and the properties of the manifest will be generated and would trigger an update to the [API Manifest][def]. -Developers can generate `openai` and `apimanifest` type of plugins. By generating `openai` or `apimanifest`, two outputs will be generated: a\) the plugin type you have chosen and b\) a sliced OpenAPI document named `{plugin-name}-openapi.json|yaml` with only the endpoints that matches `--include-path` and `--exclude-path`, if provided. +Developers can generate `name_to_be_defined`, `openai` and `apimanifest` type of plugins. By generating plugins, three outputs will be generated: 1\) a sliced OpenAPI document named `{plugin-name}-openapi.json|yaml`, 2\) the plugin type you have chosen and 3\) a `manifest.json` file which will follow the [app manifest](https://learn.microsoft.com/en-us/microsoftteams/platform/resources/schema/manifest-schema) schema. > [!NOTE] > In one's solution, there might be two different [API Manifests][def]. The `apimanifest.json` in the `./kiota` folder represents a single artifact surface of all APIs and it will always be generated. The second one, specific to each plugin when providing `--type apimanifest`, will be named `{plugin-name}-apimanifest.json` and saved in the chosen output directory. -Once the `workspace.json` file is generated and the OpenAPI document file is saved locally, the generation will be executed and the plugin and the sliced OpenAPI document will become available. +Once the `workspace.json` file is generated and the OpenAPI document file is saved locally, the generation will be executed and the plugin, the sliced OpenAPI document and the `manifest.json` will become available. + +### Sliced OpenAPI document +The generated sliced OpenAPI document +will include only the endpoints that matches `--include-path` and `--exclude-path`, if provided, along with their referenced components. All unused components will be removed as well as all OpenAPI extensions (starts with x-) that don't match with our internal documentation of supported extensions in Kiota. + +### Plugins +For `name_to_be_defined` plugins, the generated plugin should follow the internal document dedicate to it that explains naming, schema and required fields. For `openai` plugins, the generated plugin will be named `openai-plugins.json` and the mapping should follow [Hidi logic to generate OpenAI Plugin](https://github.com/microsoft/OpenAPI.NET/blob/vnext/src/Microsoft.OpenApi.Hidi/OpenApiService.cs#L748). Requiring fields default as the following: @@ -38,6 +45,60 @@ For `apimanifest`, the generated file will be named `{plugin-name}-apimanifest.j | publisherEmail | Defaults to the contact email from the OpenAPI document. If the contact email is not available, it defaults to 'publisher-email@example.com'. | | | | +### App manifest +For `manifest.json` file, we will: +1. Add a plugin node to the `manifest.json` file if a `manifest.json` file already exists in the output directory. + +```json +"copilotExtensions": { + "plugins": [ + { + "id": {plugin-name} + "file": ".json" + } + ] +}, +``` +2. If a `plugins` node already exists and there is no plugin with the same `id`, add the new plugin information. If the `id` already exists, replace the current content. +3. If there is no `manifest.json` file, we should create a basic manifest with only required information as the following example: + +```json +{ + "$schema": "https://developer.microsoft.com/json-schemas/teams/vDevPreview/MicrosoftTeams.schema.json", + "manifestVersion": "devPreview", + "version": "1.0.0", + "id": "", + "developer": { + "name": "", + "websiteUrl": "", + "privacyUrl": "", + "termsOfUseUrl": "" + }, + "packageName": "com.microsoft.kiota.plugin.", + "name": { + "short": "", + "full": "API Plugin for " + }, + "description": { + "short": "API Plugin for . If the description is not available, it defaults to `API Plugin for `", + "full": "API Plugin for . If the description is not available, it defaults to `API Plugin for `" + }, + "icons": { + "color": "color.png", //we could default it to a color version of Kiota logo 192x192 pixels where the icon symbol is 96x96 pixels or just use the same as [Teams Toolkit](https://github.com/OfficeDev/TeamsFx/blob/dev/templates/js/workflow/appPackage/color.png) + "outline": "outline.png" //we could default it to an outline Kiota icon 32x32 pixels or just use the same as [Teams Toolkit](https://github.com/OfficeDev/TeamsFx/blob/dev/templates/js/workflow/appPackage/outline.png). + }, + "accentColor": "#FFFFFF", //always white + "copilotExtensions": { + "plugins": [ + { + "id": {plugin-name} + "file": ".json" + } + ] + } +} +``` + ## Parameters | Parameters | Required | Example | Description | Telemetry | @@ -46,7 +107,7 @@ For `apimanifest`, the generated file will be named `{plugin-name}-apimanifest.j | `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI document in JSON or YAML format to use to generate the plugin. Accepts a URL or a local directory. | No | | `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value | | `--exclude-path \| -e` | No | /advisories | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value | -| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`.| Yes | +| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `name_to_be_defined`, `openai` and `apimanifest`.| Yes | | `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes | | `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the location of `workspace.json`. Defaults to `./output`. | Yes, without its value | @@ -182,4 +243,4 @@ _The resulting API Manifest named `apimanifest.json` in the `./kiota` folder (co └─github-openapi.json # Sliced and augmented OpenAPI document ``` -[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html +[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html \ No newline at end of file diff --git a/specs/cli/plugin-edit.md b/specs/cli/plugin-edit.md index 407c4cc10c..a0f67dc463 100644 --- a/specs/cli/plugin-edit.md +++ b/specs/cli/plugin-edit.md @@ -18,7 +18,7 @@ Once the `workspace.json` file and the API Manifest are updated, the code genera | `--openapi \| -d` | Yes | https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json | The location of the OpenAPI document in JSON or YAML format to use to generate the plugin. Accepts a URL or a local directory. | Yes, without its value | | `--include-path \| -i` | No | /repos/{owner}/{repo} | A glob pattern to include paths from generation. Accepts multiple values. Defaults to no value which includes everything. | Yes, without its value | | `--exclude-path \| -e` | No | /repos/{owner}/{repo}#DELETE | A glob pattern to exclude paths from generation. Accepts multiple values. Defaults to no value which excludes nothing. | Yes, without its value | -| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `openai` and `apimanifest`.| Yes | +| `--type \| -t` | Yes | openai | The target type of plugin for the generated output files. Accepts multiple values. Possible values are `name_to_be_defined`, `openai` and `apimanifest`.| Yes | | `--skip-generation \| --sg` | No | true | When specified, the generation would be skipped. Defaults to false. | Yes | | `--output \| -o` | No | ./generated/plugins/github | The output directory or file path for the generated output files. This is relative to the current working directory. Defaults to `./output`. | Yes, without its value | @@ -146,4 +146,4 @@ _The resulting API Manifest named `apimanifest.json` in the `./kiota` folder (co └─github-openapi.json # Sliced and augmented OpenAPI document ``` -[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html +[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html \ No newline at end of file diff --git a/specs/scenarios/plugins.md b/specs/scenarios/plugins.md index af5b4f8844..07c9458dee 100644 --- a/specs/scenarios/plugins.md +++ b/specs/scenarios/plugins.md @@ -4,11 +4,11 @@ Kiota generates client code for an API. With the advancement of the AI, API clie Today's AI models can easily generate messages and images for users. While this is helpful when building a simple chat app, it is not enough to build fully automated AI agents that can automate business processe and needs specific to one's company and empower users to achieve more. To do so, users need to combine AI models with other sources, such as APIs. -OpenAI has defined [OpenAI plugins](https://platform.openai.com/docs/plugins/introduction), one way to enable GPT to interact with APIs, allowing it to perform several actions. To build a plugin, it's necessary to create a [plugin manifest file](https://platform.openai.com/docs/plugins/getting-started/plugin-manifest) that defines relevant metadata information that allows GPT to call an API. +OpenAI has defined [OpenAI plugins](https://platform.openai.com/docs/plugins/introduction), which used to be the way to enable GPT to interact with APIs. OpenAI plugins are now deprecated, and OpenAI has moved to [`actions in GPTs`](https://platform.openai.com/docs/actions/introduction). -In addition to OpenAI plugin manifest, [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html) is another way to declare dependencies of APIs and their characteristics. API Manifest addresses a limitation present in both the OpenAI plugin manifest and OpenAPI document, it can references one or more OpenAPI documents as dependencies. +In addition to OpenAI plugin manifest and actions in GPTs, [API Manifest](https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html) is another way to declare dependencies of APIs and their characteristics. API Manifest addresses a limitation present in both OpenAI plugin manifest and OpenAPI document, it can references one or more OpenAPI documents as dependencies. For developers using [Semantic Kernel]( -https://learn.microsoft.com/en-us/semantic-kernel/overview/) as their AI orchestractor, [API Manifest is supported as a input format](https://github.com/microsoft/semantic-kernel/pull/4961), in preview state, for plugin generation. +https://learn.microsoft.com/en-us/semantic-kernel/overview/) as their AI orchestractor, [API Manifest is also supported as a input format](https://github.com/microsoft/semantic-kernel/pull/4961), in preview state, for plugin generation. ## Current Challenges @@ -18,7 +18,7 @@ https://learn.microsoft.com/en-us/semantic-kernel/overview/) as their AI orchest ## Goals -- Enable developers to customize Copilot to be more helpful in their daily lives, at specific tasks, at work, at home by providing tools to output OpenAI plugin manifests. +- Enable developers to customize Copilot to be more helpful in their daily lives, at specific tasks, at work, at home by providing tools to output API plugins. - Enable developers to generate API Manifest that can be converted into Semantic Kernel API Manifest Plugins. ## Proposal From 1c359a6997c0fbfca9637d5056eb8e1f20810ac8 Mon Sep 17 00:00:00 2001 From: Maisa Rissi Date: Mon, 20 May 2024 12:24:11 -0300 Subject: [PATCH 2/4] Adding more info around app manifest --- specs/cli/plugin-add.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/specs/cli/plugin-add.md b/specs/cli/plugin-add.md index faeeea3e30..d7cb80b9eb 100644 --- a/specs/cli/plugin-add.md +++ b/specs/cli/plugin-add.md @@ -10,7 +10,7 @@ Every time a plugin is added, a copy of the OpenAPI document file will be stored An [API Manifest][def] file named `apimanifest.json` will be generated (if non existing) or updated (if already existing) in the root folder `./kiota` next to `workspace.json`. API Manifest represents a snapshot of API dependencies and permissions required to access those APIs. This file will represent a concatenated surface of all APIs used across plugins and clients. Both files, `apimanifest.json` and `workspace.json` will be used to generate the code files. A new hash composed of the Kiota version, the OpenAPI document location and the properties of the manifest will be generated and would trigger an update to the [API Manifest][def]. -Developers can generate `name_to_be_defined`, `openai` and `apimanifest` type of plugins. By generating plugins, three outputs will be generated: 1\) a sliced OpenAPI document named `{plugin-name}-openapi.json|yaml`, 2\) the plugin type you have chosen and 3\) a `manifest.json` file which will follow the [app manifest](https://learn.microsoft.com/en-us/microsoftteams/platform/resources/schema/manifest-schema) schema. +Developers can generate `name_to_be_defined`, `openai` and `apimanifest` type of plugins. By generating plugins, three outputs will be generated: 1\) a sliced OpenAPI document named `{plugin-name}-openapi.json|yaml`, 2\) the plugin type you have chosen and 3\) an [app manifest](https://learn.microsoft.com/en-us/microsoftteams/platform/resources/schema/manifest-schema) file named `manifest.json` which conforms with the schema https://learn.microsoft.com/en-us/microsoftteams/platform/resources/schema/manifest-schema. > [!NOTE] > In one's solution, there might be two different [API Manifests][def]. The `apimanifest.json` in the `./kiota` folder represents a single artifact surface of all APIs and it will always be generated. The second one, specific to each plugin when providing `--type apimanifest`, will be named `{plugin-name}-apimanifest.json` and saved in the chosen output directory. @@ -46,6 +46,7 @@ For `apimanifest`, the generated file will be named `{plugin-name}-apimanifest.j | | | ### App manifest +The app manifest file describes how one's plugin integrates into Microsoft 365 and it's not related to plugin types. App manifests are required for [publishing apps to Microsoft 365 app stores](https://learn.microsoft.com/en-us/partner-center/marketplace/checklist#step-3-check-that-your-manifest-is-compliant). For `manifest.json` file, we will: 1. Add a plugin node to the `manifest.json` file if a `manifest.json` file already exists in the output directory. @@ -99,6 +100,8 @@ For `manifest.json` file, we will: } ``` +4. [Validate](https://learn.microsoft.com/en-us/office/dev/add-ins/testing/troubleshoot-manifest) the generated `manifest.json` file. + ## Parameters | Parameters | Required | Example | Description | Telemetry | From 5a391c2c490ea4be6859da20c2d50a2d1136bd33 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ma=C3=ADsa=20Rissi?= Date: Thu, 23 May 2024 11:34:29 -0300 Subject: [PATCH 3/4] Added manifest.json in the file structure --- specs/cli/plugin-add.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/specs/cli/plugin-add.md b/specs/cli/plugin-add.md index d7cb80b9eb..14f944e0a7 100644 --- a/specs/cli/plugin-add.md +++ b/specs/cli/plugin-add.md @@ -241,9 +241,10 @@ _The resulting API Manifest named `apimanifest.json` in the `./kiota` folder (co └─generated └─plugins └─github + └─manifest.json # App manifest └─github-apimanifest.json # Specific API Manifest └─openai-plugins.json #OpenAI Plugin └─github-openapi.json # Sliced and augmented OpenAPI document ``` -[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html \ No newline at end of file +[def]: https://www.ietf.org/archive/id/draft-miller-api-manifest-01.html From 95db74121fc643f44df34e0f510eea5005f32464 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ma=C3=ADsa=20Rissi?= Date: Thu, 23 May 2024 11:35:14 -0300 Subject: [PATCH 4/4] Apply suggestions from code review Co-authored-by: Vincent Biret --- specs/cli/plugin-add.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/cli/plugin-add.md b/specs/cli/plugin-add.md index 14f944e0a7..79fae0b1ff 100644 --- a/specs/cli/plugin-add.md +++ b/specs/cli/plugin-add.md @@ -54,7 +54,7 @@ For `manifest.json` file, we will: "copilotExtensions": { "plugins": [ { - "id": {plugin-name} + "id": "{plugin-name}", "file": ".json" } ] @@ -63,7 +63,7 @@ For `manifest.json` file, we will: 2. If a `plugins` node already exists and there is no plugin with the same `id`, add the new plugin information. If the `id` already exists, replace the current content. 3. If there is no `manifest.json` file, we should create a basic manifest with only required information as the following example: -```json +```jsonc { "$schema": "https://developer.microsoft.com/json-schemas/teams/vDevPreview/MicrosoftTeams.schema.json", "manifestVersion": "devPreview",