Skip to content

Commit

Permalink
Exercise 5 updates
Browse files Browse the repository at this point in the history
  • Loading branch information
feaselkl committed Nov 21, 2024
1 parent 41fe8e4 commit 498bba8
Show file tree
Hide file tree
Showing 3 changed files with 245 additions and 170 deletions.
46 changes: 23 additions & 23 deletions docs/04_implement_audio_transcription/04_01.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,29 +53,29 @@ The Contoso Suites development team tried to implement transcription functionali

The code to complete the `create_transcription_request()` function is as follows:

```python
# Subscribe to the events fired by the conversation transcriber
transcriber.transcribed.connect(handle_final_result)
transcriber.session_started.connect(lambda evt: print(f'SESSION STARTED: {evt}'))
transcriber.session_stopped.connect(lambda evt: print(f'SESSION STOPPED {evt}'))
transcriber.canceled.connect(lambda evt: print(f'CANCELED {evt}'))
# stop continuous transcription on either session stopped or canceled events
transcriber.session_stopped.connect(stop_cb)
transcriber.canceled.connect(stop_cb)

transcriber.start_transcribing_async()

# Read the whole wave files at once and stream it to sdk
_, wav_data = wavfile.read(audio_file)
stream.write(wav_data.tobytes())
stream.close()
while not done:
time.sleep(.5)

transcriber.stop_transcribing_async()
```

This code satisfies all of the `TODO` blocks and should go immediately after the `stop_cb()` function and before `return all_results`.
```python
# Subscribe to the events fired by the conversation transcriber
transcriber.transcribed.connect(handle_final_result)
transcriber.session_started.connect(lambda evt: print(f'SESSION STARTED: {evt}'))
transcriber.session_stopped.connect(lambda evt: print(f'SESSION STOPPED {evt}'))
transcriber.canceled.connect(lambda evt: print(f'CANCELED {evt}'))
# stop continuous transcription on either session stopped or canceled events
transcriber.session_stopped.connect(stop_cb)
transcriber.canceled.connect(stop_cb)

transcriber.start_transcribing_async()

# Read the whole wave files at once and stream it to sdk
_, wav_data = wavfile.read(audio_file)
stream.write(wav_data.tobytes())
stream.close()
while not done:
time.sleep(.5)

transcriber.stop_transcribing_async()
```

This code satisfies all of the `TODO` blocks and should go immediately after the `stop_cb()` function and before `return all_results`.

</details>

Expand Down
142 changes: 86 additions & 56 deletions docs/05_enhance_api_with_semantic_kernel/05_01.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,6 @@ One of the primary features of Semantic Kernel is its ability to add different A

In exercise 3, you used the Azure OpenAI SDK to create a client capable of generating embeddings. In this task, you will update the `VectorizationService` and its `GetEmbeddings` function to take advantage of the built-in embedding service offered by Semantic Kernel. You will update the code in that service to replace the `AzureOpenAIClient` and its associated calls with a `Kernel` service to perform text embedding.

The key tasks are as follows:

1. Add the Text Embedding Generation built-in service to the Kernel builder in the `Program.cs` file. Suppress any warnings using `# pragma` directives.
2. Remove the `AzureOpenAIClient` singleton service from `Program.cs`, as it will no longer be necessary.
3. Update the `VectorizationService` primary constructor to remove the `AzureOpenAIClient` and inject the `Kernel` service.
4. Change the `GetEmbeddings` method to use the built-in vector embedding capabilities of Semantic Kernel.
5. Test the updated `VectorizationService` by running the Streamlit dashboard and executing the following maintenance request query on the **Vector Search** page:
- Enter a search query of "Show me requests relating to elevator noises" with max results of ten and a minimum similarity score of 0.78.

## Success Criteria

- You have added the Azure OpenAI Text Embedding Generation service to the Kernel builder in the `Program.cs` file.
Expand All @@ -47,13 +38,18 @@ The key tasks are as follows:
{: .note }
> The update may take up to 10 minutes before you see the change apply.
## Solution
## Key Tasks

### 01: add the Text Embedding Generation service

Add the Text Embedding Generation built-in service to the Kernel builder in the `Program.cs` file. Suppress any warnings using `# pragma` directives.

<details markdown="block">
<summary>Expand this section to view the solution</summary>

- To update the `Program.cs` file to add the built-in `TextEmbeddingGeneration` service:
- Add `AddAzureOpenAITextEmbeddingGeneration(embeddingDeploymentName, endpoint, key)` to the `Kernel` singleton builder call added in Exercise 2 Task 2 (line 224 of the 0202.MD file).
Update the `Program.cs` file to add the built-in `TextEmbeddingGeneration` service:

1. Add `AddAzureOpenAITextEmbeddingGeneration(embeddingDeploymentName, endpoint, key)` to the `Kernel` singleton builder call added in Exercise 2 Task 2 (line 224 of the 0202.MD file).

```csharp
kernelBuilder.AddAzureOpenAITextEmbeddingGeneration(
Expand All @@ -66,12 +62,12 @@ The key tasks are as follows:
The code will be underlined by a red squiggly line, indicating an issue.

The `AddAzureOpenAITextEmbeddingGeneration` feature of Semantic Kernel is flagged for evaluation purposes only at this time, so you must suppress that issue to be able to use it in the project.
- To suppress the issue, hover your mouse cursor over the `AddAzureOpenAITextEmbeddingGeneration` function call and select the dropdown in the tooltip icon that appears.
- In the flyout menu, select **Suppress of configure issues -> Suppress SKEXP0010** and then select **in Source**.
2. To suppress the issue, hover your mouse cursor over the `AddAzureOpenAITextEmbeddingGeneration` function call and select the dropdown in the tooltip icon that appears.
1. In the flyout menu, select **Suppress of configure issues -> Suppress SKEXP0010** and then select **in Source**.

![In Program.cs, the tip button for the inserted code is highlighted, and Suppress or configure issues items are highlighted in the flyout menu. Suppress SKEXP0010 and in Source are highlighted in submenus.](../../media/Solution/0501-suppress-issue-in-source.png)

- This will wrap the code in `#pragma warning disable` and `#pragma warning restore` directives, and the final code will look like this:
2. This will wrap the code in `#pragma warning disable` and `#pragma warning restore` directives, and the final code will look like this:

```csharp
#pragma warning disable SKEXP0010 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
Expand All @@ -83,75 +79,109 @@ The key tasks are as follows:
#pragma warning restore SKEXP0010 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
```

- To remove the `AzureOpenAIClient` singleton service from `Program.cs`:
- In `Program.cs`, locate the line that starts with `builder.Services.AddSingleton<AzureOpenAIClient>` and delete that line and the code block it encapsulates.
</details>

- To update the `VectorizationService` primary constructor to remove the `AzureOpenAIClient` and inject the `Kernel` service:
- Update the library references in the file:
- Remove the `Azure.AI.OpenAI` using statement.
- Add the following Semantic Kernel references:
### 02: Remove AzureOpenAIClient service

```csharp
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Embeddings;
```
Remove the `AzureOpenAIClient` singleton service from `Program.cs`, as it will no longer be necessary.

<details markdown="block">
<summary>Expand this section to view the solution</summary>

To remove the `AzureOpenAIClient` singleton service from `Program.cs`, locate the line that starts with `builder.Services.AddSingleton<AzureOpenAIClient>` and delete that line and the code block it encapsulates.

- Change the primary constructor for the `VectorizationService` to swap out the use of `AzureOpenAIClient` for Semantic Kernel, deleting the injected `AzureOpenAIClient` and replacing it with `Kernel`. The new primary constructor for the class will look like:
</details>

### 03: Update the vectorization service

Update the `VectorizationService` primary constructor to remove the `AzureOpenAIClient` and inject the `Kernel` service.

<details markdown="block">
<summary>Expand this section to view the solution</summary>

To update the `VectorizationService` primary constructor to remove the `AzureOpenAIClient` and inject the `Kernel` service, perform the following steps:

1. Remove the `Azure.AI.OpenAI` using statement.
2. Add the following Semantic Kernel references:

```csharp
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Embeddings;
```

3. Change the primary constructor for the `VectorizationService` to swap out the use of `AzureOpenAIClient` for Semantic Kernel, deleting the injected `AzureOpenAIClient` and replacing it with `Kernel`. The new primary constructor for the class will look like:

```csharp
public class VectorizationService(Kernel kernel, CosmosClient cosmosClient, IConfiguration configuration) : IVectorizationService
```

- Delete the `_client` class variable and create a new one named `_kernel` with a type of `Kernel`.
4. Delete the `_client` class variable and create a new one named `_kernel` with a type of `Kernel`.

```csharp
private readonly Kernel _kernel = kernel;
```

- The steps to update the `VectorizationService` to use the built-in vector embedding capabilities of Semantic Kernel:
</details>

- In the `GetEmbeddings` function:
- Remove the `embeddingClient` variable, as it is no longer necessary.
- Replace the call to the embedding client for the built-in `GenerateEmbeddingAsync()` method of Semantic Kernel.
### 04: Update the GetEmbeddings method

```csharp
// Generate a vector for the provided text.
var embeddings = await _kernel.GetRequiredService<ITextEmbeddingGenerationService>().GenerateEmbeddingAsync(text);
```
Change the `GetEmbeddings` method to use the built-in vector embedding capabilities of Semantic Kernel.

- As you did in the `Program.cs` class, you must suppress the issue with the `ITextEmbeddingGenerationService` feature using `# pragma warning` directives to use it in this class. The final code will look similar to the following:
<details markdown="block">
<summary>Expand this section to view the solution</summary>

```csharp
#pragma warning disable SKEXP0001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Generate a vector for the provided text.
var embeddings = await _kernel.GetRequiredService<ITextEmbeddingGenerationService>().GenerateEmbeddingAsync(text);
#pragma warning restore SKEXP0001
```
The steps to update the `VectorizationService` to use the built-in vector embedding capabilities of Semantic Kernel are:

- Update the `vector` variable to handle the different type returned by the `GenerateEmbeddingsAsync` method.
1. In the `GetEmbeddings` function, remove the `embeddingClient` variable, as it is no longer necessary.
2. Replace the call to the embedding client for the built-in `GenerateEmbeddingAsync()` method of Semantic Kernel.

```csharp
var vector = embeddings.ToArray();
```
```csharp
// Generate a vector for the provided text.
var embeddings = await _kernel.GetRequiredService<ITextEmbeddingGenerationService>().GenerateEmbeddingAsync(text);
```

3. As you did in the `Program.cs` class, you must suppress the issue with the `ITextEmbeddingGenerationService` feature using `# pragma warning` directives to use it in this class. The final code will look similar to the following:

- Save the `VectorizationService` file.
```csharp
#pragma warning disable SKEXP0001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
// Generate a vector for the provided text.
var embeddings = await _kernel.GetRequiredService<ITextEmbeddingGenerationService>().GenerateEmbeddingAsync(text);
#pragma warning restore SKEXP0001
```

- To test the updated `VectorizationService`, start the web API and run the Steamlit dashboard.
- In Visual Studio Code, open a new terminal window and change the directory to `scr\ContotoSuitesWebAPI`.
- At the terminal prompt, enter the following command to run the API locally:
4. Update the `vector` variable to handle the different type returned by the `GenerateEmbeddingsAsync` method.

```bash
dotnet run
```csharp
var vector = embeddings.ToArray();
```

- Once the API has started, as indicated by output in the terminal stating `Now listening on: http://localhost:5292`, open a new terminal window in Visual Studio Code, navigate to the `src\ContosoSuitesDashboard` folder, and run the following command to start the Streamlit dashboard:
5. Save the `VectorizationService` file.

</details>

### 05: Test the service

Test the updated `VectorizationService` by running the Streamlit dashboard and executing the following maintenance request query on the **Vector Search** page. Enter a search query of "Show me requests relating to elevator noises" with max results of ten and a minimum similarity score of 0.78.

<details markdown="block">
<summary>Expand this section to view the solution</summary>

1. To test the updated `VectorizationService`, start the web API and run the Steamlit dashboard.
1. In Visual Studio Code, open a new terminal window and change the directory to `scr\ContotoSuitesWebAPI`.
2. At the terminal prompt, enter the following command to run the API locally:

```bash
dotnet run
```

2. Once the API has started, as indicated by output in the terminal stating `Now listening on: http://localhost:5292`, open a new terminal window in Visual Studio Code, navigate to the `src\ContosoSuitesDashboard` folder, and run the following command to start the Streamlit dashboard:
```python
python -m streamlit run Index.py
```

- Navigate to the **Vector Search** page in the Streamlit dashboard, and then submit the following query for maintenance requests:
- Enter a search query of "Show me requests relating to elevator noises" with max results of ten and a minimum similarity score of 0.78.
- You should see several results about elevators and noise, indicating the updates to the API and `VectorizationService` are working correctly.
3. Navigate to the **Vector Search** page in the Streamlit dashboard, and then submit the following query for maintenance requests:
1. Enter a search query of "Show me requests relating to elevator noises" with max results of ten and a minimum similarity score of 0.78.
2. You should see several results about elevators and noise, indicating the updates to the API and `VectorizationService` are working correctly.

</details>
Loading

0 comments on commit 498bba8

Please sign in to comment.