Using Bedrock Agent Streaming Responds with Access Denied.

[TheShoes]

Describe the issue

I am trying to use the new AWS Bedrock agent streaming. Calling the agent works normally but when i add

  streamingConfigurations: {
    streamFinalResponse: true,
    applyGuardrailInterval: 1000,
  }
  
 I then get Error processing stream: AccessDeniedException: Access denied when calling Bedrock. Check your request permissions and retry the request.
 
 I should have all the permissions correct but I dont see any where on what permission I am missing.

Links

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeAgent.html

[tonisama]

Same issue.

ERROR: Can't invoke code interpreter agent. Reason: An error occurred (accessDeniedException) when calling the InvokeAgent operation: Access denied when calling Bedrock. Check your request permissions and retry the request.

Setting streamFinalResponse: false works fine.

[zshzbh]

Hey @TheShoes

I can reproduce this issue.
The code I have

import { BedrockAgentRuntimeClient, InvokeAgentCommand } from "@aws-sdk/client-bedrock-agent-runtime";

// Create the client
const client = new BedrockAgentRuntimeClient({ 
  region: "us-east-1",
});

// Streaming version
async function invokeAgentWithStreaming() {
  const command = new InvokeAgentCommand({
    agentId: "XXX",
    agentAliasId: "XXXX",
    inputText: "what is bedrock agent?",
    enableTrace: true,
    sessionId: "test",
    streamingConfigurations: {
      streamFinalResponse: false,
      applyGuardrailInterval: 1000
    }
  });

  try {
    const response = await client.send(command);
    
    // Handle the streaming response
    for await (const chunk of response.completion) {
      if (chunk.chunk?.bytes) {
        // Convert the bytes to text
        const textDecoder = new TextDecoder();
        const text = textDecoder.decode(chunk.chunk.bytes);
        console.log("Received chunk:", text);
      }
    }
  } catch (error) {
    console.error("Error in streaming:", error);
    throw error;
  }
}
invokeAgentWithStreaming()

When I set streamFinalResponse to true, I got the error AccessDeniedException: Access denied when calling Bedrock. Check your request permissions and retry the request.
When I set streamFinalResponse to false, I can get the result.

Please set streamFinalResponse to false now to unblock your project. At the same time, I will dive deep into this issue and ask service team to see if there's any special requirement.

Thanks!
Maggie

[zshzbh]

Hey @tonisama @TheShoes

I'm able to figure out the root cause. streamFinalResponse specifies whether to enable streaming for the final response. You need to add the permission policy - bedrock:InvokeModelWithResponseStream to your execution role's policy. The execution role's permission policy would look like -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AmazonBedrockAgentBedrockFoundationModelPolicyProd",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ],
            "Resource": [
                "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20240620-v1:0"
            ]
        }
    ]
}

I added bedrock:InvokeModelWithResponseStream and the issue fixed. Please let me know if you have any other questions.

Thansk!
Maggie

[TheShoes]

@zshzbh I can confirm adding this permission worked.

The only thing I see is that the returned chunks are still pretty big, several sentences. It would be nice if the chunks were smaller to get the response to the user quicker. For example it would stream back every senetence. But that might be at the model level or something else and not sure we would be able to affect that. Just a nice to have.

Thank you for your help

[TheShoes]

@zshzbh I have also noticed that when you add a Knowledge Base to an agent the streaming stops working. The code still works but it seems like aws sends back on large chunk .

I have not tested with multi agents turned on yet but i assume streaming is going to stop working there also.