[Python] InvokeAgent API - Send images to AWS BEDROCK Agent

Is there a way I can send image files as base64 or s3 repository link to a BedRock Agent using InvokeAgent API?

I've noted that in the syntax we have this part:

response = client.invoke_agent(
agentAliasId='string',
agentId='string',
enableTrace=True|False,
endSession=True|False,
inputText='string',
memoryId='string',
sessionId='string',
sessionState={
'files': [
{
'name': 'string',
'source': {
'byteContent': {
'data': b'bytes',
'mediaType': 'string'
},
's3Location': {
'uri': 'string'
},
'sourceType': 'S3'|'BYTE_CONTENT'
},
'useCase': 'CODE_INTERPRETER'|'CHAT'
},
],



Unfortunately I didn't find any documents or examples on how to fill these parts.

I've tried passing a base64 image in

'files': [
{
'name': 'uploaded_picture',
'source': {
'byteContent': {
'data': b'{base64}',
'mediaType': 'image/jpeg'
},

'sourceType': 'BYTE_CONTENT'
},
'useCase': 'CHAT'
},
],


But Bedrock returned me the error:

EventStreamError: An error occurred (validationException) when calling the InvokeAgent operation: The overridden prompt that you provided is incorrectly formatted. Check the format for errors, such as invalid JSON, and retry your request.


Hi There.

Is there a way I can send image files as base64 or s3 repository link to a BedRock Agent using InvokeAgent API?

I've noted that in the syntax we have this part:

response = client.invoke_agent(
agentAliasId='string',
agentId='string',
enableTrace=True|False,
endSession=True|False,
inputText='string',
memoryId='string',
sessionId='string',
sessionState={
'files': [
{
'name': 'string',
'source': {
'byteContent': {
'data': b'bytes',
'mediaType': 'string'
},
's3Location': {
'uri': 'string'
},
'sourceType': 'S3'|'BYTE_CONTENT'
},
'useCase': 'CODE_INTERPRETER'|'CHAT'
},
],



Unfortunately I didn't find any documents or examples on how to fill these parts.

I've tried passing a base64 image in

'files': [
{
'name': 'uploaded_picture',
'source': {
'byteContent': {
'data': b'{base64}',
'mediaType': 'image/jpeg'
},

'sourceType': 'BYTE_CONTENT'
},
'useCase': 'CHAT'
},
],



But Bedrock returned me the error:

EventStreamError: An error occurred (validationException) when calling the InvokeAgent operation: The overridden prompt that you provided is incorrectly formatted. Check the format for errors, such as invalid JSON, and retry your request.



Same error occurs in Bedrock UI.

enter image description here


Here is the full Python code:

import boto3

# Initialize the boto3 client
agents_runtime_client = boto3.client('bedrock-agent-runtime', region_name='sa-east-1')


def invoke_agent(agent_id, agent_alias_id, session_id, prompt, image_path = None):

image_data = None
if image_path:
with open(image_path, "rb") as image_file:
image_data = base64.b64encode(image_file.read()).decode('utf-8')

# Note: The execution time depends on the foundation model, complexity of the agent,
# and the length of the prompt. In some cases, it can take up to a minute or more to
# generate a response.
response = agents_runtime_client.invoke_agent(
agentId=agent_id,
agentAliasId=agent_alias_id,
sessionId=session_id,
inputText=input_text,
sessionState={
'files': [
{
'name': 'picture',
'source': {
'byteContent': {
'data': image_data,
'mediaType': 'image/jpeg'
},
'sourceType': 'BYTE_CONTENT'
},
'useCase': 'CHAT'
}
]
}
)


completion = ""

for event in response.get("completion"):
chunk = event["chunk"]
completion = completion + chunk["bytes"].decode()

return completion


# Define the chat parameters
agent_alias_id = "XXXXX"
agent_id = "XXXXX"
session_id = str(uuid.uuid4())
input_text = "Hello"


I've tried reading the documentation and using the parameters provided.

I'm expecting that I can send images for an Agent to interpretate.

Continue reading...