AWS Bedrock: How to create a chat app using Boto3 Library?

Hello guys, in this blog we will learn how can we create a simple chat application using AWS Bedrock and the AWS Boto3 python library.

We will be creating a simple script which will take the prompt provided by the user as an input and will provide with relavant output.

By following along with the blog you will also learn how you can make a basic API call directly to the AWS Bedrock service.

Table of contents

Prerequisite

In this example we will be using the AWS Bedrock service along with the models provided by the AI21 Labs So before getting started with the example you must request access for the AI21 Labs models.

Also since the bedrock service is not available in all the AWS regions we will be using the us-east-1 region for this example.

Action Items

1. Access for the AI21 Labs models in the us-east-1 region.
2. Generate and configure access keys to execute the below python script. You will need the Bedrock service access.

Lets get the python script ready

1. First install the AWS Boto3 library

pip3 install boto3

2. First we will add the import statement.

   These statements allow us to use the AWS Boto3 library to call Amazon Bedrock.

import json
import boto3

3. Initialize the Bedrock client library.

session = boto3.Session()
#creates a Bedrock client
bedrock = session.client(service_name='bedrock-runtime')

4. Build the payload for the API call.

   Here we are identifying the model to use, the prompt, and the inference parameters for the specified model.

bedrock_model_id = "ai21.j2-ultra-v1" #set the foundation model

prompt = "Which is the largest city in India?" #the prompt to send to the model

 #build the request payload
body = json.dumps({
    "prompt": prompt, #AI21
    "maxTokens": 1024,
    "temperature": 0,
    "topP": 0.5,
    "stopSequences": [],
    "countPenalty": {"scale": 0 },
    "presencePenalty": {"scale": 0 },
    "frequencyPenalty": {"scale": 0 }
})

5. Call the Bedrock API.

   We use Bedrock’s invoke_model function to make the call.

response = bedrock.invoke_model(body=body, modelId=bedrock_model_id, accept='application/json', contentType='application/json') #send the payload to Bedrock

6. Display the response.

   This extracts & prints the returned text from the model’s response JSON.

response_body = json.loads(response.get('body').read()) # read the response

response_text = response_body.get("completions")[0].get("data").get("text") #extract the text from the JSON response

print(response_text)

7. Save the file. Great! Now you are ready to run the script!
8. Run the script using the below command

python3 bedrock_api.py

Finally Here is the code for complete script

import json
import boto3

session = boto3.Session()
bedrock = session.client(service_name='bedrock-runtime') #creates a Bedrock client

bedrock_model_id = "ai21.j2-ultra-v1" #set the foundation model

prompt = "What is the largest city in New Hampshire?" #the prompt to send to the model

body = json.dumps({
    "prompt": prompt, #AI21
    "maxTokens": 1024,
    "temperature": 0,
    "topP": 0.5,
    "stopSequences": [],
    "countPenalty": {"scale": 0 },
    "presencePenalty": {"scale": 0 },
    "frequencyPenalty": {"scale": 0 }
}) #build the request payload


response = bedrock.invoke_model(body=body, modelId=bedrock_model_id, accept='application/json', contentType='application/json') #send the payload to Bedrock

response_body = json.loads(response.get('body').read()) # read the response

response_text = response_body.get("completions")[0].get("data").get("text") #extract the text from the JSON response

print(response_text)

Conclusion

In conclusion, creating a chat app using AWS Bedrock and the Boto3 library is a streamlined process that leverages the robust capabilities of AWS’s AI and ML services.

By following the steps outlined in this blog, you can efficiently set up and deploy a chat application using Python that harnesses the power of GenAI for intelligent interactions. AWS Bedrock provides a flexible and scalable foundation, while Boto3 simplifies the integration and management of AWS services.

Whether you’re a seasoned developer or new to cloud-based AI, this combination offers a powerful solution for building sophisticated chat applications. Explore further possibilities and continue innovating with AWS Bedrock, Boto3, Python, and GenAI to enhance your app’s functionality and user experience.