Cloud Services for Generative AI: AWS Offerings
A practical map of AWS's generative AI services: Amazon Bedrock and Nova, Knowledge Bases, Guardrails, AgentCore, SageMaker AI, Trainium, Kiro and pricing.
AWS sells generative AI at three levels. At the top are finished assistants you buy per user. In the middle is Amazon Bedrock, where you call hosted models through an API and pay per token. At the bottom are SageMaker AI and EC2 instances with GPUs or AWS’s own Trainium chips, for teams that train or host models themselves. Most projects only need the middle layer, and knowing where the boundaries sit saves you from building on the wrong one.
This article walks through each layer as it stands today, with the services you are likely to use, what they cost in principle, and where older tutorials go wrong. The companion articles cover Azure and Google Cloud, and Comparing Cloud Platforms for Generative AI puts the three side by side.
How the pieces fit together
flowchart TB
subgraph Apps["Finished assistants, per user"]
Kiro["Kiro"]
Quick["Amazon Quick"]
end
subgraph BR["Amazon Bedrock, per token"]
Models["Hosted models"]
KB["Knowledge Bases"]
GR["Guardrails"]
AC["AgentCore"]
end
subgraph Own["Run it yourself"]
SM["SageMaker AI"]
EC2["EC2: Trainium, Inferentia, NVIDIA GPUs"]
end
Apps --> BR
BR --> Own
SM --> EC2
The arrows show dependency, not a path you have to follow. You can build directly on Bedrock without touching the assistants, or go straight to EC2 if you need full control over a model. Everything is tied together by AWS IAM, so the same roles, policies and CloudTrail logs you use for other services apply to model calls as well. If IAM is new to you, AWS Identity and Access Management is worth reading first.
Amazon Bedrock
Bedrock is a managed service for foundation models. AWS runs the models; you send requests and pay for the tokens in and out. There are no servers to size and no model weights to download.
Which models it hosts
The catalog is large and changes every few months. The main families at the time of writing:
| Provider | Models on Bedrock | Typical use |
|---|---|---|
| Amazon | Nova 2 Lite, Nova 2 Sonic; Nova 2 Pro and Nova 2 Omni in preview | Low-cost text and multimodal work, speech agents |
| Anthropic | Claude models | Reasoning, coding, long documents, agents |
| OpenAI | GPT-5.5 and GPT-5.4 (since June 2026), gpt-oss open-weight models | General reasoning and coding |
| Meta, Mistral AI, DeepSeek, Qwen | Open-weight models | Cheaper inference, models you could also self-host |
| Cohere, others | Embedding and rerank models | Search and retrieval |
Two things in this table catch people out. First, not every model is offered in every Region. When OpenAI’s GPT-5.5 arrived in June 2026 it was available in a few US Regions only, and that pattern is common for new models. Check Region availability before you design around one. Second, models are retired on a published schedule. The older Amazon Titan text models, which earlier versions of this article presented as AWS’s flagship, have moved to legacy and end-of-life status. For new work, Amazon’s own models are the Nova family.
Bedrock does not use your prompts or responses to train the models, and it does not share them with the model providers. Traffic goes over AWS’s network to an AWS-run endpoint; if you want it to stay off the public internet entirely, add a VPC interface endpoint (AWS PrivateLink).
Calling a model
The Converse API gives you one request format for every chat model on Bedrock, so switching from one provider to another means changing the model ID, not the code. This is a minimal call with the AWS SDK for Python:
import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.converse(
modelId="MODEL_OR_INFERENCE_PROFILE_ID",
system=[{"text": "Answer in plain English, in under 100 words."}],
messages=[{"role": "user", "content": [{"text": "What is an S3 lifecycle rule?"}]}],
inferenceConfig={"maxTokens": 300, "temperature": 0.2},
)
print(response["output"]["message"]["content"][0]["text"])
print(response["usage"]) # input and output token counts, which is what you pay for
Copy the model ID from the Bedrock console’s model catalog rather than from a tutorial, because IDs include version numbers that change. Many newer models are called through an inference profile, an ID that starts with a geography prefix such as us. and lets Bedrock route the request to another Region in that geography when one is busy. The credentials come from the environment: an IAM role on Lambda, ECS or EC2, or your SSO session on a laptop. Keep the model ID in configuration so you can change it without a deploy.
Your application uses the Bedrock Converse API with a Claude model. What do you need to change to try a Nova model instead?
Correct! Converse uses the same message format for all chat models, so the model ID and Region availability are what change.
Grounding answers in your own data
A model only knows what it was trained on. To answer questions about your policies, products or tickets, you retrieve the relevant passages and include them in the prompt. This pattern is retrieval-augmented generation (RAG). The original version of this article confused it with fine-tuning: RAG changes what the model sees at request time, while fine-tuning changes the model’s weights. For most business questions, RAG is cheaper, easier to keep current and easier to audit.
Bedrock Knowledge Bases
Knowledge Bases is Bedrock’s managed RAG service. You point it at a data source, most often an S3 bucket, and it splits the documents into chunks, turns each chunk into an embedding vector, and stores the vectors in an index. Supported vector stores include Amazon OpenSearch Serverless, Aurora PostgreSQL and Amazon S3 Vectors, which became generally available in December 2025 and is the low-cost option for large indexes that don’t need the lowest query latency.
At query time the flow looks like this:
sequenceDiagram
participant U as User
participant App as Your app
participant KB as Knowledge Base
participant V as Vector store
participant M as Model
U->>App: Question
App->>KB: RetrieveAndGenerate
KB->>V: Find similar chunks
V-->>KB: Top matching chunks
KB->>M: Question plus chunks
M-->>KB: Answer
KB-->>App: Answer with citations
App-->>U: Answer and sources
One API call does the whole round trip and returns citations pointing back to the source documents:
import boto3
agent_runtime = boto3.client("bedrock-agent-runtime", region_name="us-east-1")
result = agent_runtime.retrieve_and_generate(
input={"text": "How many days of parental leave do new employees get?"},
retrieveAndGenerateConfiguration={
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": "YOUR_KB_ID",
"modelArn": "MODEL_OR_INFERENCE_PROFILE_ARN",
},
},
)
print(result["output"]["text"])
for citation in result["citations"]:
for ref in citation["retrievedReferences"]:
print(ref["location"])
If you want more control, the separate retrieve call returns only the matching chunks, and you build the prompt yourself. That is the better choice when you need to filter results by the caller’s permissions before the model sees them, which matters as soon as different users are allowed to see different documents.
Guardrails
Bedrock Guardrails screens inputs and outputs against a policy you define: harmful content categories, topics the application must refuse, words to block, personal information to mask, and a check that answers stay grounded in the retrieved sources. A guardrail is a separate resource with its own version, and you attach it to a Converse call with the guardrailConfig parameter, so one policy can cover every model you use.
Guardrails reduce risk; they don’t remove it. A prompt-injection attack hidden in a retrieved document can still steer a model, so treat model output as untrusted input: validate it, and never let it run commands or change data without a check.
Agents with AgentCore
An agent is a model in a loop that can call tools: look up an order, query a database, open a web page, run code. The model decides which tool to call, your system runs it, and the result goes back to the model until the task is done.
Amazon Bedrock AgentCore, generally available since October 2025, is AWS’s set of services for running agents in production. It is framework-agnostic: you can write the agent with an open-source framework such as Strands Agents or LangGraph and use any model, including ones outside Bedrock. The services can be used separately:
| AgentCore service | What it handles |
|---|---|
| Runtime | Hosts the agent code in isolated sessions that scale on demand |
| Memory | Short-term conversation state and long-term memory across sessions |
| Gateway | Turns APIs and Lambda functions into tools the agent can call, including over MCP |
| Identity | Credentials for the agent to act on a user’s behalf in other systems |
| Browser and Code Interpreter | Managed, sandboxed web browsing and code execution |
| Observability | Traces of each step, tool call and model call |
Policy and Evaluations, for limiting what tools an agent may call and for scoring its behaviour, were added after launch. The older Bedrock Agents feature, where you configure an agent in the console without writing the loop yourself, still exists and suits simple cases.
The biggest risk with agents is permissions, not models. Give each agent its own IAM role with only the actions it needs, keep write and delete actions behind a human approval step, and log every tool call. An agent that can read untrusted text, such as emails or web pages, should not also hold broad write access.
Your support agent reads customer emails and can issue refunds. What is the most important control?
Correct! An email can contain instructions aimed at the model, so the permissions and approval step are what actually limit harm.
SageMaker AI: when you run the model
Amazon SageMaker AI (the machine learning service formerly just called SageMaker) is for teams that want to train, fine-tune or host models themselves. The parts that matter for generative AI:
- SageMaker JumpStart is a catalog of open-weight and proprietary models you can deploy to an endpoint in your account with a few clicks or one SDK call. You choose the instance type and pay for it by the hour, whether or not requests arrive.
- SageMaker HyperPod manages large training clusters, replacing failed nodes and resuming jobs from checkpoints, and can also serve models on the same cluster.
- Training jobs and endpoints let you bring your own container and code when nothing in the catalog fits.
The choice between Bedrock and SageMaker AI usually comes down to who runs the model:
| Question | Amazon Bedrock | SageMaker AI | EC2 directly |
|---|---|---|---|
| Who manages servers | AWS | AWS manages the platform, you choose instances | You |
| Billing | Per token, or reserved throughput | Per instance hour | Per instance hour |
| Model choice | The Bedrock catalog, plus imported custom models | Any model you can package | Anything |
| Idle cost | None on demand | Endpoints cost money while running | Instances cost money while running |
| Best for | Most applications | Custom or fine-tuned open models, training | Specialised serving stacks, research |
Start with Bedrock unless you have a concrete reason not to: a model it doesn’t host, a fine-tuning approach it doesn’t support, or traffic steady and large enough that dedicated instances work out cheaper than tokens.
Infrastructure for training and serving
If you go below the managed services, AWS offers two kinds of accelerator.
AWS’s own chips. Trainium is designed for training and, in its newer generations, for inference too. Trn2 instances and UltraServers are widely used, and Trn3 UltraServers, built on the 3 nm Trainium3 chip, became generally available in December 2025. Inferentia2, in Inf2 instances, is aimed at inference. Both are programmed through the AWS Neuron SDK, which supports PyTorch and JAX; popular open models generally run on them, but a custom model may need porting work.
NVIDIA GPUs. The P5 family (H100 and H200 GPUs) and the newer P6 family (Blackwell B200 and B300 GPUs, plus P6e UltraServers with GB200 and GB300) are for large training and inference jobs. The G-family instances carry smaller GPUs for modest inference, image generation and fine-tuning small models. Deep Dive into AWS EC2 Instances explains the instance naming and purchasing options.
Large GPU instances are scarce. For a training run with a fixed start and end, EC2 Capacity Blocks for ML let you reserve a cluster for a set period in advance, which is more predictable than hoping on-demand capacity is free.
flowchart TD
Q{"Is the model you need on Bedrock?"} -->|Yes| BR["Use Bedrock on demand"]
Q -->|No| FT{"Is it an open model or a fine-tune of one?"}
FT -->|Yes| IMP{"Can Bedrock import it?"}
IMP -->|Yes| CMI["Bedrock Custom Model Import"]
IMP -->|No| JS["SageMaker AI endpoint"]
FT -->|"No, training from scratch"| TR["HyperPod or EC2 with Trainium or GPUs"]
BR --> V{"Steady, high, predictable volume?"}
V -->|Yes| PT["Consider provisioned throughput"]
V -->|No| OD["Stay on demand"]
Assistants you buy per user
AWS also sells finished products built on these services. They are bought per seat, not per token.
- Kiro is AWS’s AI development environment and CLI, with agentic coding, chat and a spec-driven workflow that writes requirements and a design before code. It is the successor to the Amazon Q Developer IDE plugins: new Q Developer sign-ups closed in May 2026, and the plugins reach end of support on April 30, 2027. Amazon Q Developer was itself the renamed CodeWhisperer, so if a tutorial mentions CodeWhisperer it is at least two product names out of date.
- Amazon Q Developer in the AWS console still answers questions about your account and AWS services, and isn’t part of the plugin wind-down.
- Amazon Quick is the workspace for business users, launched in October 2025 as Quick Suite and absorbing QuickSight. It provides chat over company data, research, and automated workflows that connect to business applications.
What it costs
Token prices change often and differ by model and Region, so this article doesn’t print them. Use the Amazon Bedrock pricing page for current numbers, and AWS Free Tier and Billing Overview for budgets and alerts.
What you can plan around is the set of ways Bedrock lets you buy inference:
| Option | How it works | Fits |
|---|---|---|
| Standard (on demand) | Pay per input and output token | Most applications, prototypes, spiky traffic |
| Priority | A premium per token for preferential processing when demand is high | Latency-sensitive user-facing calls |
| Flex | A discount per token; requests wait behind standard traffic at busy times | Background jobs that can tolerate delay |
| Batch | Submit a file of requests, collect results from S3 later; about half the on-demand price for supported models | Bulk summarising, classification, evaluations |
| Provisioned throughput | Reserve dedicated model capacity for a term | Steady, high-volume traffic, and some custom models |
| Prompt caching | Repeated prompt prefixes are billed at a reduced rate | Long fixed instructions or documents sent on every call |
Tokens are rarely the whole bill. A Knowledge Base adds the cost of its vector store, an OpenSearch Serverless collection has a minimum charge whether or not you query it, and agents add runtime, memory and logging charges. A few habits keep spending under control:
- Use the smallest model that passes your tests. Build an evaluation set of real questions first, so you can measure rather than guess.
- Cap
maxTokensand keep system prompts short, or cache them if they must be long. - Move anything that isn’t interactive to batch or Flex.
- Set an AWS Budgets alert before launch, and rate-limit public endpoints per user.
Each night you summarise 200,000 support tickets, and results are needed by morning. Which option usually costs least?
Correct! Batch inference is priced well below on-demand for supported models and suits large jobs that can wait hours.
Getting started
A sensible first project on AWS uses only a few of these services:
- Pick one narrow use case with a clear measure of success, such as answering questions from a set of internal documents.
- Try two or three models in the Bedrock console’s playground with real questions, and note which answers are good enough.
- Put the documents in S3, create a Knowledge Base, and attach a guardrail.
- Call it from a Lambda function with an IAM role that can invoke only those resources.
- Log token usage from each response, set a budget alert, and review real questions weekly to improve the prompts and documents.
Agents, fine-tuning and dedicated capacity can come later, once you know where the simple version falls short. For the model side of the picture, The Role of Transformers in Generative AI explains how these models work, and Exploring OpenAI’s GPT Models covers the GPT family now also offered on Bedrock.
Further reading
- What is Amazon Bedrock? and supported foundation models
- Bedrock model lifecycle, for legacy and end-of-life dates
- Bedrock Knowledge Bases and Bedrock Guardrails
- Amazon Bedrock AgentCore developer guide
- SageMaker JumpStart foundation models
- Amazon Q Developer IDE plugins end of support
Key takeaways
- AWS’s generative AI services sit in three layers: per-user assistants, Amazon Bedrock for hosted models, and SageMaker AI and EC2 for running models yourself. Most applications belong on Bedrock.
- Bedrock hosts Amazon Nova, Anthropic Claude, OpenAI GPT and many open-weight models behind one Converse API, but Region availability and retirement dates differ by model.
- Knowledge Bases handle retrieval, Guardrails filter inputs and outputs, and AgentCore runs agents with any framework; IAM permissions remain the main safety control.
- Titan text models and CodeWhisperer are retired names: use Nova for Amazon’s own models, and Kiro for coding.
- Pay attention to the buying option as much as the model: batch and Flex for work that can wait, provisioned throughput only for steady volume, and budget for vector stores and agent runtime as well as tokens.
Part 25 of 30 · Introduction to Generative AI
- 01Introduction to Generative AI: What Is It?
- 02The History of Generative AI: From Concept to Reality
- 03Key Definitions and Concepts in Generative AI
- 04Pioneers of Generative AI: Meet the Inventors
- 05Understanding Neural Networks: The Backbone of AI
- 06Deep Learning Basics: Layers and Learning Rates
- 07Generative vs. Discriminative Models: What's the Difference?
- 08Overview of Generative Models: VAEs, GANs, and More
- 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
- 10Introduction to Generative Adversarial Networks (GANs)
- 11The Role of Transformers in Generative AI
- 12Applications of Generative AI in Everyday Life
- 13Ethical Considerations in Generative AI: Navigating the New Frontier
- 14Generative AI in Art and Music: The Creative Revolution
- 15Case Study: AI-Generated Artwork
- 16Generative AI in Text Generation
- 17Exploring OpenAI's GPT Models: Word Up! Bot
- 18Real-World Use Cases: Deepfakes and Their Impact
- 19Generative AI in Image and Video Synthesis
- 20Introduction to Reinforcement Learning in Generative AI
- 21The Future of Generative AI: Trends to Watch
- 22Generative AI in Healthcare: Opportunities and Challenges
- 23Generative AI in Finance: Risk Modeling and Beyond
- 24Generative AI in Marketing and Personalization
- 25Cloud Services for Generative AI: AWS Offerings
- 26Cloud Services for Generative AI: GCP Offerings
- 27Cloud Services for Generative AI: Azure Offerings
- 28Comparing Cloud Platforms for Generative AI
- 29Recap and Q&A: Introduction to Generative AI
- 30Building Your First Generative AI Model: A Practical Guide