Exploring OpenAI's GPT Models: Word Up! Bot

How OpenAI's GPT models are offered as an API, how to pick a tier, call them with the Responses API, and run them through Azure, Bedrock or Vertex AI.

Generative AI

OpenAI’s GPT models are among the most widely used large language models, but most of the decisions you make when building with them have little to do with the model itself. Which tier you call, which API you use, where your API key lives, which cloud you route through and how you keep token costs predictable matter more to a working application than benchmark scores. This article covers those decisions, with a small cloud Q&A assistant as the running example.

If you want the background on how these models generate text, read The Role of Transformers in Generative AI and Generative AI in Text Generation first. This article assumes you know what a token and a prompt are.

What a GPT model is, in practical terms

GPT stands for generative pre-trained transformer. The model is trained on a very large body of text to predict the next token, then tuned to follow instructions and hold a conversation. You never download the proprietary GPT models. You send a request over HTTPS with some input, and you get generated output back, billed per token in each direction.

Three properties shape almost every design choice:

  • Stateless by default. The model remembers nothing between requests. Any conversation history has to be sent again, either by you or by an API feature that stores it for you, and you pay for those tokens as input each turn.
  • A fixed context window. Instructions, history, retrieved documents and the answer all have to fit in one window. Large windows are available, but filling them is slow and expensive.
  • Non-deterministic output. The same prompt can produce different answers. Anything downstream that parses model output needs to validate it.

Model tiers rather than one model

OpenAI ships models in tiers: a flagship for hard reasoning and coding, smaller variants that are faster and much cheaper, and sometimes specialised models. GPT-5, released in August 2025, came as gpt-5, gpt-5-mini and gpt-5-nano, and later releases have kept the idea of tiers while changing the names. Model names and prices change several times a year, so treat the models page and the pricing page as the source of truth, and never hard-code a model name deep in your application.

Most current GPT models are reasoning models: they can spend extra hidden tokens thinking before they answer. The API lets you set a reasoning effort level. Lower effort is faster and cheaper; higher effort helps on multi-step problems. Those reasoning tokens are billed as output even though you never see them, which surprises people reading their first invoice.

OpenAI also publishes open-weight models, gpt-oss-120b and gpt-oss-20b, released in August 2025 under the Apache 2.0 licence. You can download and run these yourself or call them as a managed service on the major clouds. They are less capable than the current flagship, but they give you control over where the model runs.

OptionGood forTrade-off
Flagship GPT modelHard reasoning, code, long multi-step tasksHighest price per token, slower
Smaller GPT tiers (mini, nano and similar)Classification, extraction, short answers, high volumeWeaker on complex reasoning
gpt-oss open-weight modelsSelf-hosting, data residency, fine control over servingYou run and scale them, or rely on a cloud’s managed offering
Lower reasoning effortLatency-sensitive chatLess accurate on hard questions

Why does a long chat get more expensive with every turn?

  • OpenAI charges a per-conversation fee that grows over time
  • The earlier turns are sent again as input tokens on each request
  • The model's output gets longer automatically as the chat goes on

Correct! The model is stateless, so prior turns count as input tokens again on every request.

The example: a cloud Q&A assistant

The running example is a small assistant that answers questions about AWS, Google Cloud and Azure, translates a concept from one provider to another (“what is the GCP equivalent of an S3 lifecycle rule?”) and produces short CLI snippets. It needs multi-turn conversation, a consistent tone, and a cost ceiling.

The browser never talks to OpenAI directly. Putting an API key in front-end code means anyone can copy it and spend your money, so a thin backend sits in the middle. It holds the key, enforces limits and decides which model to call.

flowchart LR
    U["Browser"] --> B["Backend: Lambda, Cloud Run or Azure Functions"]
    B --> S["Secret store"]
    B --> M["Moderation check"]
    B --> R{"Route"}
    R --> O["OpenAI API"]
    R --> A["Azure OpenAI in Foundry"]
    R --> W["Bedrock or Vertex AI"]
    B --> L["Logs: token usage per user"]

The backend is small. Its jobs are to authenticate the user, rate-limit them, read the key from a secret store at startup, call the model, record token usage, and return the answer. Any serverless platform does this well: AWS Lambda, Cloud Run functions on Google Cloud, or Azure Functions. Pick the one where the rest of your stack already lives.

Calling the model with the Responses API

OpenAI has two main text APIs. Chat Completions is the older one and is still supported. The Responses API is the newer one, recommended for new projects, with built-in tools and optional server-side conversation state. The older Assistants API was shut down on 26 August 2026, so tutorials that create “assistants” and “threads” no longer work.

Install the SDK and set two environment variables. Keeping the model name in configuration means switching tiers is a deploy, not a code change.

pip install openai
export OPENAI_API_KEY="..."        # from your secret store, never committed
export OPENAI_MODEL="gpt-5-mini"   # check the models page for current names

A single request:

import os
from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY
MODEL = os.environ["OPENAI_MODEL"]

INSTRUCTIONS = (
    "You answer questions about AWS, Google Cloud and Azure. "
    "Be concise. When you compare providers, name the equivalent "
    "service on each. If you are not sure a limit or price is "
    "current, say so and point to the official docs."
)

response = client.responses.create(
    model=MODEL,
    instructions=INSTRUCTIONS,
    input="What is the Azure equivalent of an S3 bucket?",
    max_output_tokens=800,
)

print(response.output_text)
print(response.usage.input_tokens, response.usage.output_tokens)

instructions plays the role of the system prompt. max_output_tokens is your per-request cost cap; with reasoning models it also covers the hidden reasoning tokens, so set it with headroom or the answer can be cut off. Log response.usage on every call. It is the only reliable way to know what each user and feature costs you.

Keeping a conversation going

You have two ways to handle history. You can keep the list of messages yourself and send it as input each time, which works with any provider and keeps you in control of storage. Or you can let OpenAI store responses and chain them with previous_response_id:

first = client.responses.create(
    model=MODEL,
    instructions=INSTRUCTIONS,
    input="How do I list my S3 buckets from the CLI?",
)

follow_up = client.responses.create(
    model=MODEL,
    instructions=INSTRUCTIONS,  # not carried over, so send it again
    previous_response_id=first.id,
    input="And the gcloud equivalent?",
)
print(follow_up.output_text)

Two details catch people out. Instructions from the earlier response are not carried forward, so pass them on every call. And chaining does not make history free: earlier turns still count as input tokens. For longer-lived conversations OpenAI also offers a Conversations API that stores items under one conversation ID.

sequenceDiagram
    participant User
    participant Backend
    participant API as Responses API
    User->>Backend: Question 1
    Backend->>API: instructions + input
    API-->>Backend: response id r1, answer
    Backend-->>User: Answer 1
    User->>Backend: Question 2
    Backend->>API: instructions + input, previous_response_id r1
    Note over API: Turn 1 counts as input tokens
    API-->>Backend: response id r2, answer
    Backend-->>User: Answer 2

If you route between providers, or need to delete history on request for privacy reasons, managing the message list yourself is simpler. Trim it as it grows: keep the instructions, the last few turns, and a short summary of anything older.

Streaming

For a chat interface, stream the answer so the user sees text immediately instead of waiting for the full response:

stream = client.responses.create(
    model=MODEL,
    instructions=INSTRUCTIONS,
    input="Explain VPC peering in three sentences.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

Your backend relays those chunks to the browser, usually as server-sent events. Streaming does not change the cost, only the perceived latency.

You chain two requests with previous_response_id but leave out instructions on the second. What happens?

  • The second request reuses the first request's instructions
  • The API rejects the request
  • The second answer is generated without your instructions

Correct! Instructions are not carried over between chained responses, so send them on every call.

Running GPT models through a cloud provider

You don’t have to call OpenAI directly. All three major clouds now offer OpenAI models, which matters when your organisation wants one bill, private networking, or identity through the cloud’s own IAM.

RouteModelsHow you authenticateNotes
OpenAI APIFull current lineupOpenAI API key, project-scopedNew models usually appear here first
Azure OpenAI in Microsoft FoundryMost GPT models, deployed per resourceAPI key or Microsoft Entra IDYou call a deployment name, and choose regional or global deployment types
Amazon Bedrockgpt-oss models; GPT-5.5 and GPT-5.4 since June 2026 in selected regionsAWS IAM or Bedrock API keysUsage is billed through AWS; check regional availability
Vertex AI on Google Cloudgpt-oss-120b and gpt-oss-20b as managed APIs; proprietary GPT models announced in limited previewGoogle Cloud IAMCheck Model Garden for what your project can use

Azure OpenAI

Azure is the longest-standing route and the closest to the OpenAI API. The current v1 API accepts the standard OpenAI client: point base_url at your resource’s /openai/v1/ path and pass your deployment name as the model. No api-version parameter is needed.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://my-resource.openai.azure.com/openai/v1/",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
)

response = client.responses.create(
    model="qa-assistant",  # your deployment name, not the model name
    input="What is the AWS equivalent of Azure Blob Storage?",
)
print(response.output_text)

In production, prefer Microsoft Entra ID over keys. Microsoft’s documentation shows passing a token provider from azure-identity as the client’s api_key, so your function’s managed identity authenticates and there is no secret to rotate. For the wider Azure picture, see Cloud Services for Generative AI: Azure Offerings.

Amazon Bedrock and Vertex AI

On AWS, the open-weight gpt-oss models have been on Bedrock since 2025, and OpenAI’s proprietary GPT-5.5 and GPT-5.4 became generally available on Bedrock in June 2026, called through the Responses API. Availability differs by region, so check the Bedrock console for yours. Bedrock exposes OpenAI-compatible endpoints, so code like the examples above usually needs only a different base_url and credentials. AWS states that prompts and responses on Bedrock are not shared with model providers.

On Google Cloud, Vertex AI offers gpt-oss-120b and gpt-oss-20b as serverless managed APIs in Model Garden, or you can deploy the weights to your own endpoint. In October 2026 Google announced that OpenAI’s proprietary models are coming to Model Garden as well, starting as a limited preview for selected customers. Until that is open to you, the native frontier model on Google Cloud is Gemini. Comparing Cloud Platforms for Generative AI covers how the three platforms differ more broadly.

flowchart TD
    Q{"Need OpenAI's newest model on day one?"} -->|Yes| OA["OpenAI API"]
    Q -->|No| C{"Where does the rest of your stack run?"}
    C -->|Azure| AZ["Azure OpenAI in Foundry"]
    C -->|AWS| BR["Bedrock: GPT-5.x or gpt-oss"]
    C -->|Google Cloud| VX{"Is an open-weight model enough?"}
    VX -->|Yes| GO["Vertex AI: gpt-oss"]
    VX -->|No| GM["Gemini, the OpenAI API, or the GPT preview"]

Keeping costs predictable

For an assistant like this one, the model calls dominate the bill. The serverless backend and a small database usually cost far less than the tokens. A monthly estimate is simple arithmetic:

monthly cost = conversations x turns per conversation
               x (input tokens x input price + output tokens x output price)
               / 1,000,000

Prices are quoted per million tokens. Take the current numbers from the pricing page for the tier you chose, and remember that input tokens per turn grow with history and that reasoning tokens count as output. Run the estimate for your expected traffic and again for ten times that, because a public chat endpoint can be scripted against.

The levers that actually move the number:

  • Route by difficulty. Send short, factual questions to a small tier and escalate to the flagship only when needed. A cheap model can do the classification.
  • Cap output. Set max_output_tokens and ask for concise answers in the instructions.
  • Lower reasoning effort for simple turns.
  • Use prompt caching. OpenAI automatically caches repeated prompt prefixes of 1,024 tokens or more, and cached input is billed at a discount. Put stable content, such as instructions and reference text, at the start of the prompt and the changing user input at the end.
  • Use the Batch API for work that doesn’t need an immediate answer, such as summarising documentation overnight. It is cheaper than synchronous calls in exchange for results within a time window.
  • Trim history rather than sending whole transcripts.

Set a budget alert and usage limits on the OpenAI project, or the equivalent billing alerts on your cloud account. On a public endpoint, also rate-limit per user in your backend, because a provider-level limit stops everyone at once.

To benefit from prompt caching, where should your long, unchanging instructions go?

  • At the start of the prompt, before the user's input
  • At the end of the prompt, after the user's input
  • It doesn't matter, caching ignores order

Correct! Caching matches on the prompt prefix, so stable content must come first.

Security and data handling

The model is the easy part to secure. Most incidents with LLM applications come from leaked keys, unbounded usage and trusting model output.

  • Keep keys out of code and the browser. Store them in AWS Secrets Manager, Google Cloud Secret Manager or Azure Key Vault, and grant read access only to the function’s identity. Use separate OpenAI projects, and therefore separate keys, for development and production. On Azure and Bedrock, use the cloud’s identity system instead of long-lived keys where you can. AWS Identity and Access Management (IAM) covers the least-privilege basics.
  • Check inputs and outputs. OpenAI’s moderation endpoint is free to use and flags harmful content. Run it on user input before the main call if your assistant is public.
  • Treat output as untrusted. If the assistant suggests a CLI command, show it to the user; never execute it automatically. If you parse model output as JSON, validate it against a schema.
  • Expect prompt injection. Text a user pastes, or a document you retrieve, can contain instructions that try to override yours. Don’t give the model tools or data that the current user isn’t allowed to use anyway.
  • Know the data terms. OpenAI does not train on API data by default, but it may keep it for a limited period for abuse monitoring, and stored responses persist until deleted. Azure, Bedrock and Vertex AI each publish their own data-handling terms. Read the ones for your route before you send customer data.

Grounding answers in your own documents

A general model doesn’t know your internal runbooks, and its knowledge of cloud services stops at its training cutoff. The usual fix is retrieval-augmented generation: split your documents into chunks, store embeddings of them in a vector index, find the chunks most similar to the user’s question, and include them in the prompt with an instruction to answer from them.

The Responses API has a built-in file search tool backed by OpenAI-hosted vector stores, which is the quickest way to try this. Each cloud also has its own option: Bedrock Knowledge Bases, Vertex AI RAG Engine and Azure AI Search. Choose the one that sits next to your data, because moving documents across providers adds cost and governance work.

Retrieval reduces made-up answers but doesn’t remove them. Ask the model to cite the chunk it used, and show those sources to the user.

Further reading

Key takeaways

  • GPT models are stateless APIs billed per input and output token; conversation history is re-sent and paid for on every turn.
  • Pick a tier per task, keep the model name in configuration, and check the models and pricing pages rather than trusting any article’s numbers, including this one.
  • Use the Responses API for new work; the Assistants API was shut down in August 2026.
  • Put a backend between users and the model, keep keys in a secret store or use cloud identity, and log token usage per request.
  • Azure, Bedrock and Vertex AI all serve OpenAI models now, with different model ranges; choose the route that matches where your stack and data already live.
  • Control cost with routing, output caps, lower reasoning effort, prompt caching and per-user rate limits.

Part 17 of 30 · Introduction to Generative AI

  1. 01Introduction to Generative AI: What Is It?
  2. 02The History of Generative AI: From Concept to Reality
  3. 03Key Definitions and Concepts in Generative AI
  4. 04Pioneers of Generative AI: Meet the Inventors
  5. 05Understanding Neural Networks: The Backbone of AI
  6. 06Deep Learning Basics: Layers and Learning Rates
  7. 07Generative vs. Discriminative Models: What's the Difference?
  8. 08Overview of Generative Models: VAEs, GANs, and More
  9. 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
  10. 10Introduction to Generative Adversarial Networks (GANs)
  11. 11The Role of Transformers in Generative AI
  12. 12Applications of Generative AI in Everyday Life
  13. 13Ethical Considerations in Generative AI: Navigating the New Frontier
  14. 14Generative AI in Art and Music: The Creative Revolution
  15. 15Case Study: AI-Generated Artwork
  16. 16Generative AI in Text Generation
  17. 17Exploring OpenAI's GPT Models: Word Up! Bot
  18. 18Real-World Use Cases: Deepfakes and Their Impact
  19. 19Generative AI in Image and Video Synthesis
  20. 20Introduction to Reinforcement Learning in Generative AI
  21. 21The Future of Generative AI: Trends to Watch
  22. 22Generative AI in Healthcare: Opportunities and Challenges
  23. 23Generative AI in Finance: Risk Modeling and Beyond
  24. 24Generative AI in Marketing and Personalization
  25. 25Cloud Services for Generative AI: AWS Offerings
  26. 26Cloud Services for Generative AI: GCP Offerings
  27. 27Cloud Services for Generative AI: Azure Offerings
  28. 28Comparing Cloud Platforms for Generative AI
  29. 29Recap and Q&A: Introduction to Generative AI
  30. 30Building Your First Generative AI Model: A Practical Guide
Ready to check your understanding?Take the Word Up! Bot quiz
esc

Type to search all articles · ↑↓ to move · ↵ to open