Cloud Services for Generative AI: Azure Offerings
A practical map of Azure's generative AI services: Microsoft Foundry, Azure OpenAI, deployment types, Foundry IQ, Agent Service, guardrails and cost.
Azure’s generative AI offering is easier to understand once you know that almost all of it now lives under one name: Microsoft Foundry. Models, retrieval, agents, safety filters and evaluation are features of a Foundry resource, and Azure OpenAI, the service most people know, is now one part of it. Around Foundry sit Azure Machine Learning and GPU virtual machines for teams that train or host their own models, and finished assistants such as GitHub Copilot and Microsoft 365 Copilot that you buy per user.
This article covers each of those layers as it stands today, explains the parts that trip people up (deployment types, data handling and the many renames), and shows a minimal call to a model. The companion articles cover AWS and Google Cloud, and Comparing Cloud Platforms for Generative AI puts the three side by side.
How the pieces fit together
flowchart TB
Users["Users and applications"] --> Apps["Assistants: GitHub Copilot, Microsoft 365 Copilot, Azure Copilot"]
Users --> Foundry["Microsoft Foundry"]
Foundry --> Models["Foundry Models: OpenAI, Microsoft, Anthropic, open models"]
Foundry --> IQ["Foundry IQ and Azure AI Search"]
Foundry --> Agents["Foundry Agent Service"]
Foundry --> Guard["Guardrails: Azure AI Content Safety"]
Foundry --> Tools["Foundry Tools: Speech, Vision, Language, Document Intelligence"]
Users --> Build["Azure Machine Learning and GPU VMs"]
Foundry -.-> Entra["Microsoft Entra ID, private endpoints, Azure Monitor"]
Build -.-> Entra
Most projects start and stay in the middle: you deploy a model in Foundry, call it from your code, and add retrieval and safety filters as the application grows. You only drop to Azure Machine Learning or raw GPU virtual machines when you need to train a model, fine-tune in ways Foundry doesn’t offer, or serve weights that aren’t in the catalog.
Everything is tied to the rest of Azure in the same way. Identities come from Microsoft Entra ID, traffic can stay on your virtual network through private endpoints, and logs go to Azure Monitor. That integration is the main reason teams already on Azure use it for AI rather than calling a model vendor directly.
A note on names
Azure’s AI products have been renamed often, and older tutorials, including an earlier version of this article, use names that no longer exist:
| Older name | Current name |
|---|---|
| Azure AI Studio, then Azure AI Foundry | Microsoft Foundry |
| Azure OpenAI Service | Azure OpenAI in Foundry Models (still widely called Azure OpenAI) |
| Cognitive Services, then Azure AI services | Foundry Tools |
| Azure Cognitive Search | Azure AI Search |
| Azure Active Directory | Microsoft Entra ID |
Microsoft also runs two versions of the Foundry portal. The current one works with Foundry projects that sit directly under a Foundry resource. The classic portal still serves older hub-based projects, which are built on Azure Machine Learning. New features arrive on Foundry resources and projects, so start new work there.
Microsoft Foundry and its models
A Foundry resource is an Azure resource (its kind is AIServices) that holds model deployments, connections and projects. A project is where a team keeps its agents, evaluations and files, and it inherits networking and access settings from the resource. You can have several projects per resource to separate teams or applications.
Which models it hosts
The Foundry catalog has thousands of models, but they fall into two groups that matter for contracts and support. Models sold directly by Azure are hosted by Microsoft, billed on your Azure invoice and covered by Microsoft’s product terms. Models from partners and the community are listed in the same catalog, and some are billed through Azure Marketplace with the provider’s own terms.
| Family | Examples | Notes |
|---|---|---|
| OpenAI | GPT-5.x models, gpt-oss, image and audio models | The widest OpenAI selection outside OpenAI itself |
| Microsoft | Phi small language models, MAI image and voice models | Phi models are also used on devices through Foundry Local |
| Anthropic | Claude models | Generally available on Azure since mid-2026 |
| Open and partner models | Meta Llama, Mistral, DeepSeek, xAI Grok | Availability differs by region and deployment type |
Two features help when you don’t want to pick one model and keep it forever. Model router is a deployment that looks at each request and sends it to a model from a pool, set to favour quality, cost or a balance of the two. And because every model sits behind the same Foundry endpoint, switching models is a change to a deployment name rather than to your code.
Model versions are retired on a published schedule. Check the Foundry model retirement page before you build on a specific version, and keep the deployment name in configuration so moving to a newer one is a one-line change.
Deployment types
You never call a catalog model directly. You create a deployment of it, give that deployment a name, and choose a deployment type. The type decides where your prompts are processed and how you pay.
| Deployment type | Where data is processed | Billing | Fits |
|---|---|---|---|
| Global Standard | Any Azure region where the model is deployed | Per token | The default for most workloads; highest quotas and earliest access to new models |
| Data Zone Standard | Only within a Microsoft data zone, such as the US or the EU | Per token | Data residency rules that allow a zone but not the whole world |
| Standard (regional) | The region of your resource | Per token | Strict single-region requirements, where the model is offered |
| Provisioned (global, data zone or regional) | As above, per scope | Reserved throughput units | Steady, high-volume traffic with latency targets |
| Batch (global or data zone) | As above, per scope | Discounted per token, results within 24 hours | Large jobs that can wait: classification, summarising archives, evaluations |
The trade-off is between reach and residency. Global deployments get the most capacity because Azure can route your request to any region with free GPUs, but your prompt may be processed outside your geography. Data at rest stays in your resource’s geography either way. If compliance rules forbid processing outside a region or zone, choose the narrower type and accept that some models arrive there later, or not at all.
Calling a model
Azure OpenAI now exposes a v1 API that works with OpenAI’s standard client libraries: you point the client at your resource’s /openai/v1/ path and no longer pass an api-version. Authenticate with Entra ID instead of an API key. Locally that uses your az login session; in Azure it uses a managed identity, so no secret lives in your code.
import os
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import OpenAI
token_provider = get_bearer_token_provider(
DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = OpenAI(
base_url="https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/",
api_key=token_provider, # called before each request to get a fresh token
)
response = client.responses.create(
model=os.environ["DEPLOYMENT_NAME"], # your deployment name, not the catalog model ID
input="Explain Azure private endpoints in two sentences.",
)
print(response.output_text)
Two details cause most first-day errors. The model parameter takes the name you gave the deployment, which may differ from the model’s catalog name. And the identity running the code needs a role on the resource that allows inference, such as Cognitive Services OpenAI User; being an Owner of the subscription is not enough on its own, because Owner covers management operations rather than data-plane calls. A 404 usually means the base URL doesn’t end in /openai/v1/; a 401 usually means a missing role or the wrong token scope.
Your call to Azure OpenAI fails because the model was not found, even though the model is in the catalog. What is the most likely cause?
Correct! You call a deployment you created, by the name you gave it, so the model parameter has to match that name.
Grounding answers in your own data
A model only knows what it was trained on. To answer questions about your policies, products or tickets, you retrieve relevant passages from your own documents and include them in the prompt. That pattern is retrieval-augmented generation (RAG), and on Azure it is built on Azure AI Search.
Azure AI Search indexes your content, stores vector embeddings next to the text, and runs hybrid queries that combine keyword and vector matching, with an optional semantic ranker to reorder the results. You can build the whole pipeline yourself: chunk documents, embed them with an embedding model deployed in Foundry, write them to an index, and query it before each model call.
Foundry IQ is the managed layer on top. You define a knowledge base that points at sources such as Blob Storage, OneLake or existing search indexes, and agents query it through one endpoint. It uses agentic retrieval: the model breaks a complex question into several searches, runs them, and merges the results with references back to the source. Foundry IQ knowledge bases became generally available in June 2026, though some connectors are still in preview, so check the source you need. It still needs an Azure AI Search resource underneath, which is where much of its cost comes from.
sequenceDiagram
participant U as User
participant App as App or agent
participant CS as Content Safety
participant KB as Foundry IQ
participant S as Azure AI Search
participant M as Model deployment
U->>App: Question
App->>CS: Screen prompt
App->>KB: Retrieve
KB->>S: One or more searches
S-->>KB: Matching chunks
KB-->>App: Merged results with references
App->>M: Instructions, chunks, question
M-->>App: Draft answer
App->>CS: Screen output
App-->>U: Answer with citations
Two design points matter more than the choice of tool. First, retrieval should respect who is asking: if the source documents have permissions, the search results should too, or the assistant becomes a way to read files a user can’t open. Second, keep retrieval close to the data. If your documents live in SharePoint and OneLake, Azure’s retrieval services are a natural fit; if they live in another cloud, copying them across adds egress charges and a second copy to secure.
Safety filters and data handling
Every model deployment in Foundry has guardrails applied, built on Azure AI Content Safety. A default policy is on unless you change it, and you can create custom policies per deployment or agent.
| Control | What it does |
|---|---|
| Harm filters | Score prompts and responses for hate, sexual content, violence and self-harm, with an adjustable threshold for each |
| Prompt Shields | Detect direct jailbreak attempts from users and indirect attacks hidden in documents or emails the model reads |
| Groundedness detection | Flags responses that aren’t supported by the source material you supplied |
| Protected material detection | Catches known copyrighted text and public code appearing in outputs |
| Blocklists | Block terms specific to your business |
These filters reduce risk; they don’t remove it. Indirect prompt injection is the hard case: an agent that reads a web page or an email can be told by that content to do something you never intended. Filters catch some of it, and the rest is handled by limiting what the agent’s identity is allowed to do.
On data handling, Microsoft’s documentation states that prompts and completions sent to models sold directly by Azure are not used to train, retrain or improve the base models, and that fine-tuning data is not used to train foundation models without your permission. Some features do store data in your resource: the stateful Responses API, agent threads and stored completions keep message history so a conversation can continue. Read the data, privacy and security page for the models you use, and pay particular attention to how global deployments process prompts and to abuse monitoring retention.
Agents with Foundry Agent Service
An agent is a model that can call tools in a loop: search a knowledge base, call an API, run code, then decide what to do next. Foundry Agent Service is Azure’s managed runtime for them. It handles conversation state, tool calls, identity and tracing, and it can host agents written with Microsoft Agent Framework or other open-source frameworks, not only agents defined in the Foundry portal. Hosted agents became generally available in 2026.
Built-in tools include Foundry IQ for retrieval, web grounding, code execution, Azure Functions, OpenAPI endpoints and MCP servers. Each agent runs with an Entra identity, so you grant it access to resources with the same role assignments you use for applications.
The feature that sets Azure apart is distribution. A Foundry agent can be published into Microsoft 365 Copilot and Teams, where it is governed by the same Entra and Microsoft 365 admin controls as other agents in the tenant. If your users spend their day in Outlook and Teams, that saves you from building and securing a separate chat front end.
flowchart LR
User["User in Teams or an app"] --> Agent["Foundry agent"]
Agent --> Model["Model deployment"]
Agent --> Know["Foundry IQ knowledge base"]
Agent --> API["Your APIs and MCP servers"]
Agent --> Code["Code execution"]
Agent -.->|"Entra identity, least privilege"| API
Agent -.-> Trace["Tracing and evaluation"]
The risk that comes with agents is that they act. Give each agent the narrowest role that works, keep write and delete actions behind a human confirmation step, and evaluate the agent on realistic tasks before release using Foundry’s evaluation and tracing tools rather than a handful of manual chats.
Your support agent reads customer emails and can update orders. What most limits the damage from an instruction hidden in an email?
Correct! Prompt Shields help, but least privilege and human confirmation limit what a misled agent can actually do.
Beyond Foundry: training, hosting and devices
Azure Machine Learning
Azure Machine Learning is the platform for building and training your own models: managed compute clusters, experiment tracking, pipelines, a model registry and managed online endpoints for serving. Use it when you train a model from your own data, run a fine-tuning recipe Foundry doesn’t offer, or host open weights with custom inference code. It is more work than a Foundry deployment, because you manage the compute, the scaling and the patching of your serving image.
GPU virtual machines
For large training runs and self-managed inference, Azure offers ND-series virtual machines with NVIDIA GPUs, up to the ND GB300 v6 series built on Blackwell Ultra GPUs in rack-scale NVLink systems. These are the same kind of machines Azure uses to serve OpenAI’s models. Capacity for the newest series is limited and usually needs a quota request or a reservation, so plan well ahead. The Azure VM article covers the basics of creating one.
Foundry Tools and Foundry Local
Foundry Tools, formerly Azure AI services, are task-specific APIs: Azure Speech for transcription and text to speech, Azure Vision, Azure Language, Translator and Document Intelligence for extracting fields from forms and invoices. They are often cheaper and more predictable than asking a general model to do the same narrow task, and they plug into agents as tools.
Foundry Local runs selected models, such as Phi, on the user’s own computer, using its CPU, GPU or NPU. It became generally available in April 2026 and suits applications that must work offline or keep data on the device.
flowchart TD
Q{"Is the model in the Foundry catalog?"} -->|Yes| D["Deploy it in Foundry"]
Q -->|No| W{"Do you train or fine-tune it yourself?"}
W -->|Yes| ML["Azure Machine Learning"]
W -->|"No, just serve it"| H{"Need full control of the stack?"}
H -->|No| ML
H -->|Yes| VM["GPU VMs or AKS"]
D --> Dev{"Must it run offline on devices?"}
Dev -->|Yes| Local["Foundry Local with a small model"]
Dev -->|No| Done["Call the Foundry deployment"]
Assistants you buy per user
Microsoft also sells finished assistants, priced per user rather than per token. For many organisations these are the first generative AI services they pay for.
| Assistant | What it does |
|---|---|
| GitHub Copilot | Code completion, chat and coding agents in editors and on GitHub |
| Microsoft 365 Copilot | Assistant in Word, Excel, Outlook and Teams, grounded in the user’s Microsoft 365 data |
| Copilot Studio | Low-code builder for agents that run in Microsoft 365 Copilot and Teams |
| Azure Copilot | Assistant in the Azure portal, with agents for troubleshooting, deployment, cost optimization and resiliency |
The line between these and Foundry is who builds the experience. If a finished assistant does the job, buying it is faster than building one. Build in Foundry when the workflow is specific to your business, needs your own data and tools, or has to run inside your own application.
What it costs
Token prices change often and differ by model, version, region and deployment type, so this article doesn’t print them. Use the official Azure OpenAI pricing page, the pricing shown for each model in the Foundry catalog, and the Azure pricing calculator.
What you can plan for is the shape of the bill:
| Cost item | How it is charged | How to control it |
|---|---|---|
| Model tokens | Per input and output token, with cached input billed at a lower rate on supported models | Smaller models for simple tasks, model router, shorter prompts, prompt caching |
| Batch jobs | Discounted per token | Move anything that can wait 24 hours to a batch deployment |
| Provisioned throughput | Reserved throughput units, hourly or on a monthly or yearly reservation | Buy only for steady load you have measured on pay-per-token first |
| Azure AI Search | Per search unit for the service tier, plus the semantic ranker | Right-size the tier; it often costs as much as the model in RAG apps |
| Agents and tools | Model tokens plus charges for tools such as code execution and web grounding | Cap tool calls per run and log them |
| Monitoring and storage | Log ingestion in Azure Monitor, storage for files and threads | Sample traces in production and set retention |
New Azure accounts get free credits, covered in the Azure free tier article, but generative AI usage burns through them quickly. Set a budget with alerts on the subscription before you deploy anything public, and rate-limit your endpoint per user, for example with Azure API Management in front of Foundry, so one script can’t spend a month’s budget in an afternoon.
You need to summarise two million archived support tickets by the end of the week. Which deployment type fits best?
Correct! Batch deployments are discounted and return results within 24 hours, which suits a large job with a deadline days away.
Getting started
A sensible first project takes an afternoon:
- Create a Foundry resource and a project in the Foundry portal, in a region that offers the models you want.
- Deploy a small current model as Global Standard, or Data Zone Standard if residency matters.
- Assign your user, and later your application’s managed identity, an inference role on the resource. Don’t use API keys for anything beyond a first test.
- Try prompts in the portal playground, then call the deployment from code as shown above.
- Add a knowledge base over a small set of documents, and run an evaluation on 20 to 50 real questions before showing it to anyone.
- Set a budget alert, and review the guardrail policy on the deployment.
If you prefer the CLI, the Foundry resource and a deployment can be created with az cognitiveservices account create --kind AIServices and az cognitiveservices account deployment create. Introduction to Microsoft Azure covers the Azure basics, and Building Your First Generative AI Model is a hands-on start on the modelling side. For the different routes to OpenAI’s models, see Exploring OpenAI’s GPT Models.
Further reading
- Microsoft Foundry documentation
- Deployment types in Foundry Models
- Azure OpenAI API version lifecycle and the v1 API
- What is Foundry IQ?
- Foundry guardrails
- Data, privacy and security for Azure Direct Models
Key takeaways
- Microsoft Foundry is the home of Azure’s generative AI: model deployments, Foundry IQ retrieval, Foundry Agent Service and guardrails all live in a Foundry resource. Azure AI Studio, Azure AI Foundry and Azure AI services are old names.
- Foundry hosts OpenAI’s models alongside Microsoft’s Phi and MAI models, Anthropic’s Claude and open models, all behind one endpoint and Entra ID authentication.
- The deployment type decides where prompts are processed and how you pay: Global Standard for reach, Data Zone or regional for residency, Provisioned for steady load, Batch for work that can wait.
- Call deployments with the standard OpenAI client and the v1 API, authenticate with Entra ID rather than keys, and keep the deployment name in configuration.
- Agents can be published straight into Microsoft 365 Copilot and Teams, which is Azure’s strongest advantage if your users already work there.
- Budget for Azure AI Search, agent tools and logging as well as tokens, and check current prices on the official pricing pages.
Part 27 of 30 · Introduction to Generative AI
- 01Introduction to Generative AI: What Is It?
- 02The History of Generative AI: From Concept to Reality
- 03Key Definitions and Concepts in Generative AI
- 04Pioneers of Generative AI: Meet the Inventors
- 05Understanding Neural Networks: The Backbone of AI
- 06Deep Learning Basics: Layers and Learning Rates
- 07Generative vs. Discriminative Models: What's the Difference?
- 08Overview of Generative Models: VAEs, GANs, and More
- 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
- 10Introduction to Generative Adversarial Networks (GANs)
- 11The Role of Transformers in Generative AI
- 12Applications of Generative AI in Everyday Life
- 13Ethical Considerations in Generative AI: Navigating the New Frontier
- 14Generative AI in Art and Music: The Creative Revolution
- 15Case Study: AI-Generated Artwork
- 16Generative AI in Text Generation
- 17Exploring OpenAI's GPT Models: Word Up! Bot
- 18Real-World Use Cases: Deepfakes and Their Impact
- 19Generative AI in Image and Video Synthesis
- 20Introduction to Reinforcement Learning in Generative AI
- 21The Future of Generative AI: Trends to Watch
- 22Generative AI in Healthcare: Opportunities and Challenges
- 23Generative AI in Finance: Risk Modeling and Beyond
- 24Generative AI in Marketing and Personalization
- 25Cloud Services for Generative AI: AWS Offerings
- 26Cloud Services for Generative AI: GCP Offerings
- 27Cloud Services for Generative AI: Azure Offerings
- 28Comparing Cloud Platforms for Generative AI
- 29Recap and Q&A: Introduction to Generative AI
- 30Building Your First Generative AI Model: A Practical Guide