Cloud Services for Generative AI: GCP Offerings
A practical map of Google Cloud's generative AI services: Gemini Enterprise Agent Platform, Gemini models, Model Garden, Agent Search, TPUs and cost.
Google Cloud’s generative AI services changed names in April 2026. Vertex AI, the platform most tutorials describe, is now called Gemini Enterprise Agent Platform, and several of its parts were renamed with it. The substance is familiar: Google’s Gemini models and a catalog of partner and open models behind one API, plus managed services for grounding answers in your data, running agents and screening prompts. Around the platform sit the TPUs and GPUs it runs on, BigQuery’s AI functions, and finished assistants such as Gemini Enterprise and Gemini Code Assist.
This article covers each of those layers as it stands today, explains what the renames mean for code and documentation you already have, and shows a minimal call to a model. The companion articles cover AWS and Azure, and Comparing Cloud Platforms for Generative AI puts the three side by side.
How the pieces fit together
flowchart TB
Users["Users and applications"] --> Apps["Assistants: Gemini Enterprise, Gemini Code Assist, Gemini Cloud Assist"]
Users --> AP["Gemini Enterprise Agent Platform"]
AP --> Models["Models: Gemini, Imagen, Veo, Gemma, partner models"]
AP --> Ground["Grounding: RAG Engine, Agent Search, Google Search"]
AP --> Agents["Agent Runtime and Agent Development Kit"]
AP --> Armor["Model Armor and safety settings"]
Users --> BQ["BigQuery AI functions"]
BQ --> Models
Users --> Infra["TPUs, GPU VMs and GKE"]
AP -.-> IAM["Cloud IAM, VPC Service Controls, Cloud Logging"]
Infra -.-> IAM
Most projects start in the middle: you pick a Gemini model, call it from your code with a service account’s credentials, and add grounding and screening as the application grows. You drop down to TPUs, GPU virtual machines or GKE when you need to train a model, fine-tune in ways the platform doesn’t offer, or serve weights that aren’t in the catalog.
The platform uses the rest of Google Cloud for identity and networking. Access is granted with Cloud IAM roles on a project, VPC Service Controls can stop data leaving a security perimeter, and request logs go to Cloud Logging. If your data already sits in BigQuery or Cloud Storage, that integration is the main reason to run generative AI on Google Cloud rather than call a model vendor from somewhere else.
What the renames mean
Google announced the new names at Cloud Next in April 2026. The older names are still everywhere: in API hostnames, client library names, IAM roles and older guides, including an earlier version of this article.
| Older name | Current name |
|---|---|
| Vertex AI | Gemini Enterprise Agent Platform (Agent Platform for short) |
| Vertex AI Studio | Agent Studio |
| Vertex AI Agent Engine | Agent Runtime |
| Vertex AI Search | Agent Search |
| Google Agentspace | Gemini Enterprise |
| Duet AI for Google Cloud | Gemini Code Assist and Gemini Cloud Assist |
| PaLM 2, Bard | Gemini models and the Gemini app |
The rename did not change the API you call. Endpoints still use aiplatform.googleapis.com, IAM roles are still roles/aiplatform.*, and old documentation links redirect. Working code doesn’t need changes, but you need both names when you search the docs or grant a role.
Models on the platform
Gemini
Gemini is Google DeepMind’s family of multimodal models. They accept text, images, audio, video and PDFs in the same request and return text, and the newer versions also handle tool calls, structured JSON output and long contexts. Each generation comes in tiers:
| Tier | Built for | Typical use |
|---|---|---|
| Pro | Hardest reasoning and coding tasks | Complex analysis, agents that plan over many steps |
| Flash | A balance of quality, speed and cost | The default for most applications |
| Flash-Lite | Lowest latency and cost | Classification, extraction and high-volume routing |
| Live | Low-latency, two-way voice and video | Voice assistants and real-time conversation |
The current models are versions of Gemini 3. Google releases new minor versions every few months and retires old ones on a published schedule: Gemini 1.5 is long gone, and Gemini 2.5 Pro, Flash and Flash-Lite were retired on the platform in October 2026. Anything that names a specific version, including a tutorial, a client library default or a cost estimate, goes out of date quickly. Check the model versions and lifecycle page before you pick one, and keep the model ID in configuration rather than in code.
Two Gemini features change how you design an application. Thinking lets the model reason before answering, and you can set a budget for it: more thinking improves hard answers but adds latency and output tokens. Grounding with Google Search lets the model look things up on the web and return sources, which is useful for questions about recent events; it is billed separately from tokens.
Media models and Gemma
Alongside Gemini, Google offers its own media models: Imagen and Gemini’s image models for creating and editing images, Veo for video, and Lyria for music. Generated images and video carry a SynthID watermark that identifies them as AI-generated.
Gemma is Google’s family of open-weight models, built from the same research as Gemini. You can download the weights and run them anywhere, call some versions as a managed API in Model Garden, or deploy them to your own endpoint, GKE cluster or a single GPU. Gemma suits workloads that must run on your own infrastructure or be fine-tuned freely.
Partner and open models in Model Garden
Model Garden is the catalog of everything the platform can serve. Its entries fall into four groups, which differ in contracts and operations:
| Kind | Examples | How you use it |
|---|---|---|
| Google models | Gemini, Imagen, Veo, Lyria | Managed API, billed per token or per output on your Google Cloud invoice |
| Partner models as a service | Anthropic Claude, Mistral, xAI Grok | Managed API; enabled per project and billed through Cloud Marketplace terms |
| Open models as a service | Gemma, Meta Llama, DeepSeek, OpenAI gpt-oss | Managed API for selected models, with no servers to run |
| Open models you deploy | Hugging Face and other open weights | Deployed to an endpoint on GPUs or TPUs you pay for by the hour |
Partner models are a common surprise. Enabling one needs a procurement permission and an organization policy that allows Cloud Commerce purchases, so a developer with full project access can still be blocked. Sort that out, and check which regions offer the model, before a deadline depends on it.
Calling a model
Google’s Gen AI SDK (google-genai for Python) calls Gemini both through the Gemini Developer API and through the Agent Platform. The older Vertex AI SDK’s generative AI modules are deprecated, so use the Gen AI SDK for new work. Pointing it at your project makes it use Google Cloud credentials rather than an API key:
pip install google-genai
gcloud auth application-default login # local development only
export GOOGLE_GENAI_USE_VERTEXAI=True
export GOOGLE_CLOUD_PROJECT=your-project-id
export GOOGLE_CLOUD_LOCATION=global
export GEMINI_MODEL=a-current-gemini-flash-model-id
import os
from google import genai
client = genai.Client() # reads the project, location and backend from the environment
response = client.models.generate_content(
model=os.environ["GEMINI_MODEL"], # a current model ID from the model versions page
contents="Explain VPC Service Controls in two sentences.",
)
print(response.text)
On Cloud Run, GKE or Compute Engine the same code picks up the attached service account through Application Default Credentials, so no key lives in your code. That service account needs a role that allows prediction, such as roles/aiplatform.user, and the project needs the aiplatform.googleapis.com API enabled.
The location matters too. The global endpoint routes each request to wherever capacity is free and usually gets new models first. A regional location such as europe-west4 keeps processing in that region, which some data residency rules require, but not every model is offered in every region. A 404 on a model that exists usually means it isn’t available in the location you chose; a 403 usually means a missing role or a disabled API.
Your code calls a Gemini model that works in Agent Studio, but from your application it returns "not found". The role and API are both in place. What is the most likely cause?
Correct! Model availability differs by location, and the global endpoint usually has the widest choice.
Grounding answers in your own data
A model only knows what it was trained on. To answer questions about your contracts, products or tickets, you retrieve relevant passages from your own documents and include them in the prompt. That pattern is retrieval-augmented generation (RAG), and Google Cloud offers it at three levels of control.
| Option | What you manage | Fits |
|---|---|---|
| Agent Search (formerly Vertex AI Search) | Data sources and settings; Google runs ingestion, ranking and the index | Search and question answering over websites, Cloud Storage, BigQuery and connected business apps |
| RAG Engine | Corpus, chunking, embedding model and choice of vector store | Custom RAG pipelines where you want to control each step |
| Your own pipeline | Everything: chunking, embeddings, AlloyDB, Cloud SQL or BigQuery vector search | Existing databases, unusual data, or strict control over storage |
Agent Search is the quickest route to good results, because it handles document parsing, hybrid retrieval and re-ranking for you. RAG Engine manages ingestion and retrieval but lets you choose the embedding model and vector store. Building it yourself makes sense when your data already sits in a database with vector search and you’d rather not copy it into another index.
sequenceDiagram
participant U as User
participant App as App or agent
participant MA as Model Armor
participant R as Agent Search or RAG Engine
participant G as Gemini model
U->>App: Question
App->>MA: Screen prompt
App->>R: Retrieve
R-->>App: Matching chunks with sources
App->>G: Instructions, chunks, question
G-->>App: Draft answer
App->>MA: Screen response
App-->>U: Answer with citations
Two design points matter more than the choice of tool. First, retrieval should respect who is asking: if the source documents have permissions, the results should too, or the assistant becomes a way to read files a user can’t open. Agent Search can enforce document-level access control for many connected sources; a custom pipeline has to do it itself. Second, keep retrieval close to the data. If your documents live in Google Workspace, BigQuery or Cloud Storage, Google’s services are a natural fit. If they live in another cloud, copying them across adds egress charges and a second copy to secure.
Agents
An agent is a model that can call tools in a loop: search a knowledge base, call an API, run code, then decide what to do next. Google Cloud splits the job into two parts.
The Agent Development Kit (ADK) is an open-source framework for writing agents in code, available for Python, Java, Go and TypeScript. It defines agents, tools, multi-agent workflows and evaluation, and it isn’t tied to Gemini: you can use other models through it. Agent Studio (formerly Vertex AI Studio) is the console’s prompt playground and low-code agent designer, which is the easiest place to try prompts and model settings before you write code.
Agent Runtime (formerly Agent Engine) hosts agents in production. It handles sessions, long-term memory, scaling, tracing and evaluation, and runs agents written with ADK or other open-source frameworks. Agents can call tools through MCP servers, OpenAPI specs and Google Cloud’s built-in tools, and they can talk to agents on other platforms through the open Agent2Agent (A2A) protocol. Agents you build can also be published to Gemini Enterprise, so employees reach them in the same place they use Google’s own agents.
flowchart LR
User["User in Gemini Enterprise or an app"] --> Agent["Agent on Agent Runtime"]
Agent --> Model["Gemini or a partner model"]
Agent --> Search["Agent Search or RAG Engine"]
Agent --> Tools["Your APIs and MCP servers"]
Agent --> Other["Other agents over A2A"]
Agent -.->|"Agent identity, least privilege"| Tools
Agent -.-> Obs["Tracing, logging and evaluation"]
The risk with agents is that they act. Give each agent its own identity with the narrowest roles that work, keep write and delete actions behind a human confirmation step, and evaluate the agent on realistic tasks before release rather than relying on a few manual chats.
Safety controls and data handling
Google Cloud’s safety controls work at different layers, and most applications use more than one.
| Control | What it does |
|---|---|
| Gemini safety settings | Built-in filters for harmful categories such as hate speech, harassment, sexual content and dangerous content, with configurable blocking thresholds |
| Model Armor | A separate service that screens prompts and responses for prompt injection and jailbreak attempts, sensitive data, malicious URLs and harmful content, using templates you define once |
| Sensitive Data Protection | Finds and masks personal and confidential data, and can be used by Model Armor |
| VPC Service Controls | Blocks data from leaving a perimeter of projects, even with valid credentials |
Model Armor’s advantage is that it isn’t tied to one model. The same template can screen traffic to Gemini, Claude or a model you host yourself, and it can be applied in front of agents and MCP tool calls. As on any platform, filters reduce risk without removing it. Indirect prompt injection, where an agent reads a web page or email that tells it to do something else, is the hard case, and least-privilege access is what limits the damage when a filter misses.
On data handling, Google’s documentation states that Google doesn’t use your prompts, responses or tuning data to train its models without your permission. Two features do store or log data and are worth checking against your policies: in-memory caching of inputs to reduce latency, and logging of prompts for abuse monitoring, which customers with an approved exception can turn off. Grounding with Google Search also sends your query to Google Search, which has its own terms. Read the data governance page for the models you use.
You call both Gemini and Claude from the same application and want one policy that blocks prompt injection attempts for both. Which service fits?
Correct! Model Armor templates are configured separately from the model, so one policy can screen traffic to any model.
Generative AI in BigQuery
If your data lives in BigQuery, you can call Gemini from SQL instead of moving rows into an application. Functions such as AI.GENERATE send each row to a model and return the answer as a column, with an optional output schema that turns free text into typed fields. That is a practical way to classify support tickets, extract fields from product descriptions or summarise reviews across millions of rows.
SELECT
ticket_id,
AI.GENERATE(
CONCAT('Classify this support ticket as billing, bug or other: ', body)
).result AS category
FROM support.tickets
WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY);
The model calls are billed as Agent Platform usage on top of BigQuery’s query charges, and each row is a separate request, so test on a LIMIT 100 sample before running across a large table. BigQuery also stores embeddings and runs vector search, which makes it a reasonable RAG store when the source data is already there. The BigQuery generative AI overview lists the available functions and models.
Infrastructure for training and serving
Google designs its own accelerators, Tensor Processing Units (TPUs), and also rents NVIDIA GPUs. Both are available as Compute Engine machines and as node pools in GKE.
| Option | Built on | Fits |
|---|---|---|
| Ironwood (TPU7x) | Google’s seventh-generation TPU, designed for inference | Serving large models at scale; training with JAX or PyTorch on TPU |
| Trillium (TPU v6e) | Previous TPU generation | Cost-efficient training and inference |
| A4 and A4X | NVIDIA Blackwell B200 and GB200 GPUs | Training and serving with CUDA-based stacks |
| A4X Max | NVIDIA GB300 NVL72 systems, bare metal only | The largest training and inference runs |
| G4 | NVIDIA RTX PRO 6000 Blackwell GPUs | Smaller inference workloads and graphics |
TPUs are a good option for your own models if your framework supports them well (JAX does natively; PyTorch through PyTorch/XLA and vLLM’s TPU backend). GPUs are the safer choice if your stack depends on CUDA libraries. Capacity for the newest accelerators is limited, so large jobs usually need a reservation, Dynamic Workload Scheduler, or Spot capacity for work that can be interrupted. The Compute Engine overview covers machine families in general.
Most teams never need this layer for generative AI. Calling a hosted model is cheaper and simpler until you have a measured reason to serve your own: a fine-tuned open model, data that can’t leave your infrastructure, or volume high enough that dedicated hardware costs less than tokens.
Assistants you buy per user
Google also sells finished assistants, priced per user rather than per token.
| Assistant | What it does |
|---|---|
| Gemini Enterprise | Employee-facing app for chat, search across company data and agents, including agents your team builds; formerly Google Agentspace |
| Gemini in Google Workspace | Gemini features in Gmail, Docs, Sheets, Meet and Drive |
| Gemini Code Assist | Code completion, chat and agent mode in IDEs, with Standard and Enterprise editions |
| Gemini Cloud Assist | Assistant in the Google Cloud console for designing, operating and troubleshooting resources |
If a finished assistant does the job, buying it is faster than building one. Build on the Agent Platform when the workflow is specific to your business, needs your own data and tools, or has to run inside your own application.
What it costs
Token prices change often and differ by model, version and consumption option, so this article doesn’t print them. Use the official Agent Platform generative AI pricing page and the Google Cloud pricing calculator.
What you can plan for is the shape of the bill:
| Cost item | How it is charged | How to control it |
|---|---|---|
| Gemini tokens | Per input and output token; thinking tokens count as output | Flash-Lite or Flash for simple tasks, thinking budgets, shorter prompts |
| Cached input | Repeated context is billed at a much lower rate; implicit caching applies automatically on supported models | Put stable content such as instructions and documents at the start of the prompt |
| Batch inference | About half the standard price, results asynchronously | Move anything that isn’t interactive to batch |
| Flex PayGo | Discounted per token in exchange for slower responses and more throttling | Background work that needs results within minutes rather than seconds |
| Provisioned Throughput | Reserved capacity, committed for a week up to a year | Buy only for steady load you’ve measured on pay-as-you-go first |
| Grounding and search | Per request for Google Search grounding; per query or storage for Agent Search | Ground only the questions that need fresh or private facts |
| Agent Runtime and logging | Compute time for hosted agents, plus Cloud Logging ingestion | Sample traces in production and set log retention |
New Google Cloud accounts get free credits, described in the GCP free tier article, but generative AI usage spends them quickly. Set a budget with alerts on the billing account before you put anything public, and limit requests per user, for example with Apigee or API Gateway in front of your service, so one script can’t spend a month’s budget in an afternoon.
You need to summarise five million archived product reviews by next week. Which way of paying fits best?
Correct! Batch inference costs about half the standard price and suits large jobs with no one waiting on each answer.
Getting started
A sensible first project takes an afternoon:
- Create a project with billing enabled, set a budget alert, and enable the Vertex AI API (
gcloud services enable aiplatform.googleapis.com). - Open Agent Studio in the console and try prompts against a current Flash model.
- Create a service account for your application and grant it
roles/aiplatform.user. Use your own credentials only for local development. - Call the model from code with the Gen AI SDK, as shown above, keeping the model ID and location in configuration.
- Add grounding over a small document set with Agent Search or RAG Engine, and evaluate on 20 to 50 real questions before showing it to anyone.
- Add a Model Armor template in front of the model, then deploy the service to Cloud Run.
Exploring Google Cloud Platform covers the Google Cloud basics, and Building Your First Generative AI Model is a hands-on start on the modelling side.
Further reading
- Gemini Enterprise Agent Platform documentation
- Google models and model versions
- Google Gen AI SDK
- Model Garden
- Model Armor overview
- Agent Development Kit
- Cloud TPU documentation
Key takeaways
- Vertex AI is now Gemini Enterprise Agent Platform, and Vertex AI Studio, Agent Engine and Vertex AI Search are now Agent Studio, Agent Runtime and Agent Search. API endpoints, IAM roles and many doc URLs still use the old names.
- Gemini models come in Pro, Flash, Flash-Lite and Live tiers. Versions retire quickly, so check the model versions page and keep the model ID in configuration.
- Model Garden adds Imagen, Veo, Gemma and partner models such as Claude and Mistral, all reached through Google Cloud IAM and billing.
- Use the Gen AI SDK with a service account, pick the
globallocation unless residency rules require a region, and grantroles/aiplatform.user. - Agent Search, RAG Engine and BigQuery vector search cover grounding; ADK and Agent Runtime cover agents; Model Armor screens traffic to any model.
- Budget for grounding, agent hosting and logging as well as tokens, and use batch, Flex PayGo and caching to cut the token bill.
Part 26 of 30 · Introduction to Generative AI
- 01Introduction to Generative AI: What Is It?
- 02The History of Generative AI: From Concept to Reality
- 03Key Definitions and Concepts in Generative AI
- 04Pioneers of Generative AI: Meet the Inventors
- 05Understanding Neural Networks: The Backbone of AI
- 06Deep Learning Basics: Layers and Learning Rates
- 07Generative vs. Discriminative Models: What's the Difference?
- 08Overview of Generative Models: VAEs, GANs, and More
- 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
- 10Introduction to Generative Adversarial Networks (GANs)
- 11The Role of Transformers in Generative AI
- 12Applications of Generative AI in Everyday Life
- 13Ethical Considerations in Generative AI: Navigating the New Frontier
- 14Generative AI in Art and Music: The Creative Revolution
- 15Case Study: AI-Generated Artwork
- 16Generative AI in Text Generation
- 17Exploring OpenAI's GPT Models: Word Up! Bot
- 18Real-World Use Cases: Deepfakes and Their Impact
- 19Generative AI in Image and Video Synthesis
- 20Introduction to Reinforcement Learning in Generative AI
- 21The Future of Generative AI: Trends to Watch
- 22Generative AI in Healthcare: Opportunities and Challenges
- 23Generative AI in Finance: Risk Modeling and Beyond
- 24Generative AI in Marketing and Personalization
- 25Cloud Services for Generative AI: AWS Offerings
- 26Cloud Services for Generative AI: GCP Offerings
- 27Cloud Services for Generative AI: Azure Offerings
- 28Comparing Cloud Platforms for Generative AI
- 29Recap and Q&A: Introduction to Generative AI
- 30Building Your First Generative AI Model: A Practical Guide