Cloud Services for Generative AI: GCP Offerings

A practical map of Google Cloud's generative AI services: Gemini Enterprise Agent Platform, Gemini models, Model Garden, Agent Search, TPUs and cost.

Generative AI

Google Cloud’s generative AI services changed names in April 2026. Vertex AI, the platform most tutorials describe, is now called Gemini Enterprise Agent Platform, and several of its parts were renamed with it. The substance is familiar: Google’s Gemini models and a catalog of partner and open models behind one API, plus managed services for grounding answers in your data, running agents and screening prompts. Around the platform sit the TPUs and GPUs it runs on, BigQuery’s AI functions, and finished assistants such as Gemini Enterprise and Gemini Code Assist.

This article covers each of those layers as it stands today, explains what the renames mean for code and documentation you already have, and shows a minimal call to a model. The companion articles cover AWS and Azure, and Comparing Cloud Platforms for Generative AI puts the three side by side.

How the pieces fit together

flowchart TB
    Users["Users and applications"] --> Apps["Assistants: Gemini Enterprise, Gemini Code Assist, Gemini Cloud Assist"]
    Users --> AP["Gemini Enterprise Agent Platform"]
    AP --> Models["Models: Gemini, Imagen, Veo, Gemma, partner models"]
    AP --> Ground["Grounding: RAG Engine, Agent Search, Google Search"]
    AP --> Agents["Agent Runtime and Agent Development Kit"]
    AP --> Armor["Model Armor and safety settings"]
    Users --> BQ["BigQuery AI functions"]
    BQ --> Models
    Users --> Infra["TPUs, GPU VMs and GKE"]
    AP -.-> IAM["Cloud IAM, VPC Service Controls, Cloud Logging"]
    Infra -.-> IAM

Most projects start in the middle: you pick a Gemini model, call it from your code with a service account’s credentials, and add grounding and screening as the application grows. You drop down to TPUs, GPU virtual machines or GKE when you need to train a model, fine-tune in ways the platform doesn’t offer, or serve weights that aren’t in the catalog.

The platform uses the rest of Google Cloud for identity and networking. Access is granted with Cloud IAM roles on a project, VPC Service Controls can stop data leaving a security perimeter, and request logs go to Cloud Logging. If your data already sits in BigQuery or Cloud Storage, that integration is the main reason to run generative AI on Google Cloud rather than call a model vendor from somewhere else.

What the renames mean

Google announced the new names at Cloud Next in April 2026. The older names are still everywhere: in API hostnames, client library names, IAM roles and older guides, including an earlier version of this article.

Older nameCurrent name
Vertex AIGemini Enterprise Agent Platform (Agent Platform for short)
Vertex AI StudioAgent Studio
Vertex AI Agent EngineAgent Runtime
Vertex AI SearchAgent Search
Google AgentspaceGemini Enterprise
Duet AI for Google CloudGemini Code Assist and Gemini Cloud Assist
PaLM 2, BardGemini models and the Gemini app

The rename did not change the API you call. Endpoints still use aiplatform.googleapis.com, IAM roles are still roles/aiplatform.*, and old documentation links redirect. Working code doesn’t need changes, but you need both names when you search the docs or grant a role.

Models on the platform

Gemini

Gemini is Google DeepMind’s family of multimodal models. They accept text, images, audio, video and PDFs in the same request and return text, and the newer versions also handle tool calls, structured JSON output and long contexts. Each generation comes in tiers:

TierBuilt forTypical use
ProHardest reasoning and coding tasksComplex analysis, agents that plan over many steps
FlashA balance of quality, speed and costThe default for most applications
Flash-LiteLowest latency and costClassification, extraction and high-volume routing
LiveLow-latency, two-way voice and videoVoice assistants and real-time conversation

The current models are versions of Gemini 3. Google releases new minor versions every few months and retires old ones on a published schedule: Gemini 1.5 is long gone, and Gemini 2.5 Pro, Flash and Flash-Lite were retired on the platform in October 2026. Anything that names a specific version, including a tutorial, a client library default or a cost estimate, goes out of date quickly. Check the model versions and lifecycle page before you pick one, and keep the model ID in configuration rather than in code.

Two Gemini features change how you design an application. Thinking lets the model reason before answering, and you can set a budget for it: more thinking improves hard answers but adds latency and output tokens. Grounding with Google Search lets the model look things up on the web and return sources, which is useful for questions about recent events; it is billed separately from tokens.

Media models and Gemma

Alongside Gemini, Google offers its own media models: Imagen and Gemini’s image models for creating and editing images, Veo for video, and Lyria for music. Generated images and video carry a SynthID watermark that identifies them as AI-generated.

Gemma is Google’s family of open-weight models, built from the same research as Gemini. You can download the weights and run them anywhere, call some versions as a managed API in Model Garden, or deploy them to your own endpoint, GKE cluster or a single GPU. Gemma suits workloads that must run on your own infrastructure or be fine-tuned freely.

Partner and open models in Model Garden

Model Garden is the catalog of everything the platform can serve. Its entries fall into four groups, which differ in contracts and operations:

KindExamplesHow you use it
Google modelsGemini, Imagen, Veo, LyriaManaged API, billed per token or per output on your Google Cloud invoice
Partner models as a serviceAnthropic Claude, Mistral, xAI GrokManaged API; enabled per project and billed through Cloud Marketplace terms
Open models as a serviceGemma, Meta Llama, DeepSeek, OpenAI gpt-ossManaged API for selected models, with no servers to run
Open models you deployHugging Face and other open weightsDeployed to an endpoint on GPUs or TPUs you pay for by the hour

Partner models are a common surprise. Enabling one needs a procurement permission and an organization policy that allows Cloud Commerce purchases, so a developer with full project access can still be blocked. Sort that out, and check which regions offer the model, before a deadline depends on it.

Calling a model

Google’s Gen AI SDK (google-genai for Python) calls Gemini both through the Gemini Developer API and through the Agent Platform. The older Vertex AI SDK’s generative AI modules are deprecated, so use the Gen AI SDK for new work. Pointing it at your project makes it use Google Cloud credentials rather than an API key:

pip install google-genai
gcloud auth application-default login   # local development only
export GOOGLE_GENAI_USE_VERTEXAI=True
export GOOGLE_CLOUD_PROJECT=your-project-id
export GOOGLE_CLOUD_LOCATION=global
export GEMINI_MODEL=a-current-gemini-flash-model-id
import os

from google import genai

client = genai.Client()  # reads the project, location and backend from the environment

response = client.models.generate_content(
    model=os.environ["GEMINI_MODEL"],  # a current model ID from the model versions page
    contents="Explain VPC Service Controls in two sentences.",
)
print(response.text)

On Cloud Run, GKE or Compute Engine the same code picks up the attached service account through Application Default Credentials, so no key lives in your code. That service account needs a role that allows prediction, such as roles/aiplatform.user, and the project needs the aiplatform.googleapis.com API enabled.

The location matters too. The global endpoint routes each request to wherever capacity is free and usually gets new models first. A regional location such as europe-west4 keeps processing in that region, which some data residency rules require, but not every model is offered in every region. A 404 on a model that exists usually means it isn’t available in the location you chose; a 403 usually means a missing role or a disabled API.

Your code calls a Gemini model that works in Agent Studio, but from your application it returns "not found". The role and API are both in place. What is the most likely cause?

  • The model isn't offered in the location your client is set to
  • The Gen AI SDK only works with API keys
  • Gemini models must be deployed to an endpoint before you can call them

Correct! Model availability differs by location, and the global endpoint usually has the widest choice.

Grounding answers in your own data

A model only knows what it was trained on. To answer questions about your contracts, products or tickets, you retrieve relevant passages from your own documents and include them in the prompt. That pattern is retrieval-augmented generation (RAG), and Google Cloud offers it at three levels of control.

OptionWhat you manageFits
Agent Search (formerly Vertex AI Search)Data sources and settings; Google runs ingestion, ranking and the indexSearch and question answering over websites, Cloud Storage, BigQuery and connected business apps
RAG EngineCorpus, chunking, embedding model and choice of vector storeCustom RAG pipelines where you want to control each step
Your own pipelineEverything: chunking, embeddings, AlloyDB, Cloud SQL or BigQuery vector searchExisting databases, unusual data, or strict control over storage

Agent Search is the quickest route to good results, because it handles document parsing, hybrid retrieval and re-ranking for you. RAG Engine manages ingestion and retrieval but lets you choose the embedding model and vector store. Building it yourself makes sense when your data already sits in a database with vector search and you’d rather not copy it into another index.

sequenceDiagram
    participant U as User
    participant App as App or agent
    participant MA as Model Armor
    participant R as Agent Search or RAG Engine
    participant G as Gemini model
    U->>App: Question
    App->>MA: Screen prompt
    App->>R: Retrieve
    R-->>App: Matching chunks with sources
    App->>G: Instructions, chunks, question
    G-->>App: Draft answer
    App->>MA: Screen response
    App-->>U: Answer with citations

Two design points matter more than the choice of tool. First, retrieval should respect who is asking: if the source documents have permissions, the results should too, or the assistant becomes a way to read files a user can’t open. Agent Search can enforce document-level access control for many connected sources; a custom pipeline has to do it itself. Second, keep retrieval close to the data. If your documents live in Google Workspace, BigQuery or Cloud Storage, Google’s services are a natural fit. If they live in another cloud, copying them across adds egress charges and a second copy to secure.

Agents

An agent is a model that can call tools in a loop: search a knowledge base, call an API, run code, then decide what to do next. Google Cloud splits the job into two parts.

The Agent Development Kit (ADK) is an open-source framework for writing agents in code, available for Python, Java, Go and TypeScript. It defines agents, tools, multi-agent workflows and evaluation, and it isn’t tied to Gemini: you can use other models through it. Agent Studio (formerly Vertex AI Studio) is the console’s prompt playground and low-code agent designer, which is the easiest place to try prompts and model settings before you write code.

Agent Runtime (formerly Agent Engine) hosts agents in production. It handles sessions, long-term memory, scaling, tracing and evaluation, and runs agents written with ADK or other open-source frameworks. Agents can call tools through MCP servers, OpenAPI specs and Google Cloud’s built-in tools, and they can talk to agents on other platforms through the open Agent2Agent (A2A) protocol. Agents you build can also be published to Gemini Enterprise, so employees reach them in the same place they use Google’s own agents.

flowchart LR
    User["User in Gemini Enterprise or an app"] --> Agent["Agent on Agent Runtime"]
    Agent --> Model["Gemini or a partner model"]
    Agent --> Search["Agent Search or RAG Engine"]
    Agent --> Tools["Your APIs and MCP servers"]
    Agent --> Other["Other agents over A2A"]
    Agent -.->|"Agent identity, least privilege"| Tools
    Agent -.-> Obs["Tracing, logging and evaluation"]

The risk with agents is that they act. Give each agent its own identity with the narrowest roles that work, keep write and delete actions behind a human confirmation step, and evaluate the agent on realistic tasks before release rather than relying on a few manual chats.

Safety controls and data handling

Google Cloud’s safety controls work at different layers, and most applications use more than one.

ControlWhat it does
Gemini safety settingsBuilt-in filters for harmful categories such as hate speech, harassment, sexual content and dangerous content, with configurable blocking thresholds
Model ArmorA separate service that screens prompts and responses for prompt injection and jailbreak attempts, sensitive data, malicious URLs and harmful content, using templates you define once
Sensitive Data ProtectionFinds and masks personal and confidential data, and can be used by Model Armor
VPC Service ControlsBlocks data from leaving a perimeter of projects, even with valid credentials

Model Armor’s advantage is that it isn’t tied to one model. The same template can screen traffic to Gemini, Claude or a model you host yourself, and it can be applied in front of agents and MCP tool calls. As on any platform, filters reduce risk without removing it. Indirect prompt injection, where an agent reads a web page or email that tells it to do something else, is the hard case, and least-privilege access is what limits the damage when a filter misses.

On data handling, Google’s documentation states that Google doesn’t use your prompts, responses or tuning data to train its models without your permission. Two features do store or log data and are worth checking against your policies: in-memory caching of inputs to reduce latency, and logging of prompts for abuse monitoring, which customers with an approved exception can turn off. Grounding with Google Search also sends your query to Google Search, which has its own terms. Read the data governance page for the models you use.

You call both Gemini and Claude from the same application and want one policy that blocks prompt injection attempts for both. Which service fits?

  • Gemini safety settings
  • VPC Service Controls
  • Model Armor

Correct! Model Armor templates are configured separately from the model, so one policy can screen traffic to any model.

Generative AI in BigQuery

If your data lives in BigQuery, you can call Gemini from SQL instead of moving rows into an application. Functions such as AI.GENERATE send each row to a model and return the answer as a column, with an optional output schema that turns free text into typed fields. That is a practical way to classify support tickets, extract fields from product descriptions or summarise reviews across millions of rows.

SELECT
  ticket_id,
  AI.GENERATE(
    CONCAT('Classify this support ticket as billing, bug or other: ', body)
  ).result AS category
FROM support.tickets
WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY);

The model calls are billed as Agent Platform usage on top of BigQuery’s query charges, and each row is a separate request, so test on a LIMIT 100 sample before running across a large table. BigQuery also stores embeddings and runs vector search, which makes it a reasonable RAG store when the source data is already there. The BigQuery generative AI overview lists the available functions and models.

Infrastructure for training and serving

Google designs its own accelerators, Tensor Processing Units (TPUs), and also rents NVIDIA GPUs. Both are available as Compute Engine machines and as node pools in GKE.

OptionBuilt onFits
Ironwood (TPU7x)Google’s seventh-generation TPU, designed for inferenceServing large models at scale; training with JAX or PyTorch on TPU
Trillium (TPU v6e)Previous TPU generationCost-efficient training and inference
A4 and A4XNVIDIA Blackwell B200 and GB200 GPUsTraining and serving with CUDA-based stacks
A4X MaxNVIDIA GB300 NVL72 systems, bare metal onlyThe largest training and inference runs
G4NVIDIA RTX PRO 6000 Blackwell GPUsSmaller inference workloads and graphics

TPUs are a good option for your own models if your framework supports them well (JAX does natively; PyTorch through PyTorch/XLA and vLLM’s TPU backend). GPUs are the safer choice if your stack depends on CUDA libraries. Capacity for the newest accelerators is limited, so large jobs usually need a reservation, Dynamic Workload Scheduler, or Spot capacity for work that can be interrupted. The Compute Engine overview covers machine families in general.

Most teams never need this layer for generative AI. Calling a hosted model is cheaper and simpler until you have a measured reason to serve your own: a fine-tuned open model, data that can’t leave your infrastructure, or volume high enough that dedicated hardware costs less than tokens.

Assistants you buy per user

Google also sells finished assistants, priced per user rather than per token.

AssistantWhat it does
Gemini EnterpriseEmployee-facing app for chat, search across company data and agents, including agents your team builds; formerly Google Agentspace
Gemini in Google WorkspaceGemini features in Gmail, Docs, Sheets, Meet and Drive
Gemini Code AssistCode completion, chat and agent mode in IDEs, with Standard and Enterprise editions
Gemini Cloud AssistAssistant in the Google Cloud console for designing, operating and troubleshooting resources

If a finished assistant does the job, buying it is faster than building one. Build on the Agent Platform when the workflow is specific to your business, needs your own data and tools, or has to run inside your own application.

What it costs

Token prices change often and differ by model, version and consumption option, so this article doesn’t print them. Use the official Agent Platform generative AI pricing page and the Google Cloud pricing calculator.

What you can plan for is the shape of the bill:

Cost itemHow it is chargedHow to control it
Gemini tokensPer input and output token; thinking tokens count as outputFlash-Lite or Flash for simple tasks, thinking budgets, shorter prompts
Cached inputRepeated context is billed at a much lower rate; implicit caching applies automatically on supported modelsPut stable content such as instructions and documents at the start of the prompt
Batch inferenceAbout half the standard price, results asynchronouslyMove anything that isn’t interactive to batch
Flex PayGoDiscounted per token in exchange for slower responses and more throttlingBackground work that needs results within minutes rather than seconds
Provisioned ThroughputReserved capacity, committed for a week up to a yearBuy only for steady load you’ve measured on pay-as-you-go first
Grounding and searchPer request for Google Search grounding; per query or storage for Agent SearchGround only the questions that need fresh or private facts
Agent Runtime and loggingCompute time for hosted agents, plus Cloud Logging ingestionSample traces in production and set log retention

New Google Cloud accounts get free credits, described in the GCP free tier article, but generative AI usage spends them quickly. Set a budget with alerts on the billing account before you put anything public, and limit requests per user, for example with Apigee or API Gateway in front of your service, so one script can’t spend a month’s budget in an afternoon.

You need to summarise five million archived product reviews by next week. Which way of paying fits best?

  • A one-year Provisioned Throughput commitment
  • Batch inference
  • Standard pay-as-you-go calls in a loop

Correct! Batch inference costs about half the standard price and suits large jobs with no one waiting on each answer.

Getting started

A sensible first project takes an afternoon:

  1. Create a project with billing enabled, set a budget alert, and enable the Vertex AI API (gcloud services enable aiplatform.googleapis.com).
  2. Open Agent Studio in the console and try prompts against a current Flash model.
  3. Create a service account for your application and grant it roles/aiplatform.user. Use your own credentials only for local development.
  4. Call the model from code with the Gen AI SDK, as shown above, keeping the model ID and location in configuration.
  5. Add grounding over a small document set with Agent Search or RAG Engine, and evaluate on 20 to 50 real questions before showing it to anyone.
  6. Add a Model Armor template in front of the model, then deploy the service to Cloud Run.

Exploring Google Cloud Platform covers the Google Cloud basics, and Building Your First Generative AI Model is a hands-on start on the modelling side.

Further reading

Key takeaways

  • Vertex AI is now Gemini Enterprise Agent Platform, and Vertex AI Studio, Agent Engine and Vertex AI Search are now Agent Studio, Agent Runtime and Agent Search. API endpoints, IAM roles and many doc URLs still use the old names.
  • Gemini models come in Pro, Flash, Flash-Lite and Live tiers. Versions retire quickly, so check the model versions page and keep the model ID in configuration.
  • Model Garden adds Imagen, Veo, Gemma and partner models such as Claude and Mistral, all reached through Google Cloud IAM and billing.
  • Use the Gen AI SDK with a service account, pick the global location unless residency rules require a region, and grant roles/aiplatform.user.
  • Agent Search, RAG Engine and BigQuery vector search cover grounding; ADK and Agent Runtime cover agents; Model Armor screens traffic to any model.
  • Budget for grounding, agent hosting and logging as well as tokens, and use batch, Flex PayGo and caching to cut the token bill.

Part 26 of 30 · Introduction to Generative AI

  1. 01Introduction to Generative AI: What Is It?
  2. 02The History of Generative AI: From Concept to Reality
  3. 03Key Definitions and Concepts in Generative AI
  4. 04Pioneers of Generative AI: Meet the Inventors
  5. 05Understanding Neural Networks: The Backbone of AI
  6. 06Deep Learning Basics: Layers and Learning Rates
  7. 07Generative vs. Discriminative Models: What's the Difference?
  8. 08Overview of Generative Models: VAEs, GANs, and More
  9. 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
  10. 10Introduction to Generative Adversarial Networks (GANs)
  11. 11The Role of Transformers in Generative AI
  12. 12Applications of Generative AI in Everyday Life
  13. 13Ethical Considerations in Generative AI: Navigating the New Frontier
  14. 14Generative AI in Art and Music: The Creative Revolution
  15. 15Case Study: AI-Generated Artwork
  16. 16Generative AI in Text Generation
  17. 17Exploring OpenAI's GPT Models: Word Up! Bot
  18. 18Real-World Use Cases: Deepfakes and Their Impact
  19. 19Generative AI in Image and Video Synthesis
  20. 20Introduction to Reinforcement Learning in Generative AI
  21. 21The Future of Generative AI: Trends to Watch
  22. 22Generative AI in Healthcare: Opportunities and Challenges
  23. 23Generative AI in Finance: Risk Modeling and Beyond
  24. 24Generative AI in Marketing and Personalization
  25. 25Cloud Services for Generative AI: AWS Offerings
  26. 26Cloud Services for Generative AI: GCP Offerings
  27. 27Cloud Services for Generative AI: Azure Offerings
  28. 28Comparing Cloud Platforms for Generative AI
  29. 29Recap and Q&A: Introduction to Generative AI
  30. 30Building Your First Generative AI Model: A Practical Guide
Ready to check your understanding?Take the Word Up! Bot quiz
esc

Type to search all articles · ↑↓ to move · ↵ to open