Generative AI in Image and Video Synthesis
How diffusion models create and edit images and video, which models AWS, Google Cloud and Azure offer, and how to build a safe production pipeline.
Image and video generation went from research demos to everyday tools in about four years. Marketing teams draft campaign visuals from a sentence, retailers place products in scenes that were never photographed, and editors remove objects from footage with a brush stroke. The models are also some of the most expensive and fastest-changing parts of generative AI: several well-known image and video services were retired in 2026 alone.
This article explains how the models work, what you can do with them beyond text-to-image, what each of the three major clouds offers today, and how to build a pipeline that holds up when models change. If you haven’t read it yet, Generative AI in Text Generation covers the language side, and many of the same habits apply here.
The model families behind visual generation
Four families of neural networks have shaped image generation. Most production systems today are built on the last two, but the older ones explain the vocabulary you will meet.
| Family | How it generates | Strengths | Weaknesses | Where you meet it |
|---|---|---|---|---|
| Generative adversarial networks (GANs) | A generator competes with a discriminator that judges real from fake | Very fast, sharp results in a narrow domain | Unstable training, poor at following text | Face generation, super-resolution, older tools |
| Variational autoencoders (VAEs) | Encodes images into a compact latent space and decodes them back | Smooth, structured latent space | Blurry on their own | As the compression stage inside diffusion models |
| Diffusion models | Learns to remove noise step by step until an image appears | High quality, strong text control, easy to edit | Slower, many steps per image | Most current image models |
| Transformers | Attention over image or video patches, sometimes predicting tokens | Scales well, handles long context and video | Very large compute needs | Video models, and image models built into multimodal LLMs |
GANs are covered in depth in Introduction to Generative Adversarial Networks (GANs) and VAEs in Introduction to Variational Autoencoders (VAEs). The rest of this section focuses on diffusion, because it is what you are calling when you use almost any current image API.
How diffusion works
Training a diffusion model starts with real images and adds random noise to them in small amounts until nothing but noise is left. The network learns to predict, at each step, the noise that was added. Once it can do that, you can run the process backwards: start from pure noise and remove a little predicted noise at each step until a clean image remains.
Two additions make this practical. First, latent diffusion runs the denoising in the compressed space of a VAE instead of on full-resolution pixels, which cuts the compute by a large factor; Stable Diffusion popularised this design. Second, conditioning lets a text encoder turn your prompt into a vector that steers each denoising step, so the noise is resolved towards “a red bicycle against a brick wall” rather than any image at all.
flowchart LR
P["Text prompt"] --> TE["Text encoder"]
N["Random noise in latent space"] --> D["Denoising network"]
TE --> D
D --> L{"More steps left?"}
L -->|Yes| D
L -->|No| V["VAE decoder"]
V --> I["Final image"]
The settings you see in image APIs map directly onto this loop:
- Steps is the number of denoising passes. More steps usually mean more detail and a slower, costlier request, with diminishing returns.
- Guidance scale controls how strongly the prompt steers each step. Too low and the image ignores the prompt; too high and it looks oversaturated or distorted.
- Seed fixes the starting noise. The same prompt, model and seed reproduce the same image, which matters for testing and for regenerating an approved asset.
- Negative prompt, where supported, describes what to steer away from, such as “text, watermark, extra fingers”.
Hosted models increasingly hide steps and guidance behind a quality or speed tier, and models built into multimodal LLMs, such as Google’s Gemini image models and OpenAI’s GPT image models, take a conversation instead of a single prompt. The underlying trade-off between quality, speed and cost stays the same.
You generated an approved product image last week and need to recreate it exactly. Which setting matters most?
Correct! The seed fixes the starting noise, so the same seed, prompt and model reproduce the same image.
From images to video
Video adds a dimension that images don’t have: time. A video model has to keep a face, a logo or a room layout consistent across dozens of frames while things move in a physically believable way. Generating each frame separately with an image model produces flicker, because every frame resolves the noise slightly differently.
Current video models handle this by treating a clip as one block of data. They compress the video into a latent space, cut it into patches that span both space and time, and denoise all of them together with a transformer, so attention can link a patch in frame 1 with the same object in frame 60. This is why video generation is so much more expensive than image generation: the model processes every frame of the clip at once.
In practice, today’s hosted video models share a few traits:
- Short clips. Outputs are typically several seconds long. Longer sequences are built by extending a clip or stitching shots together.
- Several inputs. Besides text, you can usually pass a starting image, and some models accept a last frame or reference images of a subject to keep it consistent.
- Asynchronous jobs. A request returns an operation ID, not a video. You poll for completion, and the result lands in object storage.
- Audio is model-specific. Some recent models generate synchronised sound along with the video; others return silent clips.
What you can do beyond text-to-image
Generating a picture from a sentence is the demo. Most business value comes from editing and controlling images you already have.
| Technique | What it does | Typical use |
|---|---|---|
| Image-to-image | Starts from an existing image instead of pure noise | Restyling, turning a sketch into a render |
| Inpainting | Regenerates a masked area to match its surroundings | Removing objects, replacing a background item |
| Outpainting | Extends an image beyond its borders | Changing aspect ratio for different ad formats |
| Background removal and replacement | Separates subject from background and generates a new scene | Product photos in lifestyle settings |
| Structural control | Guides generation with an edge map, depth map or pose | Keeping a product’s exact shape while changing the scene |
| Upscaling | Adds detail while increasing resolution | Print-quality output from a web-sized image |
| Personalisation | Fine-tunes a model on a few images of a subject or style, often with LoRA adapters | Consistent brand style or a recurring character |
| Image-to-video | Animates a still image | Short product spins and social clips |
Inpainting and structural control are worth calling out, because they solve the biggest problem with generated product imagery: a text-to-image model will happily invent a slightly different product. If you keep the real product pixels and only generate the scene around them, what the customer sees is what they will receive.
Personalisation techniques such as DreamBooth and LoRA (low-rank adaptation) train a small set of extra weights on a handful of example images. A LoRA file is often tens of megabytes rather than gigabytes, so you can keep one per brand or product line and load the right one at request time. This is usually done on open-weight models you host yourself; hosted APIs offer it only on some models.
An online shop wants lifestyle photos of its real kettle in different kitchens. Which approach keeps the kettle accurate?
Correct! Background replacement or inpainting keeps the real product pixels, so the model can't invent a different handle or logo.
Image and video models on each cloud
Each cloud offers image and video generation as a managed API, billed on your cloud invoice and controlled with your cloud’s identity system. The line-up changed sharply during 2026, so treat this section as a snapshot and check each provider’s model lifecycle page before you build on a specific model.
| Amazon Bedrock | Google Cloud | Azure (Microsoft Foundry) | |
|---|---|---|---|
| Image models | Stability AI models such as Stable Image Ultra, Stable Image Core and Stable Diffusion 3.5 Large | Gemini image models | OpenAI GPT image models, plus partner models such as Black Forest Labs FLUX |
| Image editing | Stability AI Image Services (inpainting, object removal, background removal, structure and style control) | Conversational editing with Gemini image models | Edits through the GPT image models |
| Video models | Luma Ray 2 | Veo 3.1 | No first-party hosted video model at the time of writing |
| Retired in 2026 | Amazon Nova Canvas and Nova Reel | Imagen | DALL-E 3 and Sora 2 |
| Provenance | Depends on the model provider | SynthID watermark on Google’s generated images and video | C2PA Content Credentials on images from OpenAI models |
| Access | IAM, model access per Region | Google Cloud IAM | Microsoft Entra ID or API key; some models need an access request |
The retirements are the important column. Amazon moved Nova Canvas and Nova Reel to legacy status in March 2026, Google replaced Imagen with its Gemini image models, Azure retired DALL-E 3 in favour of the GPT image series, and OpenAI wound down Sora, including its video API, during 2026. Code written against any of these needs more than a new model ID, because request formats differ between models.
The examples below keep the model ID in an environment variable for that reason.
Amazon Bedrock
Bedrock’s image models are called with invoke_model, and each provider defines its own request body. This example uses a Stability AI model, which returns base64-encoded images.
import base64
import json
import os
import boto3
client = boto3.client("bedrock-runtime", region_name="us-west-2")
body = {
"prompt": "A steel water bottle on a granite rock beside a mountain lake, morning light",
"aspect_ratio": "16:9",
"output_format": "png",
}
response = client.invoke_model(
modelId=os.environ["BEDROCK_IMAGE_MODEL_ID"],
body=json.dumps(body),
)
result = json.loads(response["body"].read())
with open("bottle.png", "wb") as f:
f.write(base64.b64decode(result["images"][0]))
Video on Bedrock runs as an asynchronous job with start_async_invoke, which writes the finished clip to an S3 bucket you own. Check the Region list for each model; image and video models are often available in fewer Regions than text models. Cloud Services for Generative AI: AWS Offerings covers the rest of Bedrock.
Google Cloud
The Google Gen AI SDK calls both the Gemini image models and Veo. Video generation is a long-running operation: you start it, poll, and read the result from Cloud Storage.
import os
import time
from google import genai
from google.genai import types
client = genai.Client(
vertexai=True,
project=os.environ["GOOGLE_CLOUD_PROJECT"],
location="us-central1",
)
operation = client.models.generate_videos(
model=os.environ["VEO_MODEL"],
prompt="Slow orbit around a steel water bottle on a rock, lake in the background",
config=types.GenerateVideosConfig(
aspect_ratio="16:9",
output_gcs_uri="gs://my-media-bucket/videos/",
),
)
while not operation.done:
time.sleep(15)
operation = client.operations.get(operation)
print(operation.result.generated_videos[0].video.uri)
For images, you call generate_content with a Gemini image model and read the image bytes from the response parts. Google’s AI platform is now called Gemini Enterprise Agent Platform, though the SDK flag and many docs still say Vertex AI. Cloud Services for Generative AI: GCP Offerings has the wider picture.
Azure
GPT image models in Microsoft Foundry work with the standard OpenAI client through the v1 endpoint. Pass your deployment name as the model.
import base64
import os
from openai import OpenAI
client = OpenAI(
base_url="https://my-resource.openai.azure.com/openai/v1/",
api_key=os.environ["AZURE_OPENAI_API_KEY"],
)
result = client.images.generate(
model=os.environ["AZURE_IMAGE_DEPLOYMENT"], # deployment name
prompt="A steel water bottle on a granite rock beside a mountain lake, morning light",
size="1024x1024",
)
with open("bottle.png", "wb") as f:
f.write(base64.b64decode(result.data[0].b64_json))
client.images.edit takes an image and an optional mask for inpainting. In production, authenticate with Microsoft Entra ID and a managed identity instead of a key. Cloud Services for Generative AI: Azure Offerings covers Foundry in more detail.
Hosting open models yourself
Open-weight image and video models, run with libraries such as Hugging Face Diffusers, give you full control: your own LoRA adapters, structural control, fixed versions that never get retired, and no per-image fee. The cost is GPUs, which you pay for whether or not they are busy, plus the work of scaling and patching the serving stack. Licences vary a lot between open models, and some forbid commercial use, so read the licence before you build a product on one.
Self-hosting tends to pay off at steady high volume or when you need customisation the APIs don’t offer. For occasional or spiky workloads, a managed API is almost always cheaper.
A production media pipeline
Calling the model is one step. A feature that publishes generated images under your name needs checks before and after it. Here is a pipeline for a retailer generating lifestyle images for new products:
sequenceDiagram
participant App as Catalogue job
participant S as Object storage
participant M as Image model API
participant F as Safety checks
participant R as Reviewer
App->>S: Read product photo and attributes
App->>M: Background replacement request
M-->>App: Generated image
App->>F: Content filter, product match, size check
alt Passes
F-->>R: Queue for review
R-->>S: Approve and publish with metadata
else Fails
F-->>App: Retry with new seed, then flag
end
What each step is for:
- Real inputs. The product photo and its attributes come from your catalogue. The model creates the scene, not the product.
- Stored metadata. Save the model ID, prompt, seed and parameters with every output. You’ll need them to reproduce an asset, to answer “was this generated?” and to rerun a batch on a new model.
- Automated checks. The cloud content filters screen for unsafe content. Add your own checks for what they can’t know: that the product still matches the original, that no text or logos were invented, and that the size fits the placement.
- Human review for anything customers see. Reviewing ten candidates is still much faster than a photo shoot.
- Asynchronous processing. Image calls take seconds and video calls take minutes, so run generation from a queue and don’t block a web request on it.
- Caching. Identical requests with the same seed return the same output, so store results and serve them from a CDN instead of generating again.
Cost
Image generation is usually billed per image, with the price depending on the model, the resolution and the quality tier. Models built into multimodal LLMs are often billed in tokens, where an output image counts as a block of output tokens. Video is billed per second of generated output, which makes it the most expensive item here by a wide margin; a few seconds of video can cost as much as many images.
Prices change often, so this article doesn’t print them. Use the official pages: Amazon Bedrock pricing, Generative AI pricing on Google Cloud and Azure OpenAI pricing.
The levers that matter most:
- Generate drafts at low resolution or a fast tier, and render only the approved ones at full quality.
- Ask for one or two candidates per request rather than eight.
- Cache and reuse outputs keyed on prompt, seed and model.
- For video, keep clips short and generate from an approved still image, so you don’t pay for clips that miss the brief.
- Set a budget alert and a per-user rate limit before a public feature launches. AWS Free Tier and Billing Overview shows how budgets work on AWS.
A team plans to generate a product video for every listing. What drives the cost most?
Correct! Video is billed per second of output, so clip length and the number of retries decide the bill.
Safety, rights and provenance
Generated images and video raise problems text doesn’t, because a realistic picture is easy to mistake for a record of something that happened.
- Misuse and deepfakes. Managed services refuse many requests involving real people and screen outputs for harmful content. They reduce risk; they don’t remove it. Real-World Use Cases: Deepfakes and Their Impact covers the wider issue.
- Provenance. Google marks its generated media with SynthID, an invisible watermark, and OpenAI’s image models attach C2PA Content Credentials, a signed metadata record. Metadata can be stripped when an image is re-encoded, so keep your own record of what you generated as well.
- Rights. Check the provider’s terms for ownership of outputs and any indemnity they offer, avoid prompts that name living artists or trademarks, and get consent before using a real person’s likeness for personalisation.
- Disclosure. Label generated media where people could be misled, and check whether advertising or election rules in your region require it. Ethical Considerations in Generative AI goes further.
Further reading
- Amazon Bedrock model lifecycle and Luma AI models on Bedrock
- Generate videos with Veo 3.1 in the Gemini API and Generative AI deprecations on Google Cloud
- Image generation models in Microsoft Foundry and the Foundry model retirement schedule
- Case Study: AI-Generated Artwork and The Role of Transformers in Generative AI
Key takeaways
- Most current image models are latent diffusion models: they remove noise step by step in a compressed space, steered by your prompt. Steps, guidance and seed control the result.
- Video models denoise a whole clip at once over patches that span space and time, which keeps frames consistent and makes video far more expensive than images.
- Editing techniques such as inpainting, background replacement and structural control deliver more business value than text-to-image, because they keep real content intact.
- Bedrock offers Stability AI and Luma models, Google Cloud offers Gemini image models and Veo, and Azure offers OpenAI’s GPT image models. Nova Canvas, Nova Reel, Imagen, DALL-E 3 and Sora were all retired in 2026.
- Keep model IDs in configuration, store the prompt, seed and model with every output, and run generation asynchronously behind checks and human review.
- Check the official pricing pages, and control cost with low-resolution drafts, caching, short clips and budget alerts.
Part 19 of 30 · Introduction to Generative AI
- 01Introduction to Generative AI: What Is It?
- 02The History of Generative AI: From Concept to Reality
- 03Key Definitions and Concepts in Generative AI
- 04Pioneers of Generative AI: Meet the Inventors
- 05Understanding Neural Networks: The Backbone of AI
- 06Deep Learning Basics: Layers and Learning Rates
- 07Generative vs. Discriminative Models: What's the Difference?
- 08Overview of Generative Models: VAEs, GANs, and More
- 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
- 10Introduction to Generative Adversarial Networks (GANs)
- 11The Role of Transformers in Generative AI
- 12Applications of Generative AI in Everyday Life
- 13Ethical Considerations in Generative AI: Navigating the New Frontier
- 14Generative AI in Art and Music: The Creative Revolution
- 15Case Study: AI-Generated Artwork
- 16Generative AI in Text Generation
- 17Exploring OpenAI's GPT Models: Word Up! Bot
- 18Real-World Use Cases: Deepfakes and Their Impact
- 19Generative AI in Image and Video Synthesis
- 20Introduction to Reinforcement Learning in Generative AI
- 21The Future of Generative AI: Trends to Watch
- 22Generative AI in Healthcare: Opportunities and Challenges
- 23Generative AI in Finance: Risk Modeling and Beyond
- 24Generative AI in Marketing and Personalization
- 25Cloud Services for Generative AI: AWS Offerings
- 26Cloud Services for Generative AI: GCP Offerings
- 27Cloud Services for Generative AI: Azure Offerings
- 28Comparing Cloud Platforms for Generative AI
- 29Recap and Q&A: Introduction to Generative AI
- 30Building Your First Generative AI Model: A Practical Guide