Generative AI in Image and Video Synthesis

How diffusion models create and edit images and video, which models AWS, Google Cloud and Azure offer, and how to build a safe production pipeline.

Generative AI

Image and video generation went from research demos to everyday tools in about four years. Marketing teams draft campaign visuals from a sentence, retailers place products in scenes that were never photographed, and editors remove objects from footage with a brush stroke. The models are also some of the most expensive and fastest-changing parts of generative AI: several well-known image and video services were retired in 2026 alone.

This article explains how the models work, what you can do with them beyond text-to-image, what each of the three major clouds offers today, and how to build a pipeline that holds up when models change. If you haven’t read it yet, Generative AI in Text Generation covers the language side, and many of the same habits apply here.

The model families behind visual generation

Four families of neural networks have shaped image generation. Most production systems today are built on the last two, but the older ones explain the vocabulary you will meet.

FamilyHow it generatesStrengthsWeaknessesWhere you meet it
Generative adversarial networks (GANs)A generator competes with a discriminator that judges real from fakeVery fast, sharp results in a narrow domainUnstable training, poor at following textFace generation, super-resolution, older tools
Variational autoencoders (VAEs)Encodes images into a compact latent space and decodes them backSmooth, structured latent spaceBlurry on their ownAs the compression stage inside diffusion models
Diffusion modelsLearns to remove noise step by step until an image appearsHigh quality, strong text control, easy to editSlower, many steps per imageMost current image models
TransformersAttention over image or video patches, sometimes predicting tokensScales well, handles long context and videoVery large compute needsVideo models, and image models built into multimodal LLMs

GANs are covered in depth in Introduction to Generative Adversarial Networks (GANs) and VAEs in Introduction to Variational Autoencoders (VAEs). The rest of this section focuses on diffusion, because it is what you are calling when you use almost any current image API.

How diffusion works

Training a diffusion model starts with real images and adds random noise to them in small amounts until nothing but noise is left. The network learns to predict, at each step, the noise that was added. Once it can do that, you can run the process backwards: start from pure noise and remove a little predicted noise at each step until a clean image remains.

Two additions make this practical. First, latent diffusion runs the denoising in the compressed space of a VAE instead of on full-resolution pixels, which cuts the compute by a large factor; Stable Diffusion popularised this design. Second, conditioning lets a text encoder turn your prompt into a vector that steers each denoising step, so the noise is resolved towards “a red bicycle against a brick wall” rather than any image at all.

flowchart LR
    P["Text prompt"] --> TE["Text encoder"]
    N["Random noise in latent space"] --> D["Denoising network"]
    TE --> D
    D --> L{"More steps left?"}
    L -->|Yes| D
    L -->|No| V["VAE decoder"]
    V --> I["Final image"]

The settings you see in image APIs map directly onto this loop:

  • Steps is the number of denoising passes. More steps usually mean more detail and a slower, costlier request, with diminishing returns.
  • Guidance scale controls how strongly the prompt steers each step. Too low and the image ignores the prompt; too high and it looks oversaturated or distorted.
  • Seed fixes the starting noise. The same prompt, model and seed reproduce the same image, which matters for testing and for regenerating an approved asset.
  • Negative prompt, where supported, describes what to steer away from, such as “text, watermark, extra fingers”.

Hosted models increasingly hide steps and guidance behind a quality or speed tier, and models built into multimodal LLMs, such as Google’s Gemini image models and OpenAI’s GPT image models, take a conversation instead of a single prompt. The underlying trade-off between quality, speed and cost stays the same.

You generated an approved product image last week and need to recreate it exactly. Which setting matters most?

  • A higher guidance scale
  • More denoising steps
  • The same seed, with the same prompt and model version

Correct! The seed fixes the starting noise, so the same seed, prompt and model reproduce the same image.

From images to video

Video adds a dimension that images don’t have: time. A video model has to keep a face, a logo or a room layout consistent across dozens of frames while things move in a physically believable way. Generating each frame separately with an image model produces flicker, because every frame resolves the noise slightly differently.

Current video models handle this by treating a clip as one block of data. They compress the video into a latent space, cut it into patches that span both space and time, and denoise all of them together with a transformer, so attention can link a patch in frame 1 with the same object in frame 60. This is why video generation is so much more expensive than image generation: the model processes every frame of the clip at once.

In practice, today’s hosted video models share a few traits:

  • Short clips. Outputs are typically several seconds long. Longer sequences are built by extending a clip or stitching shots together.
  • Several inputs. Besides text, you can usually pass a starting image, and some models accept a last frame or reference images of a subject to keep it consistent.
  • Asynchronous jobs. A request returns an operation ID, not a video. You poll for completion, and the result lands in object storage.
  • Audio is model-specific. Some recent models generate synchronised sound along with the video; others return silent clips.

What you can do beyond text-to-image

Generating a picture from a sentence is the demo. Most business value comes from editing and controlling images you already have.

TechniqueWhat it doesTypical use
Image-to-imageStarts from an existing image instead of pure noiseRestyling, turning a sketch into a render
InpaintingRegenerates a masked area to match its surroundingsRemoving objects, replacing a background item
OutpaintingExtends an image beyond its bordersChanging aspect ratio for different ad formats
Background removal and replacementSeparates subject from background and generates a new sceneProduct photos in lifestyle settings
Structural controlGuides generation with an edge map, depth map or poseKeeping a product’s exact shape while changing the scene
UpscalingAdds detail while increasing resolutionPrint-quality output from a web-sized image
PersonalisationFine-tunes a model on a few images of a subject or style, often with LoRA adaptersConsistent brand style or a recurring character
Image-to-videoAnimates a still imageShort product spins and social clips

Inpainting and structural control are worth calling out, because they solve the biggest problem with generated product imagery: a text-to-image model will happily invent a slightly different product. If you keep the real product pixels and only generate the scene around them, what the customer sees is what they will receive.

Personalisation techniques such as DreamBooth and LoRA (low-rank adaptation) train a small set of extra weights on a handful of example images. A LoRA file is often tens of megabytes rather than gigabytes, so you can keep one per brand or product line and load the right one at request time. This is usually done on open-weight models you host yourself; hosted APIs offer it only on some models.

An online shop wants lifestyle photos of its real kettle in different kitchens. Which approach keeps the kettle accurate?

  • Keep the product photo and generate only the background around it
  • Describe the kettle in detail in a text-to-image prompt
  • Raise the number of denoising steps

Correct! Background replacement or inpainting keeps the real product pixels, so the model can't invent a different handle or logo.

Image and video models on each cloud

Each cloud offers image and video generation as a managed API, billed on your cloud invoice and controlled with your cloud’s identity system. The line-up changed sharply during 2026, so treat this section as a snapshot and check each provider’s model lifecycle page before you build on a specific model.

Amazon BedrockGoogle CloudAzure (Microsoft Foundry)
Image modelsStability AI models such as Stable Image Ultra, Stable Image Core and Stable Diffusion 3.5 LargeGemini image modelsOpenAI GPT image models, plus partner models such as Black Forest Labs FLUX
Image editingStability AI Image Services (inpainting, object removal, background removal, structure and style control)Conversational editing with Gemini image modelsEdits through the GPT image models
Video modelsLuma Ray 2Veo 3.1No first-party hosted video model at the time of writing
Retired in 2026Amazon Nova Canvas and Nova ReelImagenDALL-E 3 and Sora 2
ProvenanceDepends on the model providerSynthID watermark on Google’s generated images and videoC2PA Content Credentials on images from OpenAI models
AccessIAM, model access per RegionGoogle Cloud IAMMicrosoft Entra ID or API key; some models need an access request

The retirements are the important column. Amazon moved Nova Canvas and Nova Reel to legacy status in March 2026, Google replaced Imagen with its Gemini image models, Azure retired DALL-E 3 in favour of the GPT image series, and OpenAI wound down Sora, including its video API, during 2026. Code written against any of these needs more than a new model ID, because request formats differ between models.

The examples below keep the model ID in an environment variable for that reason.

Amazon Bedrock

Bedrock’s image models are called with invoke_model, and each provider defines its own request body. This example uses a Stability AI model, which returns base64-encoded images.

import base64
import json
import os
import boto3

client = boto3.client("bedrock-runtime", region_name="us-west-2")

body = {
    "prompt": "A steel water bottle on a granite rock beside a mountain lake, morning light",
    "aspect_ratio": "16:9",
    "output_format": "png",
}

response = client.invoke_model(
    modelId=os.environ["BEDROCK_IMAGE_MODEL_ID"],
    body=json.dumps(body),
)

result = json.loads(response["body"].read())
with open("bottle.png", "wb") as f:
    f.write(base64.b64decode(result["images"][0]))

Video on Bedrock runs as an asynchronous job with start_async_invoke, which writes the finished clip to an S3 bucket you own. Check the Region list for each model; image and video models are often available in fewer Regions than text models. Cloud Services for Generative AI: AWS Offerings covers the rest of Bedrock.

Google Cloud

The Google Gen AI SDK calls both the Gemini image models and Veo. Video generation is a long-running operation: you start it, poll, and read the result from Cloud Storage.

import os
import time
from google import genai
from google.genai import types

client = genai.Client(
    vertexai=True,
    project=os.environ["GOOGLE_CLOUD_PROJECT"],
    location="us-central1",
)

operation = client.models.generate_videos(
    model=os.environ["VEO_MODEL"],
    prompt="Slow orbit around a steel water bottle on a rock, lake in the background",
    config=types.GenerateVideosConfig(
        aspect_ratio="16:9",
        output_gcs_uri="gs://my-media-bucket/videos/",
    ),
)

while not operation.done:
    time.sleep(15)
    operation = client.operations.get(operation)

print(operation.result.generated_videos[0].video.uri)

For images, you call generate_content with a Gemini image model and read the image bytes from the response parts. Google’s AI platform is now called Gemini Enterprise Agent Platform, though the SDK flag and many docs still say Vertex AI. Cloud Services for Generative AI: GCP Offerings has the wider picture.

Azure

GPT image models in Microsoft Foundry work with the standard OpenAI client through the v1 endpoint. Pass your deployment name as the model.

import base64
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://my-resource.openai.azure.com/openai/v1/",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
)

result = client.images.generate(
    model=os.environ["AZURE_IMAGE_DEPLOYMENT"],  # deployment name
    prompt="A steel water bottle on a granite rock beside a mountain lake, morning light",
    size="1024x1024",
)

with open("bottle.png", "wb") as f:
    f.write(base64.b64decode(result.data[0].b64_json))

client.images.edit takes an image and an optional mask for inpainting. In production, authenticate with Microsoft Entra ID and a managed identity instead of a key. Cloud Services for Generative AI: Azure Offerings covers Foundry in more detail.

Hosting open models yourself

Open-weight image and video models, run with libraries such as Hugging Face Diffusers, give you full control: your own LoRA adapters, structural control, fixed versions that never get retired, and no per-image fee. The cost is GPUs, which you pay for whether or not they are busy, plus the work of scaling and patching the serving stack. Licences vary a lot between open models, and some forbid commercial use, so read the licence before you build a product on one.

Self-hosting tends to pay off at steady high volume or when you need customisation the APIs don’t offer. For occasional or spiky workloads, a managed API is almost always cheaper.

A production media pipeline

Calling the model is one step. A feature that publishes generated images under your name needs checks before and after it. Here is a pipeline for a retailer generating lifestyle images for new products:

sequenceDiagram
    participant App as Catalogue job
    participant S as Object storage
    participant M as Image model API
    participant F as Safety checks
    participant R as Reviewer
    App->>S: Read product photo and attributes
    App->>M: Background replacement request
    M-->>App: Generated image
    App->>F: Content filter, product match, size check
    alt Passes
        F-->>R: Queue for review
        R-->>S: Approve and publish with metadata
    else Fails
        F-->>App: Retry with new seed, then flag
    end

What each step is for:

  • Real inputs. The product photo and its attributes come from your catalogue. The model creates the scene, not the product.
  • Stored metadata. Save the model ID, prompt, seed and parameters with every output. You’ll need them to reproduce an asset, to answer “was this generated?” and to rerun a batch on a new model.
  • Automated checks. The cloud content filters screen for unsafe content. Add your own checks for what they can’t know: that the product still matches the original, that no text or logos were invented, and that the size fits the placement.
  • Human review for anything customers see. Reviewing ten candidates is still much faster than a photo shoot.
  • Asynchronous processing. Image calls take seconds and video calls take minutes, so run generation from a queue and don’t block a web request on it.
  • Caching. Identical requests with the same seed return the same output, so store results and serve them from a CDN instead of generating again.

Cost

Image generation is usually billed per image, with the price depending on the model, the resolution and the quality tier. Models built into multimodal LLMs are often billed in tokens, where an output image counts as a block of output tokens. Video is billed per second of generated output, which makes it the most expensive item here by a wide margin; a few seconds of video can cost as much as many images.

Prices change often, so this article doesn’t print them. Use the official pages: Amazon Bedrock pricing, Generative AI pricing on Google Cloud and Azure OpenAI pricing.

The levers that matter most:

  • Generate drafts at low resolution or a fast tier, and render only the approved ones at full quality.
  • Ask for one or two candidates per request rather than eight.
  • Cache and reuse outputs keyed on prompt, seed and model.
  • For video, keep clips short and generate from an approved still image, so you don’t pay for clips that miss the brief.
  • Set a budget alert and a per-user rate limit before a public feature launches. AWS Free Tier and Billing Overview shows how budgets work on AWS.

A team plans to generate a product video for every listing. What drives the cost most?

  • The length of the text prompt
  • The number of seconds of video generated, including rejected attempts
  • The number of denoising steps the provider uses internally

Correct! Video is billed per second of output, so clip length and the number of retries decide the bill.

Safety, rights and provenance

Generated images and video raise problems text doesn’t, because a realistic picture is easy to mistake for a record of something that happened.

  • Misuse and deepfakes. Managed services refuse many requests involving real people and screen outputs for harmful content. They reduce risk; they don’t remove it. Real-World Use Cases: Deepfakes and Their Impact covers the wider issue.
  • Provenance. Google marks its generated media with SynthID, an invisible watermark, and OpenAI’s image models attach C2PA Content Credentials, a signed metadata record. Metadata can be stripped when an image is re-encoded, so keep your own record of what you generated as well.
  • Rights. Check the provider’s terms for ownership of outputs and any indemnity they offer, avoid prompts that name living artists or trademarks, and get consent before using a real person’s likeness for personalisation.
  • Disclosure. Label generated media where people could be misled, and check whether advertising or election rules in your region require it. Ethical Considerations in Generative AI goes further.

Further reading

Key takeaways

  • Most current image models are latent diffusion models: they remove noise step by step in a compressed space, steered by your prompt. Steps, guidance and seed control the result.
  • Video models denoise a whole clip at once over patches that span space and time, which keeps frames consistent and makes video far more expensive than images.
  • Editing techniques such as inpainting, background replacement and structural control deliver more business value than text-to-image, because they keep real content intact.
  • Bedrock offers Stability AI and Luma models, Google Cloud offers Gemini image models and Veo, and Azure offers OpenAI’s GPT image models. Nova Canvas, Nova Reel, Imagen, DALL-E 3 and Sora were all retired in 2026.
  • Keep model IDs in configuration, store the prompt, seed and model with every output, and run generation asynchronously behind checks and human review.
  • Check the official pricing pages, and control cost with low-resolution drafts, caching, short clips and budget alerts.

Part 19 of 30 · Introduction to Generative AI

  1. 01Introduction to Generative AI: What Is It?
  2. 02The History of Generative AI: From Concept to Reality
  3. 03Key Definitions and Concepts in Generative AI
  4. 04Pioneers of Generative AI: Meet the Inventors
  5. 05Understanding Neural Networks: The Backbone of AI
  6. 06Deep Learning Basics: Layers and Learning Rates
  7. 07Generative vs. Discriminative Models: What's the Difference?
  8. 08Overview of Generative Models: VAEs, GANs, and More
  9. 09Introduction to Variational Autoencoders (VAEs): From Basics to Advanced Applications
  10. 10Introduction to Generative Adversarial Networks (GANs)
  11. 11The Role of Transformers in Generative AI
  12. 12Applications of Generative AI in Everyday Life
  13. 13Ethical Considerations in Generative AI: Navigating the New Frontier
  14. 14Generative AI in Art and Music: The Creative Revolution
  15. 15Case Study: AI-Generated Artwork
  16. 16Generative AI in Text Generation
  17. 17Exploring OpenAI's GPT Models: Word Up! Bot
  18. 18Real-World Use Cases: Deepfakes and Their Impact
  19. 19Generative AI in Image and Video Synthesis
  20. 20Introduction to Reinforcement Learning in Generative AI
  21. 21The Future of Generative AI: Trends to Watch
  22. 22Generative AI in Healthcare: Opportunities and Challenges
  23. 23Generative AI in Finance: Risk Modeling and Beyond
  24. 24Generative AI in Marketing and Personalization
  25. 25Cloud Services for Generative AI: AWS Offerings
  26. 26Cloud Services for Generative AI: GCP Offerings
  27. 27Cloud Services for Generative AI: Azure Offerings
  28. 28Comparing Cloud Platforms for Generative AI
  29. 29Recap and Q&A: Introduction to Generative AI
  30. 30Building Your First Generative AI Model: A Practical Guide
Ready to check your understanding?Take the Word Up! Bot quiz
esc

Type to search all articles · ↑↓ to move · ↵ to open