POST https://generativelanguage.googleapis.com/v1beta/models/{MODEL_ID}:generateContent?key={API_KEY}
Content-Type: application/json

gemini-omni
gemini-omni-preview
gemini-omni-video
gemini-omni-latest

POST https://generativelanguage.googleapis.com/v1beta/models/{MODEL_ID}:generateContent?key={API_KEY}
Content-Type: application/json

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "text": "Create a 10-second cinematic video. A lone astronaut walks across a glowing alien ocean at sunset. Wide-angle shot, slow dolly forward, warm ethereal lighting, realistic style, one continuous shot."
        }
      ]
    }
  ],
  "generationConfig": {
    "responseModalities": ["VIDEO"]
  }
}

Change the butterfly to a bee.
Change the camera angle to be over the violinist's shoulder.
The lights of the apartments start turning on in sync with the music.

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "fileData": {
            "mimeType": "video/mp4",
            "fileUri": "https://example.com/input-video.mp4"
          }
        },
        {
          "text": "Change the butterfly to a bee. Keep the rest of the video unchanged."
        }
      ]
    }
  ],
  "generationConfig": {
    "responseModalities": ["VIDEO"]
  }
}

The birds from <video> loosely form the imperfect shape of a bird based on <image>. They move to the music from <audio> and dissipate as they fly.

{
  "contents": [
    {
      "role": "user",
      "parts": [
        {
          "fileData": {
            "mimeType": "video/mp4",
            "fileUri": "https://example.com/birds.mp4"
          }
        },
        {
          "fileData": {
            "mimeType": "image/png",
            "fileUri": "https://example.com/bird-shape.png"
          }
        },
        {
          "fileData": {
            "mimeType": "audio/mpeg",
            "fileUri": "https://example.com/music.mp3"
          }
        },
        {
          "text": "The birds from the video loosely form the imperfect shape of the bird in the image. They move to the rhythm of the audio and dissipate as they fly."
        }
      ]
    }
  ],
  "generationConfig": {
    "responseModalities": ["VIDEO"]
  }
}

POST /upload/v1beta/files
-> 返回 fileUri

POST /v1beta/models/{MODEL_ID}:generateContent
-> 返回生成结果或 operation id

GET /v1beta/{operation}
-> 轮询视频生成状态

POST /v1beta/models/{MODEL_ID}:predictLongRunning

POST /v1beta/models/{MODEL_ID}:generateContent

{
  "name": "operations/abc123",
  "metadata": {
    "state": "RUNNING"
  }
}

GET https://generativelanguage.googleapis.com/v1beta/operations/abc123?key={API_KEY}

{
  "name": "operations/abc123",
  "done": true,
  "response": {
    "generatedVideos": [
      {
        "uri": "https://example.com/output.mp4"
      }
    ]
  }
}

Create a 10-second cinematic video of a small wooden boat drifting across a misty lake at sunrise. Wide-angle shot, slow dolly forward, soft golden light, realistic style, one continuous shot.

Change the butterfly to a bee. Keep the flower, lighting, camera movement, and background unchanged.

Change the camera angle to be over the violinist's shoulder. Keep the music performance and room lighting the same.

The lights of the apartments start turning on in sync with the music.

Transform this video into a watercolor animation. Preserve the original motion, composition, and character placement. Use soft paper texture, hand-painted edges, and gentle color bleeding.

Show one word on screen at a time: did, you, know, that, this, model, can, do, pretty, good, text. Each word appears with a different animated style, perfectly paced to the rhythm.

Follow this storyboard exactly in order, starting from the top left. Turn the entire story into a 10-second cinematic video.

import { GoogleGenAI } from "@google/genai";

const ai = new GoogleGenAI({
  apiKey: process.env.GEMINI_API_KEY,
});

const response = await ai.models.generateContent({
  model: "REPLACE_WITH_ACTUAL_OMNI_MODEL_ID",
  contents: [
    {
      role: "user",
      parts: [
        {
          text: "Create a 10-second cinematic video of a glowing city at night. Slow drone shot, neon reflections, realistic style."
        }
      ]
    }
  ],
  config: {
    responseModalities: ["VIDEO"]
  }
});

console.log(response);

from google import genai
import os

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

response = client.models.generate_content(
    model="REPLACE_WITH_ACTUAL_OMNI_MODEL_ID",
    contents=[
        {
            "role": "user",
            "parts": [
                {
                    "text": "Create a 10-second cinematic video of a glowing city at night. Slow drone shot, neon reflections, realistic style."
                }
            ],
        }
    ],
    config={
        "response_modalities": ["VIDEO"]
    },
)

print(response)

curl "https://generativelanguage.googleapis.com/v1beta/models/REPLACE_WITH_ACTUAL_OMNI_MODEL_ID:generateContent?key=$GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '{
    "contents": [
      {
        "role": "user",
        "parts": [
          {
            "text": "Create a 10-second cinematic video of a glowing city at night. Slow drone shot, neon reflections, realistic style."
          }
        ]
      }
    ],
    "generationConfig": {
      "responseModalities": ["VIDEO"]
    }
  }'

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "fileData": {
              "mimeType": "video/mp4",
              "fileUri": "https://example.com/generated-video.mp4"
            }
          }
        ]
      },
      "finishReason": "STOP"
    }
  ]
}

{
  "name": "operations/video-generation-abc123",
  "done": false
}

{
  "name": "operations/video-generation-abc123",
  "done": true,
  "response": {
    "videos": [
      {
        "uri": "https://example.com/output.mp4",
        "mimeType": "video/mp4"
      }
    ]
  }
}

type OmniInput = {
  prompt: string;
  videoUri?: string;
  imageUris?: string[];
  audioUri?: string;
};

type OmniOutput = {
  status: "queued" | "running" | "succeeded" | "failed";
  operationId?: string;
  videoUri?: string;
  error?: string;
};

async function generateOmniVideo(input: OmniInput): Promise<OmniOutput> {
  // 1. 组装 contents.parts
  // 2. 调用 Gemini 或 Vertex API
  // 3. 如果是异步任务，返回 operationId
  // 4. 如果同步完成，返回 videoUri
  throw new Error("Implement after official Omni model ID is available");
}