Omni-Video
Create Task
The Omni model can achieve various capabilities through Prompt with elements, images, videos, and other content.
Request Header
Data Exchange Format
Authentication information, refer to API authentication
Request Body
Model Name
Whether to generate multi-shot video.
When true: the prompt parameter is invalid, and it does not support setting the start & end frames to generate videos
When false: the shot_type and multi_prompt parameters are invalid.
Storyboard method.
When multi_shot is true, this parameter is required.
Text prompt words, which can include positive and negative descriptions.
- The prompt words can be templated to meet different video generation needs
- Must not exceed 2,500 characters
When the "multi_shot" parameter is set to false or when the "multi_shot" parameter is set to true and the "shot-type" parameter is set to intelligence, the current parameter must not be empty.
The Omni model can achieve various capabilities through Prompt with elements, images, videos, and other content:
- Specify elements/images/videos using <<<>>> format, e.g.: <<<element_1>>>, <<<image_1>>>, <<<video_1>>>
- For detailed capabilities, see: KLING Omni Model User Guide, Kling VIDEO 3.0 Omni Model User Guide
The support range for different model versions and video modes varies. For details, see Capability Map
Information about each storyboard, such as prompts and duration.
Define shot sequence number, corresponding prompt and duration via index, prompt, and duration parameters.
- Supports up to 6 storyboards, minimum 1.
- Max length per storyboard content: 512.
- Each storyboard duration must not exceed total task duration and must be ≥ 1.
- Sum of all storyboard durations must equal total task duration.
When multi_shot is true and shot_type is customize, this parameter is required. Format:
Reference Image List, including element, scene, style reference images.
- Including reference images of the element, scene, style, etc., it can also be used as the start or end frame to generate videos; When generating a video as the start or end frame:
- Define whether the image is in the first and last frames using the
typeparameter:first_frameis the start frame,end_frameis the end frame.- Currently does not support only the end frame, which means that when there is a end frame image, there must be a first frame image.
- If the image is not the start & end frame, do not configure the
typeparameter.
- When generating a video using the first frame or the first and last frames, video editing functions cannot be used.
- Define whether the image is in the first and last frames using the
Load with key:value, details as follows:
Image Requirements:
- Supports Base64 encoding or image URL (ensure accessibility)
- Formats: .jpg / .jpeg / .png
- File size: ≤10MB
- Dimensions: min 300px, aspect ratio 1:2.5 ~ 2.5:1
Quantity Limits:
- When there are reference video, the sum of the amount of reference image and the amount of reference element shall not exceed 4.
- When there is no reference video, the sum of the amount of reference image and the amount of reference element shall not exceed 7.
- When using the O1 model, setting the start & end frame is not supported when there are more than two images in the array.
image_urlparameter value must not be empty
Image URL or Base64
Frame type: first_frame or end_frame
Reference Element List, based on element ID from element library.
- When generating a video using the first frame, a maximum of 3 elements are supported.
- When generating videos using the start & end frames, the kling-v3-omni model supports up to 3 elements, while the kling-video-o1 model does not support 1 element.
- When there is a reference video, the sum of the number of reference images and the number of reference subjects must not exceed 4, and the use of video subjects is not supported.
- When there is no reference video, the sum of the number of reference images and the number of reference subjects must not exceed 7.
Elements are categorized into video character elements and Multi-Image Elements, with different scopes. See Kling Element Library User Guide.
- Load with key:value format as follows:
The support range for different model versions and video modes varies. For details, see Capability Map
Element ID from element library
Reference Video, obtained via URL.
Video Types:
- Can be used as feature reference video OR base video for editing (default: base)
- Use
refer_typeparameter:featurefor feature reference,basefor video to be edited - When reference video is base type, first/end frames cannot be defined
- Use
keep_original_soundto keep original sound:yesto keep,noto discard (also applies to feature type)
When there is a reference video, the value of the sound parameter can only be off.
- Load with key:value format as follows:
Video Requirements:
- Format: MP4/MOV only
- Duration: ≥3 seconds, upper limit depends on model version (see Capability Map)
- Resolution: 720px-2160px (width and height)
- Frame rate: 24-60fps (output will be 24fps)
- Max 1 video, size ≤200MB
video_urlparameter value must not be empty
The support range for different model versions and video modes varies. For details, see Capability Map
Video URL
Reference type: feature (feature reference) or base (video to be edited)
base- Transformation (Video Edit):
Edit video content - add/remove/modify elements, change shot composition, angles, styles, colors, weather, etc.
feature- Video Reference:
Reference video content to generate next/previous shot, or reference video style/camera movement.
Keep original sound: yes to keep, no to discard
Whether to generate sound simultaneously when generating videos.
The support range for different model versions and video modes varies. For details, see Capability Map
Video generation mode
std: Standard mode, cost-effectivepro: Professional mode (high quality), better video quality output
The support range for different model versions and video modes varies. For details, see Capability Map
Aspect ratio of generated video frame (width:height)
- This parameter is required when NOT using first-frame reference or video editing function.
Video duration in seconds
- When using the video editing function ("refer_type": "base"), the output result is the same as the duration of the incoming video, and the current parameter is invalid. Calculate billing by rounding the input video duration to the nearest integer.
The support range for different model versions and video modes varies. For details, see Capability Map
Whether to generate watermarked results simultaneously
- Defined by the enabled parameter, format:
- true: generate watermarked result, false: do not generate
- Custom watermarks are not currently supported
Callback notification URL for task result. If configured, server will actively notify when task status changes.
- For specific message schema, see Callback Protocol
Custom task ID defined by user.
- Will not overwrite system-generated task ID, but supports querying task by this ID.
- Must be unique within a single user account.
Scenario invocation examples
The following is an example of scene code. For more effects and prompt words, please refer to: Kling Omni Model Example
Image/Element Reference
- Image/Element Reference:Supports reference images/elements, including characters, items, backgrounds, and more, to generate with more creativity and consistency.
Transformation
- Input-based Modification: Supports Inpainting/outpainting, or changing shot compositions or angles. It also supports localized or full-scale adjustments, such as modifying/swapping subjects, backgrounds, partial areas, styles, colors, weather, and more.
Video Reference
- Video Reference: Supports using reference video content to generate previous or next shots within the same context or set. It can also reference video actions or camera movements for generation.
Start & End Frames
Text To Video
Single shot and multiple shot
Image To Video with Multiple Shot
Text To Video with Multiple Shot
Text To Video with Single Shot
FAQ
1、 Video Duration Support
- Text-to-video and image-to-video (NOT including first/last frame): Optional duration of 3~10s.
- When video input is provided (video_list is not empty) and video editing functionality (type = base) is used: Duration cannot be specified and will align with the input video.
- Other cases (when no video is provided but an image + subject is used for video generation, or when a video is provided with video type = feature): Optional duration of 3-10s.
2、 How to Extend a Video?
- This can be achieved via “video reference”.
- By inputting a video and using a prompt to direct the model to “generate the next shot” or “generate the previous shot”.
3、 Aspect Ratio Support
- Not Supported: Instruction-based transformation (video editing), image-to-video (not including first/last frame).
- Supported: Text-to-video, image/subject reference, video reference (other scenarios), video reference (generating next/previous shot).
Query Task (Single)
Query the status and result of a single task by ID.
Request Header
Data Exchange Format
Authentication information, refer to API authentication
Path Parameters
Task ID for video generation task.
Request path parameter, fill the value directly in the request path. Choose either task_id OR external_task_id for querying.
Custom Task ID for video generation task.
The external_task_id provided when creating the task. Choose either task_id OR external_task_id for querying.
Query Task (List)
Query a list of tasks with pagination.
Request Header
Data Exchange Format
Authentication information, refer to API authentication
Query Parameters
Page number
Value range: [1, 1000]
Data volume per page
Value range: [1, 500]