39. Video Decode and Encode Operations

Vulkan implementations can expose video decode and encode engines, which are independent from the graphics and compute engines. Video decode and encode is performed by recording video operations and submitting them to video decode and encode queues. Vulkan provides core support for video decode and encoce and can support a variety of video codecs through individual extensions built on the core video support.

The subsections below detail the fundamental components and operation of Vulkan video.

39.1. Technical Terminology and Semantics

39.1.1. Video Picture Resources

Video Picture Resources contain format information, can be multidimensional and may have associated metadata. The metadata can include implementation-private details required for the decode or encode operations and application managed color-space related information.

In Vulkan, a Video Picture Resource is represented by a VkImage. The VkImageView, representing the VkImage, is used with the decode operations as Output and Decoded Picture Buffer (Dpb), and with the encode operation as Input and Reconstructed Video Picture Resource.

39.1.2. Reference Picture

Video Reference Picture is a Video Picture Resource that can be used in the video decode or encode process to provide predictions of the values of samples in the subsequently decoded or encoded pictures.

39.1.3. Decoded Output Picture

The pixels resulting from the video decoding process are stored in a Decoded Output Picture, represented by a VkImageView. This can be shared with the Encoder Reconstructed or Decoder Dpb Video Picture Resources. It can also be used as an input for Video Encode, Graphics, Compute processing, or WSI presentation.

39.1.4. Input Picture to Encode

The primary source of input pixels for the video encoding process is the Input Picture to Encode, represented by a VkImageView. This can be shared with the Encoder Reconstructed or Decoder Dpb Video Picture Resources. It can be a direct target of Video Decode, Graphics, Compute processing, or WSI presentation.

39.1.5. Decoded Picture Buffer (Dpb)

Previously decoded pictures are used by video codecs to provide predictions of the values of samples in the subsequently decoded pictures. At the decoder, such Video Picture Resources are stored in a Decoded Picture Buffer (Dpb) as an indexed set of Reference Pictures.

39.1.6. Reconstructed Pictures

An integral part of the video decoding pipeline is the reconstruction of pictures from the compressed stream. A similar stage exists in the video encoding pipeline as well. Such reconstructed pictures may be used as Reference Pictures for subsequently decoded or encoded pictures. The correct use of such Reference Pictures is driven by the video compression standard, the implementation, and the application-specific use cases.

This specification refers to the collection of the Decoded Picture Buffer and Reconstructed Pictures as Decoded Picture Buffer (Dpb) Set, or only, Dpb.

39.1.7. Decoded Picture Buffer (Dpb) Slot

Decoded Picture Buffer (Dpb) Slot represents a single or multi-layer indexed Reference Picture’s entry within the Video Session’s Dpb Set. A valid Dpb slot index starts from zero and goes up to the maximum of N - 1, where N is the number of Reference Picture entries requested for a Video Session.

39.1.8. Reference Picture Metadata

The opaque Dpb Slot state managed by the implementation may contain Reference Picture Metadata, present when the picture resource associated with the Dpb Slot is used as a reference picture in one or more video decode or encode operations.

An implementation or application may have other Picture Metadata related to the Video Picture Resource or the Dpb slot, but such data is outside the scope of this specification.

Note:

The video decode or encode implementation does not maintain internal references to the Reference Pictures, beyond the Reference Picture Metadata. It is the responsibility of the Vulkan Application to create, manage, and destroy, as well as to provide those Video Picture Resources, when required, during the decoding or encoding process.

39.1.9. Color Space Metadata

Color Space Metadata is the additional static or dynamic state associated with a Video Picture Resource specifying the color volume (the color primaries, white point, and luminance range) of the display that was used in mastering the video content. The use of Color Space Metadata is outside the scope of the current version of the video core specification.

39.2. Introduction

This chapter discusses extensions supporting Video Decode or Encode operations. Video Decode and Encode operations are supported by queues with an advertised queue capability of VK_QUEUE_VIDEO_DECODE_BIT_KHR and VK_QUEUE_VIDEO_ENCODE_BIT_KHR, respectively. Video Decode or Encode queue operation support allows for Vulkan applications to cooperate closely with other graphics or compute operations seamlessly and efficiently, therefore improving the overall application performance.

39.2.1. Video Decode Queue

VK_KHR_video_decode_queue adds a video decode queue type bit VK_QUEUE_VIDEO_DECODE_BIT_KHR to VkQueueFlagBits. As in the case of other queue types, an application must use vkGetPhysicalDeviceQueueFamilyProperties to query whether the physical device has support for the Video Decode Queue. When the implementation reports the VK_QUEUE_VIDEO_DECODE_BIT_KHR bit for a queue family, it advertises general support for Vulkan queue operations described in Devices and Queues.

39.2.2. Video Encode Queue

VK_KHR_video_encode_queue adds a video encode queue type bit VK_QUEUE_VIDEO_ENCODE_BIT_KHR to VkQueueFlagBits. As in the case of other queue types, an application must use vkGetPhysicalDeviceQueueFamilyProperties to query whether the physical device has support for the Video Encode Queue. When the implementation reports the VK_QUEUE_VIDEO_ENCODE_BIT_KHR bit for a queue family, it advertises general support for Vulkan queue operations described in Devices and Queues.

The rest of the chapter focuses, specifically, on Video Decode and Encode queue operations.

39.2.3. Video Session

Before performing any video decoding or encoding operations, the application must create a Video Session instance, of type VkVideoSessionKHR. A Video Session instance is an immutable object and supports a single compression standard (for example, H.264, H.265, VP9, AV1, etc.). The implementation uses the VkVideoSessionKHR object to maintain the video state for the video decode or video encode operation. A Video Session instance is created specifically:

  • For a particular video compression standard;

  • For video decoding or video encoding;

  • With maximum supported decoded or encoded picture width/height;

  • With the maximum number of supported Dpb or Reconstructed Pictures slots that can be allocated;

  • With the maximum number of Reference Pictures that can be used simultaneously for video decode or encode operations;

  • Codec color and features profile;

  • Color Space format description (not supported with this version of the specification);

VkVideoSessionKHR represents a single video decode or encode stream. For each concurrently used stream, a separate instance of VkVideoSessionKHR is required. After the application has finished with the processing of a stream, it can reuse the Video Session instance for another, provided that the configuration parameters between the two usages are compatible (as determined by the video compression standard in use). Once the VkVideoSessionKHR instance has been created, the video compression standard and profiles, Input / Output / Dpb formats, and the settings like the maximum extent cannot be changed.

The values of the following VkVideoSessionKHR parameters can be updated each frame, subject to the restrictions imposed on parameter updates by the video compression standard in use:

The updated parameters must not exceed the maximum limits specified when creating the VkVideoSessionKHR instance.

39.2.4. Video Session Device Memory Heaps

After creating a Video Session instance, and before the object can be used for any of the decode or encode operations, the application must allocate and bind device memory resources to the Video Session object. An implementation may require one or more device memory heaps of different memory types, as reported by the vkGetVideoSessionMemoryRequirementsKHR function, to be bound with the vkBindVideoSessionMemoryKHR function to the Video Session, For more information about the Video Session Device Memory, please refer to the Binding the Session Object Device Memory section, below.

39.2.5. Video Session Parameters

A lot of codec standards require parameters that are in use for the entire video stream. For example, H.264/AVC and HEVC standards require sequence and picture parameter sets (SPS and PPS) that apply to multiple Video Decode and Encode frames, layers, and sub-layers. Vulkan Video uses Video Session Parameters objects to store such standard parameters. The application creates one or more Video Session Parameters Objects against a Video Session, with a set of common Video Parameters that are required for the processing of the video content. During the object creation, the implementation stores the parameters to the created instance. During command buffer recording, it is the responsibility of the application to provide the Video Session Parameters object containing the parameters that are necessary for the processing the portion of the stream under consideration.

39.2.6. Video Picture Subresources

For Video Picture Resources, an application has the option to use single or multi-layer images for image views. The layer to be used during decode or encode operations can be specified when the image view is being created with the VkImageSubresourceRange::baseArrayLayer parameter, and/or within the resource binding operations in command buffer by using the VkVideoPictureResourceKHR::baseArrayLayer parameter.

Note:

Both Video Decode and Encode operations only work with a single layer at the time.

The Image views representing the Input / Output / Dpb Video Picture Resources could have been created with sizes bigger than the coded size that is used with Video Decode and Encode operations. This allows for the same Video Picture Resources to be reused when there is a change in the input video content resolution. The effective coded size of the Video Picture Resources used for Video Decode and Encode operations is provided with VkVideoPictureResourceKHR::extent parameter of each resource in use.

Note:

Many codec standards require the coded and Video Picture Resources' sizes to match.

Video Session Dpb and Reconstructed Video Picture Resources

The video compression standard chosen may require the use of Reference Pictures. In Vulkan Video, like any other Video Picture Resources, the Reference Pictures are represented with Image Views.

When an application requires Reference Picture Resources, it creates and then associates image views, representing these resources, with Video Session Dpb or Reconstructed slots while recording the command buffer.

Decoded output pictures may be used as reference pictures in future video decode operations. The same pictures may be used in texture sampling operations or in the (WSI) presentation pipeline. Representing the Dpb’s Video Picture Resources by image views makes it possible to accommodate all these use cases in a "zero-copy" fashion. Also, it provides more fine-grained control of the application over the efficient usage of the Dpb and Reconstructed Device Memory Resources.

Video Session Dpb and Reconstructed Slot Resource Management

Before Video Picture Resources can be used as Reference Picture Resources, Video Session Dpb or Reconstructed Slots must be associated with those resources.

The application allocates a Dpb or Reconstructed Slot and associates it with a Video Picture Resource and then sets up the resource as a target of decode or encode operation. After successfully decoding or encoding a picture with the targeted Dpb or Reconstructed Slot , in addition to the Reference Picture pixel data, the implementation may generate an opaque Reference Picture Metadata for that video session Slot and its associated Video Picture Resource.

Subsequently, one or more Dpb or Reconstructed video session Slots, along with their associated Video Picture Resources, can be used as Reference Picture’s source for the video decode or encode operations.

If Reference Pictures were to be required for decoding or encoding of the video bitstream, the VkVideoSessionCreateInfoKHR::maxReferencePicturesSlotsCount must be set to a value bigger than "zero" when the instance of the Video Session object is created.

Up to VkVideoSessionCreateInfoKHR::maxReferencePicturesSlotsCount slots can be activated with Video Picture Resources for a video session and up to VkVideoSessionCreateInfoKHR::maxReferencePicturesActiveCount active slots can be used as Dpb or Reconstructed Reference Pictures within a single decode or encode operation.

When the implementation is associating Reference Picture Metadata with the Video Picture Resources themselves, such data must be independent of the Video Session to allow for those Video Picture Resources to be shared with other Video Session instances. All of the Video Session-dependent Reference Picture Metadata must only be associated with the Video Session Dpb or Reconstructed Slots.

The application with the help of the implementation is responsible for managing the individual Dpb, or Reconstructed Slots that belong to a single Video Session Dpb set:

The application also manages the mapping between the codec-specific picture IDs and Dpb Slots.

When a Video Picture is decoded and is set as a Reference Picture against a Video Session Dpb slot, or is encoded and a Reconstructed Video Picture Resource is associated with a Video Session Dpb Slot then:

When a Dpb’s Slot is deactivated, or a different Video Picture Resource is used with the Slot, or the content of the Video Picture Resource is modified, the Reference Picture Metadata associated with the Dpb slot gets invalidated by the implementation. Subsequent attempts to use such, invalidated, Dpb Slot as a Reference source would produce undefined results.

Video Session Dpb Slot subresources

Dpb Reference Picture’s coded width and height can change, dynamically, via VkVideoPictureResourceKHR::extent, and the picture parameters from the codec-specific extensions. When a Dpb Slot is activated as a Reference Picture and a decode or encode operation is performed against that slot, the coded extent can be recorded by the implementation to the corresponding Dpb slot’s metadata state. Subsequently, when the Reference Pictures are used with the decoded Output or encoded Input Picture, their coded extent can differ. Decoding or encoding pictures, using picture sizes, different from the previously produced Reference Pictures should be used with care, not to conflict with the codec standard and the implementation’s support for that. It is the responsibility of the application to ensure that valid Dpb Set of Reference Pictures are in use, according to the codec standard.

In addition, the Video Picture Resources extent cannot exceed the VkVideoSessionCreateInfoKHR::maxCodedExtent.

Note:

Coding Standards such as VP9 and AV1 allow for images with different sizes to be used as Reference Pictures. Others, like H.264 and H.265, do not support Reference Pictures with different sizes. Using Reference Pictures with incompatible sizes with such standards would render undefined results.

The application is in control of the allocation and use of the system resource

In Vulkan Video, the application has complete control over how and when system resources are used. The Vulkan Video framework provides the following tools to ensure that device and host memory resources are used in an optimal way:

Using Dpb and Reconstructed Slot’s Associated Resources

Before a Dpb Slot is to become Valid for use with a Reference Picture, it requires memory resources to be bound to it.

Some of the memory resources required for the Dpb Slot, are opaquely managed by the implementation and, internally, allocated from the Session’s Device Memory Heaps. The application provides the image resources of one or more Reference Pictures, in the VkVideoBeginCodingInfoKHR::pReferenceEntries as part of the vkCmdBeginVideoCoding command.

If a Dpb slot was already used with an image view, and a new image view or a NULL handle is used with that Slot, then the Dpb Slot’s state will be invalidated by the implementation. If a Dpb Slot were to be reused with the same image view, the state of the Slot would not change.

Video Session Activating Dpb Slot as a Reference

Before a Dpb Slot is to be used for a Reference Pictures index, it must be activated. The activation of a Dpb Slot is done within the vkCmdDecodeVideoKHR command’s VkVideoDecodeInfoKHR::dpbSlotIndex field for the decode operations, and within the vkCmdEncodeVideoKHR command’s VkVideoEncodeInfoKHR::dpbSlotIndex field for the encode operations.

While activating a Slot for Dpb, it must already have an associated image view, within the VkVideoBeginCodingInfoKHR::pReferenceEntries in the vkCmdBeginVideoCodingKHR command and Device Memory backing of the the image resources must be resident.

When a Dpb Slot were to be activated, the VkVideoDecodeInfoKHR::dpbSlotIndex for decode, or VkVideoDecodeInfoKHR::dpbSlotIndex for encode, must be set to the application’s allocated Dpb Slot’s index. When activating a Dpb Slot, the application will perform a decode or encode operation against its Slot’s index in order to enable its state as a Valid Picture Reference. If a Dpb Slot is activated, but a decode or encode operation is not performed against that Slot’s index, or the decode or encode operation was unsuccessful, then the Dpb Slot would remain in the Invalid Picture Reference state (see below the Dpb Slot States).

By just providing a Video Picture Resources for a Dpb Slot within the VkVideoBeginCodingInfoKHR::pReferenceEntries, and without successfully performing a decode or encode operation against that Slot, the Dpb Slot’s state cannot be changed to Valid Picture Reference. If the Dpb slots were already in Valid Picture Reference, and there is no Video Picture Resources associated with the Dpb Slot for a decode or encode operation, the state Dpb Slot would not change. However, if an application is referring to a valid Dpb Slot in its current decode or encode operations, then a valid image view must be provided for that Slot within VkVideoPictureResourceKHR::imageViewBinding for that decode or encode operation.

Video Session Invalidating Dpb Slot’s Reference State

When a Dpb Slot is invalidated, its state is set to Invalid Picture Reference. Using a Dpb Slot as a Reference Picture index for video decode or encode operations while the Slot is in Invalid Picture Reference state would render undefined results.

Video Session Dpb Slot States

To help understand the valid use of the Video Session Dpb and its resource management, this section aims to explain the different states and correct usage of Dpb Slots.

There are four (4) states that a Dpb slot could be in:

  • Picture Reference Unused;

  • Invalid Picture Reference;

  • Updating Picture Reference;

  • Valid Picture Reference;

The different states are outlined within the Dpb Slot States and Dpb Slot States Flow Diagram below.

All Dpb Slot management operations are performed via the VkVideoDecodeInfoKHR::dpbSlotIndex or VkVideoEncodeInfoKHR::dpbSlotIndex field.

All Dpb resource binding, invalidating, and activating Slot management operations are performed, by the implementation, before the decoding or encoding commands, based on the VkVideoDecodeInfoKHR::dpbSlotIndex or VkVideoEncodeInfoKHR::dpbSlotIndex field and the entries from the VkVideoBeginCodingInfoKHR::pReferenceEntries. The application cannot move a Dpb Slot from a Picture Reference Unused to Updating Picture Reference state, implicitly, within a decode or encode command operation. Such a Dpb Slot must first be transitioned to an Invalid Picture Reference state using VkVideoDecodeInfoKHR::dpbSlotIndex or VkVideoEncodeInfoKHR::dpbSlotIndex, as part of a decode command. For more details, see Video Picture Decode Modes.

When using sparse memory resources, it would be acceptable and valid behavior for the application to unbind the memory while the Dpb Slot is any of the Dpb Slot states, provided the command buffers, in a pending state, do not reference any such Video Picture Resources.

Accessing unbound regions of the sparse memory resources by the decoder or encoder, regardless if those are used as Output, Input, Dpb or Reconstructed Video Picture Resources, would render undefined results. The VkPhysicalDeviceSparseProperties::residencyNonResidentStrict property reported by the implementation does not offer guarantees on the behavior of decode or encode operations when it comes to accessing unbound regions. However, both reads and writes are still considered safe and will not affect other resources or populated regions of the image.

Table 50. Video Session Dpb Slot States
Dpb Slot State Moving to Dpb Slot State Exiting Dpb Slot State Retain Video Picture Resource Memory

Picture Reference Unused

* Bind Device Memory;
* Reset decode or encode state;
* Invalidate, delete or unbind memory of a Picture Reference associated with Reference Dpb Slot

* Activate Reference Dpb Slot → Invalid Picture Reference

Application Controlled

Invalid Picture Reference

* Activate Reference Dpb Slot;
* Unsuccessful video decode or encode operation;

* Start decode or encode operation with an active Reference Dpb Slot target → Updating Picture Reference;
* Updating a Picture Resource outside the decoder or encoder or deleting or removing the memory binding(sparse) → Picture Reference Unused;

Application Controlled

Updating Picture Reference

Start decode or encode operation with an active Reference Dpb Slot target;

Decode or encode operation with an active Reference Dpb Slot target Completed Successfully → Valid Picture Reference;
Unsuccessful video decode or encode operation → Invalid Picture Reference

Yes

Valid Picture Reference

Video decode or encode operation with an active Reference Dpb Slot target Completed Successfully;

* Replace Reference Dpb slot → Invalid Picture Reference;
* Invalidate, delete or unbind memory of a Picture Reference of the Reference Dpb Slot → Picture Reference Unused;

Yes

Figure 27. Dpb Slot States Flow Diagram

39.3. Video Physical Device Capabilities

39.3.1. Supported Video Codec Operations Enumeration

The structure VkVideoQueueFamilyProperties2KHR may be chained to VkQueueFamilyProperties2 when calling vkGetPhysicalDeviceQueueFamilyProperties2 to retrieve the video codec operations supported for the physical device queue family index.

The VkVideoQueueFamilyProperties2KHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoQueueFamilyProperties2KHR {
    VkStructureType                  sType;
    void*                            pNext;
    VkVideoCodecOperationFlagsKHR    videoCodecOperations;
} VkVideoQueueFamilyProperties2KHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • videoCodecOperations is the supported video codec operation(s) as defined in VkVideoCodecOperationFlagBitsKHR.

Valid Usage (Implicit)
  • VUID-VkVideoQueueFamilyProperties2KHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_QUEUE_FAMILY_PROPERTIES_2_KHR

  • VUID-VkVideoQueueFamilyProperties2KHR-videoCodecOperations-parameter
    videoCodecOperations must be a valid combination of VkVideoCodecOperationFlagBitsKHR values

  • VUID-VkVideoQueueFamilyProperties2KHR-videoCodecOperations-requiredbitmask
    videoCodecOperations must not be 0

The codec operations are defined with the VkVideoCodecOperationFlagBitsKHR enum:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCodecOperationFlagBitsKHR {
    VK_VIDEO_CODEC_OPERATION_INVALID_BIT_KHR = 0,
#ifdef VK_ENABLE_BETA_EXTENSIONS
  // Provided by VK_EXT_video_encode_h264
    VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT = 0x00010000,
#endif
#ifdef VK_ENABLE_BETA_EXTENSIONS
  // Provided by VK_EXT_video_decode_h264
    VK_VIDEO_CODEC_OPERATION_DECODE_H264_BIT_EXT = 0x00000001,
#endif
#ifdef VK_ENABLE_BETA_EXTENSIONS
  // Provided by VK_EXT_video_decode_h265
    VK_VIDEO_CODEC_OPERATION_DECODE_H265_BIT_EXT = 0x00000002,
#endif
} VkVideoCodecOperationFlagBitsKHR;

Each decode or encode codec-specific extension extends this enumeration with the appropriate bit corresponding to the extension’s codec operation:

  • VK_VIDEO_CODEC_OPERATION_INVALID_BIT_KHR - No video operations are supported for this queue family.

  • VK_VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_EXT - H.264 video encode operations are supported by this queue family.

  • VK_VIDEO_CODEC_OPERATION_DECODE_H264_BIT_EXT - H.264 video decode operations are supported by this queue family.

  • VK_VIDEO_CODEC_OPERATION_DECODE_H265_BIT_EXT - H.265 video decode operations are supported by this queue family.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCodecOperationFlagsKHR;

VkVideoCodecOperationFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCodecOperationFlagBitsKHR.

39.3.2. Video Profiles

A video profile is defined by VkVideoProfileKHR structure as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoProfileKHR {
    VkStructureType                     sType;
    void*                               pNext;
    VkVideoCodecOperationFlagBitsKHR    videoCodecOperation;
    VkVideoChromaSubsamplingFlagsKHR    chromaSubsampling;
    VkVideoComponentBitDepthFlagsKHR    lumaBitDepth;
    VkVideoComponentBitDepthFlagsKHR    chromaBitDepth;
} VkVideoProfileKHR;
Valid Usage (Implicit)
  • VUID-VkVideoProfileKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_PROFILE_KHR

  • VUID-VkVideoProfileKHR-videoCodecOperation-parameter
    videoCodecOperation must be a valid VkVideoCodecOperationFlagBitsKHR value

  • VUID-VkVideoProfileKHR-chromaSubsampling-parameter
    chromaSubsampling must be a valid combination of VkVideoChromaSubsamplingFlagBitsKHR values

  • VUID-VkVideoProfileKHR-chromaSubsampling-requiredbitmask
    chromaSubsampling must not be 0

  • VUID-VkVideoProfileKHR-lumaBitDepth-parameter
    lumaBitDepth must be a valid combination of VkVideoComponentBitDepthFlagBitsKHR values

  • VUID-VkVideoProfileKHR-lumaBitDepth-requiredbitmask
    lumaBitDepth must not be 0

  • VUID-VkVideoProfileKHR-chromaBitDepth-parameter
    chromaBitDepth must be a valid combination of VkVideoComponentBitDepthFlagBitsKHR values

  • VUID-VkVideoProfileKHR-chromaBitDepth-requiredbitmask
    chromaBitDepth must not be 0

The video format chroma subsampling is defined with the following enums:

// Provided by VK_KHR_video_queue
typedef enum VkVideoChromaSubsamplingFlagBitsKHR {
    VK_VIDEO_CHROMA_SUBSAMPLING_INVALID_BIT_KHR = 0,
    VK_VIDEO_CHROMA_SUBSAMPLING_MONOCHROME_BIT_KHR = 0x00000001,
    VK_VIDEO_CHROMA_SUBSAMPLING_420_BIT_KHR = 0x00000002,
    VK_VIDEO_CHROMA_SUBSAMPLING_422_BIT_KHR = 0x00000004,
    VK_VIDEO_CHROMA_SUBSAMPLING_444_BIT_KHR = 0x00000008,
} VkVideoChromaSubsamplingFlagBitsKHR;
  • VK_VIDEO_CHROMA_SUBSAMPLING_MONOCHROME_BIT_KHR - the format is monochrome.

  • VK_VIDEO_CHROMA_SUBSAMPLING_420_BIT_KHR - the format is 4:2:0 chroma subsampled. The two chroma components are each subsampled at a factor of 2 both horizontally and vertically.

  • VK_VIDEO_CHROMA_SUBSAMPLING_422_BIT_KHR - the format is 4:2:2 chroma subsampled. The two chroma components are sampled at half the sample rate of luma. The horizontal chroma resolution is halved.

  • VK_VIDEO_CHROMA_SUBSAMPLING_444_BIT_KHR - the format is 4:4:4 chroma sampled. Each of the three YCbCr components have the same sample rate, thus there is no chroma subsampling.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoChromaSubsamplingFlagsKHR;

VkVideoChromaSubsamplingFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoChromaSubsamplingFlagBitsKHR.

The video format component bit depth is defined with the following enums:

// Provided by VK_KHR_video_queue
typedef enum VkVideoComponentBitDepthFlagBitsKHR {
    VK_VIDEO_COMPONENT_BIT_DEPTH_INVALID_KHR = 0,
    VK_VIDEO_COMPONENT_BIT_DEPTH_8_BIT_KHR = 0x00000001,
    VK_VIDEO_COMPONENT_BIT_DEPTH_10_BIT_KHR = 0x00000004,
    VK_VIDEO_COMPONENT_BIT_DEPTH_12_BIT_KHR = 0x00000010,
} VkVideoComponentBitDepthFlagBitsKHR;
  • VK_VIDEO_COMPONENT_BIT_DEPTH_8_BIT_KHR - the format component bit depth is 8 bits.

  • VK_VIDEO_COMPONENT_BIT_DEPTH_10_BIT_KHR - the format component bit depth is 10 bits.

  • VK_VIDEO_COMPONENT_BIT_DEPTH_12_BIT_KHR - the format component bit depth is 12 bits.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoComponentBitDepthFlagsKHR;

VkVideoComponentBitDepthFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoComponentBitDepthFlagBitsKHR.

A video profile is provided when querying capabilities or image formats for video using vkGetPhysicalDeviceVideoCapabilitiesKHR or vkGetPhysicalDeviceVideoFormatPropertiesKHR, respectively. A video profile is also provided when creating resources (images, video sessions, etc.) used by video queues. Each instance of VkVideoProfileKHR must chain a codec-operation specific video profile extension structure, corresponding to the codec-operation specified in VkVideoProfileKHR::videoCodecOperation. Additional information is provided in each codec-operation-specific video extension.

39.3.3. Supported Video Decode or Encode Capabilities

To query video decode or encode capabilities for a specific codec, call:

// Provided by VK_KHR_video_queue
VkResult vkGetPhysicalDeviceVideoCapabilitiesKHR(
    VkPhysicalDevice                            physicalDevice,
    const VkVideoProfileKHR*                    pVideoProfile,
    VkVideoCapabilitiesKHR*                     pCapabilities);
  • physicalDevice is the physical device whose video decode or encode capabilities will be queried.

  • pVideoProfile is a pointer to an instance of the VkVideoProfileKHR structure with a chained codec-operation specific video profile structure.

  • pCapabilities is a pointer to an instance of the VkVideoCapabilitiesKHR structure in which the capabilities are returned.

Valid Usage (Implicit)
  • VUID-vkGetPhysicalDeviceVideoCapabilitiesKHR-physicalDevice-parameter
    physicalDevice must be a valid VkPhysicalDevice handle

  • VUID-vkGetPhysicalDeviceVideoCapabilitiesKHR-pVideoProfile-parameter
    pVideoProfile must be a valid pointer to a valid VkVideoProfileKHR structure

  • VUID-vkGetPhysicalDeviceVideoCapabilitiesKHR-pCapabilities-parameter
    pCapabilities must be a valid pointer to a VkVideoCapabilitiesKHR structure

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_EXTENSION_NOT_PRESENT

  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_FEATURE_NOT_PRESENT

  • VK_ERROR_FORMAT_NOT_SUPPORTED

If pVideoProfile and chained codec-operation specific profile is not supported, VK_ERROR_FORMAT_NOT_SUPPORTED is returned.

Otherwise, the implementation will fill pCapabilities with capabilities associated with this video profile.

The VkVideoCapabilitiesKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoCapabilitiesKHR {
    VkStructureType                sType;
    void*                          pNext;
    VkVideoCapabilitiesFlagsKHR    capabilityFlags;
    VkDeviceSize                   minBitstreamBufferOffsetAlignment;
    VkDeviceSize                   minBitstreamBufferSizeAlignment;
    VkExtent2D                     videoPictureExtentGranularity;
    VkExtent2D                     minExtent;
    VkExtent2D                     maxExtent;
    uint32_t                       maxReferencePicturesSlotsCount;
    uint32_t                       maxReferencePicturesActiveCount;
} VkVideoCapabilitiesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • capabilityFlags is the capability flags.

  • minBitstreamBufferOffsetAlignment is the minimum alignment for the input or output bitstream buffer offset.

  • minBitstreamBufferSizeAlignment is the minimum alignment for the input or output bitstream buffer size

  • videoPictureExtentGranularity is the minimum size alignment of the extent with the required padding for the decoded or encoded video images.

  • minExtent is the minimum width and height of the decoded or encoded video.

  • maxExtent is the maximum width and height of the decoded or encoded video.

  • maxReferencePicturesSlotsCount - the maximum number of Dpb slots supported by the implementation for a single video session instance.

  • maxReferencePicturesActiveCount - is the maximum slots that can be used as Reference Pictures with a single decode or encode operation.

Valid Usage (Implicit)

The VkVideoCapabilitiesKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCapabilitiesFlagBitsKHR {
    VK_VIDEO_CAPABILITIES_PROTECTED_CONTENT_BIT_KHR = 0x00000001,
    VK_VIDEO_CAPABILITIES_SEPARATE_REFERENCE_IMAGES_BIT_KHR = 0x00000002,
} VkVideoCapabilitiesFlagBitsKHR;
  • VK_VIDEO_CAPABILITIES_PROTECTED_CONTENT_BIT_KHR - the decode or encode session supports protected content.

  • VK_VIDEO_CAPABILITIES_SEPARATE_REFERENCE_IMAGES_BIT_KHR - the DPB or Reconstructed Video Picture Resources for the video session may be created as a separate VkImage for each DPB picture. If not supported, the DPB must be created as single multi-layered image where each layer represents one of the DPB Video Picture Resources.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCapabilitiesFlagsKHR;

VkVideoCapabilitiesFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCapabilitiesFlagBitsKHR.

39.3.4. Enumeration of Supported Video Output, Input and Dpb Formats

To enumerate the supported output, input and Dpb image formats for a specific codec operation and video profile, call:

// Provided by VK_KHR_video_queue
VkResult vkGetPhysicalDeviceVideoFormatPropertiesKHR(
    VkPhysicalDevice                            physicalDevice,
    const VkPhysicalDeviceVideoFormatInfoKHR*   pVideoFormatInfo,
    uint32_t*                                   pVideoFormatPropertyCount,
    VkVideoFormatPropertiesKHR*                 pVideoFormatProperties);
  • physicalDevice is the physical device being queried.

  • pVideoFormatInfo is a pointer to an instance of VkPhysicalDeviceVideoFormatInfoKHR, the input structure.

  • pVideoFormatPropertyCount is a pointer to an integer related to the number of video format properties available or queried, as described below.

  • pVideoFormatProperties is a pointer to an array of VkVideoFormatPropertiesKHR, where the implementation would populate the supported formats.

If pVideoFormatProperties is NULL, then the number of video format properties supported for the given physicalDevice is returned in pVideoFormatPropertyCount. Otherwise, pVideoFormatPropertyCount must point to a variable set by the user to the number of elements in the pVideoFormatProperties array, and on return the variable is overwritten with the number of values actually written to pVideoFormatProperties. If the value of pVideoFormatPropertyCount is less than the number of video format properties supported, at most pVideoFormatPropertyCount values will be written to pVideoFormatProperties, and VK_INCOMPLETE will be returned instead of VK_SUCCESS, to indicate that not all the available values were returned.

Valid Usage
  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-imageUsage-04844
    The imageUsage enum of VkPhysicalDeviceVideoFormatInfoKHR must contain at least one of the following video image usage bit(s): VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR, VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR, VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR, or VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR.

Note:

For most use cases, only decode or encode related usage flags are going to be specified. For a use case such as transcode, if the image were to be shared between decode and encode session(s), then both decode and encode related usage flags can be set.

Valid Usage (Implicit)
  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-physicalDevice-parameter
    physicalDevice must be a valid VkPhysicalDevice handle

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatInfo-parameter
    pVideoFormatInfo must be a valid pointer to a valid VkPhysicalDeviceVideoFormatInfoKHR structure

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatPropertyCount-parameter
    pVideoFormatPropertyCount must be a valid pointer to a uint32_t value

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatProperties-parameter
    If pVideoFormatProperties is not NULL, pVideoFormatProperties must be a valid pointer to an array of pVideoFormatPropertyCount VkVideoFormatPropertiesKHR structures

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatPropertyCount-arraylength
    If pVideoFormatProperties is not NULL, the value referenced by pVideoFormatPropertyCount must be greater than 0

Return Codes
Success
  • VK_SUCCESS

  • VK_INCOMPLETE

Failure
  • VK_ERROR_EXTENSION_NOT_PRESENT

  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_FORMAT_NOT_SUPPORTED

The VkPhysicalDeviceVideoFormatInfoKHR input structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkPhysicalDeviceVideoFormatInfoKHR {
    VkStructureType              sType;
    const void*                  pNext;
    VkImageUsageFlags            imageUsage;
    const VkVideoProfilesKHR*    pVideoProfiles;
} VkPhysicalDeviceVideoFormatInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • imageUsage is the intended video image usage as specified in sVkImageUsageFlagBits.

  • pVideoProfiles is a pointer to an instance of VkVideoProfilesKHR providing the video profile(s) of video session(s) that will use the image. For most use cases, the image is used by a single video session and a single video profile is provided. For a use case such as transcode, where a decode session output image may be used as encode input for one or more encode sessions, multiple video profiles representing the video sessions that will share the image may be provided.

Valid Usage (Implicit)
  • VUID-VkPhysicalDeviceVideoFormatInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VIDEO_FORMAT_INFO_KHR

  • VUID-VkPhysicalDeviceVideoFormatInfoKHR-pNext-pNext
    pNext must be NULL

The VkVideoProfilesKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoProfilesKHR {
    VkStructureType             sType;
    void*                       pNext;
    uint32_t                    profileCount;
    const VkVideoProfileKHR*    pProfiles;
} VkVideoProfilesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • profileCount is an integer which holds the number of video profiles included in pProfiles.

  • pProfiles is a pointer to an array of VkVideoProfileKHR structures. Each VkVideoProfileKHR structure must chain the corresponding codec-operation specific extension video profile structure.

Valid Usage (Implicit)
  • VUID-VkVideoProfilesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_PROFILES_KHR

  • VUID-VkVideoProfilesKHR-pProfiles-parameter
    pProfiles must be a valid pointer to a valid VkVideoProfileKHR structure

// Provided by VK_KHR_video_queue
typedef struct VkVideoFormatPropertiesKHR {
    VkStructureType    sType;
    void*              pNext;
    VkFormat           format;
} VkVideoFormatPropertiesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • format is one of the supported formats reported by the implementation.

If the pVideoProfiles provided in input structure pVideoFormatInfo are not supported, VK_ERROR_FORMAT_NOT_SUPPORTED is returned. If the implementation requires an opaque video decode or encode DPB, then when querying with the corresponding video decode or encode DPB image usage in imageUsage, only one image format is returned: VK_FORMAT_UNDEFINED.

Valid Usage (Implicit)
  • VUID-VkVideoFormatPropertiesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_FORMAT_PROPERTIES_KHR

  • VUID-VkVideoFormatPropertiesKHR-pNext-pNext
    pNext must be NULL

Before creating an image, the application should obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 or vkGetPhysicalDeviceImageFormatProperties2 using one of the reported pImageFormats and adding VkVideoProfilesKHR to the pNext chain of VkFormatProperties2.

39.4. Video Session Objects

39.4.1. Video Session

Video session objects are abstracted and represented by VkVideoSessionKHR handles:

// Provided by VK_KHR_video_queue
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkVideoSessionKHR)
Creating a Video Session

To create a video session object, call:

// Provided by VK_KHR_video_queue
VkResult vkCreateVideoSessionKHR(
    VkDevice                                    device,
    const VkVideoSessionCreateInfoKHR*          pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkVideoSessionKHR*                          pVideoSession);
  • device is the logical device that creates the decode or encode session object.

  • pCreateInfo is the pointer to VkVideoSessionCreateInfoKHR structure containing parameters specifying the creation of the decode or encode session.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pVideoSession is the pointer for the decode or encode video session object which will be created by this function when it returns VK_SUCCESS

Valid Usage (Implicit)
  • VUID-vkCreateVideoSessionKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkCreateVideoSessionKHR-pCreateInfo-parameter
    pCreateInfo must be a valid pointer to a valid VkVideoSessionCreateInfoKHR structure

  • VUID-vkCreateVideoSessionKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkCreateVideoSessionKHR-pVideoSession-parameter
    pVideoSession must be a valid pointer to a VkVideoSessionKHR handle

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_INCOMPATIBLE_DRIVER

  • VK_ERROR_FEATURE_NOT_PRESENT

The VkVideoSessionCreateInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoSessionCreateInfoKHR {
    VkStructureType                 sType;
    const void*                     pNext;
    uint32_t                        queueFamilyIndex;
    VkVideoSessionCreateFlagsKHR    flags;
    const VkVideoProfileKHR*        pVideoProfile;
    VkFormat                        pictureFormat;
    VkExtent2D                      maxCodedExtent;
    VkFormat                        referencePicturesFormat;
    uint32_t                        maxReferencePicturesSlotsCount;
    uint32_t                        maxReferencePicturesActiveCount;
} VkVideoSessionCreateInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • queueFamilyIndex is the queue family of the created video session.

  • flags is the creation flags defined in VkVideoSessionCreateFlagBitsKHR

  • pVideoProfile is a pointer to an instance of VkVideoProfileKHR.

  • pictureFormat is the format of the image views representing decoded Output or encoded Input pictures.

  • maxCodedExtent is the maximum width and height of the coded pictures that this instance will be able to support.

  • referencePicturesFormat is the format of the Dpb image views representing the Reference Pictures.

  • maxReferencePicturesSlotsCount is the maximum number of Dpb slots that can be activated with associated Video Picture Resources for the created video session.

  • maxReferencePicturesActiveCount is the maximum number of active Dpb slots that can be used as Dpb or Reconstructed Reference Pictures within a single decode or encode operation for the created video session.

Valid Usage
  • VUID-VkVideoSessionCreateInfoKHR-pVideoProfile-04845
    pVideoProfile must be a pointer to a valid VkVideoProfileKHR structure whose pNext chain must include a valid codec-specific profile structure.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesSlotsCount-04846
    If Reference Pictures are required for use with the created video session, the maxReferencePicturesSlotsCount must be set to a value bigger than "zero".

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesSlotsCount-04847
    maxReferencePicturesSlotsCount cannot exceed the implementation reported VkVideoCapabilitiesKHR::maxReferencePicturesSlotsCount.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesActiveCount-04848
    If Reference Pictures are required for use with the created video session, the maxReferencePicturesActiveCount must be set to a value bigger than "zero".

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesActiveCount-04849
    maxReferencePicturesActiveCount cannot exceed the implementation reported VkVideoCapabilitiesKHR::maxReferencePicturesActiveCount.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesActiveCount-04850
    maxReferencePicturesActiveCount cannot exceed the maxReferencePicturesSlotsCount.

  • VUID-VkVideoSessionCreateInfoKHR-maxCodedExtent-04851
    maxCodedExtent cannot be smaller than VkVideoCapabilitiesKHR::minExtent and bigger than VkVideoCapabilitiesKHR::maxExtent.

  • VUID-VkVideoSessionCreateInfoKHR-referencePicturesFormat-04852
    referencePicturesFormat must be one of the supported formats in VkVideoFormatPropertiesKHR format returned by the vkGetPhysicalDeviceVideoFormatPropertiesKHR when the VkPhysicalDeviceVideoFormatInfoKHR imageUsage contains VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR or VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR depending on the session codec operation.

  • VUID-VkVideoSessionCreateInfoKHR-pictureFormat-04853
    pictureFormat for decode output must be one of the supported formats in VkVideoFormatPropertiesKHR format returned by the vkGetPhysicalDeviceVideoFormatPropertiesKHR when the VkPhysicalDeviceVideoFormatInfoKHR imageUsage contains VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR.

  • VUID-VkVideoSessionCreateInfoKHR-pictureFormat-04854
    pictureFormat targeting encode operations must be one of the supported formats in VkVideoFormatPropertiesKHR format returned by the vkGetPhysicalDeviceVideoFormatPropertiesKHR when the VkPhysicalDeviceVideoFormatInfoKHR imageUsage contains VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR.

Valid Usage (Implicit)
  • VUID-VkVideoSessionCreateInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_SESSION_CREATE_INFO_KHR

  • VUID-VkVideoSessionCreateInfoKHR-pNext-pNext
    Each pNext member of any structure (including this one) in the pNext chain must be either NULL or a pointer to a valid instance of VkVideoDecodeH264SessionCreateInfoEXT, VkVideoDecodeH265SessionCreateInfoEXT, or VkVideoEncodeH264SessionCreateInfoEXT

  • VUID-VkVideoSessionCreateInfoKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoSessionCreateInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoSessionCreateFlagBitsKHR values

  • VUID-VkVideoSessionCreateInfoKHR-pVideoProfile-parameter
    pVideoProfile must be a valid pointer to a valid VkVideoProfileKHR structure

  • VUID-VkVideoSessionCreateInfoKHR-pictureFormat-parameter
    pictureFormat must be a valid VkFormat value

  • VUID-VkVideoSessionCreateInfoKHR-referencePicturesFormat-parameter
    referencePicturesFormat must be a valid VkFormat value

The decode or encode session creation flags defined with the following enums:

// Provided by VK_KHR_video_queue
typedef enum VkVideoSessionCreateFlagBitsKHR {
    VK_VIDEO_SESSION_CREATE_DEFAULT_KHR = 0,
    VK_VIDEO_SESSION_CREATE_PROTECTED_CONTENT_BIT_KHR = 0x00000001,
} VkVideoSessionCreateFlagBitsKHR;
  • VK_VIDEO_SESSION_CREATE_PROTECTED_CONTENT_BIT_KHR - create the video session for use with protected video content

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoSessionCreateFlagsKHR;

VkVideoSessionCreateFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoSessionCreateFlagBitsKHR.

39.4.2. Destroying a Video Session

To destroy a decode session object, call:

// Provided by VK_KHR_video_queue
void vkDestroyVideoSessionKHR(
    VkDevice                                    device,
    VkVideoSessionKHR                           videoSession,
    const VkAllocationCallbacks*                pAllocator);
  • device is the device that was used for the creation of the video session.

  • videoSession is the decode or encode video session to be destroyed.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage (Implicit)
  • VUID-vkDestroyVideoSessionKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkDestroyVideoSessionKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-vkDestroyVideoSessionKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkDestroyVideoSessionKHR-videoSession-parent
    videoSession must have been created, allocated, or retrieved from device

39.4.3. Video Session Memory Resource Management

Obtaining the Video Session Object Device Memory Requirements

To get memory requirements for a video session, call:

// Provided by VK_KHR_video_queue
VkResult vkGetVideoSessionMemoryRequirementsKHR(
    VkDevice                                    device,
    VkVideoSessionKHR                           videoSession,
    uint32_t*                                   pVideoSessionMemoryRequirementsCount,
    VkVideoGetMemoryPropertiesKHR*              pVideoSessionMemoryRequirements);
  • device is the logical device that owns the video session.

  • videoSession is the video session to query.

  • pVideoSessionMemoryRequirementsCount is the number of the memory requirements structures of type VkVideoGetMemoryPropertiesKHR that pVideoSessionMemoryRequirements points to.

  • pVideoSessionMemoryRequirements is the pointer to an instance of the VkVideoGetMemoryPropertiesKHR in which the memory heap requirements of the video session are returned - one entry for each memory heap type required by the implementation.

If pVideoSessionMemoryRequirements is NULL, then the number of memory heap types required for the video session is returned in pVideoSessionMemoryRequirementsCount. Otherwise, pVideoSessionMemoryRequirementsCount must point to a variable set by the user with the number of elements in the pVideoSessionMemoryRequirements array, and on return the variable is overwritten with the number of formats actually written to pVideoSessionMemoryRequirements. If pVideoSessionMemoryRequirementsCount is less than the number of memory heap types required for the video session then at most pVideoSessionMemoryRequirementsCount elements will be written to pVideoSessionMemoryRequirements, and VK_INCOMPLETE will be returned, instead of VK_SUCCESS, to indicate that not all required memory heap types were returned.

Valid Usage (Implicit)
  • VUID-vkGetVideoSessionMemoryRequirementsKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-pVideoSessionMemoryRequirementsCount-parameter
    pVideoSessionMemoryRequirementsCount must be a valid pointer to a uint32_t value

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-pVideoSessionMemoryRequirements-parameter
    If pVideoSessionMemoryRequirements is not NULL, pVideoSessionMemoryRequirements must be a valid pointer to an array of pVideoSessionMemoryRequirementsCount VkVideoGetMemoryPropertiesKHR structures

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-pVideoSessionMemoryRequirementsCount-arraylength
    If pVideoSessionMemoryRequirements is not NULL, the value referenced by pVideoSessionMemoryRequirementsCount must be greater than 0

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-videoSession-parent
    videoSession must have been created, allocated, or retrieved from device

Return Codes
Success
  • VK_SUCCESS

  • VK_INCOMPLETE

Failure
  • VK_ERROR_INITIALIZATION_FAILED

The VkVideoGetMemoryPropertiesKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoGetMemoryPropertiesKHR {
    VkStructureType           sType;
    const void*               pNext;
    uint32_t                  memoryBindIndex;
    VkMemoryRequirements2*    pMemoryRequirements;
} VkVideoGetMemoryPropertiesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • memoryBindIndex is the memory bind index of the memory heap type described by the information returned in pMemoryRequirements.

  • pMemoryRequirements returns the requested memory heap requirements entry of type VkMemoryRequirements2 for heap with index memoryBindIndex.

Valid Usage (Implicit)
  • VUID-VkVideoGetMemoryPropertiesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_GET_MEMORY_PROPERTIES_KHR

  • VUID-VkVideoGetMemoryPropertiesKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoGetMemoryPropertiesKHR-pMemoryRequirements-parameter
    pMemoryRequirements must be a valid pointer to a VkMemoryRequirements2 structure

Binding the Session Object Device Memory

To attach memory to a video session object, call:

// Provided by VK_KHR_video_queue
VkResult vkBindVideoSessionMemoryKHR(
    VkDevice                                    device,
    VkVideoSessionKHR                           videoSession,
    uint32_t                                    videoSessionBindMemoryCount,
    const VkVideoBindMemoryKHR*                 pVideoSessionBindMemories);
  • device is the logical device that owns the video session’s memory.

  • videoSession is the video session to be bound with device memory.

  • videoSessionBindMemoryCount is the number of pVideoSessionBindMemories to be bound.

  • pVideoSessionBindMemories is a pointer to the device memory heap entries of type VkVideoBindMemoryKHR which are to be bound to the video session.

Valid Usage (Implicit)
  • VUID-vkBindVideoSessionMemoryKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkBindVideoSessionMemoryKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-vkBindVideoSessionMemoryKHR-pVideoSessionBindMemories-parameter
    pVideoSessionBindMemories must be a valid pointer to an array of videoSessionBindMemoryCount valid VkVideoBindMemoryKHR structures

  • VUID-vkBindVideoSessionMemoryKHR-videoSessionBindMemoryCount-arraylength
    videoSessionBindMemoryCount must be greater than 0

  • VUID-vkBindVideoSessionMemoryKHR-videoSession-parent
    videoSession must have been created, allocated, or retrieved from device

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_INITIALIZATION_FAILED

The VkVideoBindMemoryKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoBindMemoryKHR {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           memoryBindIndex;
    VkDeviceMemory     memory;
    VkDeviceSize       memoryOffset;
    VkDeviceSize       memorySize;
} VkVideoBindMemoryKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • memoryBindIndex is the index of the device memory heap returned in VkVideoGetMemoryPropertiesKHR::memoryBindIndex from vkGetVideoSessionMemoryRequirementsKHR.

  • memory is the allocated device memory to be bound to the video session’s heap with index memoryBindIndex.

  • memoryOffset is the start offset of the region of memory which is to be bound.

  • memorySize is the size in bytes of the region of memory, starting from memoryOffset bytes, to be bound.

Valid Usage (Implicit)
  • VUID-VkVideoBindMemoryKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_BIND_MEMORY_KHR

  • VUID-VkVideoBindMemoryKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoBindMemoryKHR-memory-parameter
    memory must be a valid VkDeviceMemory handle

39.4.4. Video Session Parameters

This specification supports several classes of preprocessed parameters stored in Video Session Parameters objects. The Video Session Parameters objects reduces the number of parameters being dispatched and then processed by the implementation while recording video command buffers.

39.4.5. Creating Video Session Parameters

Video session parameter objects are represented by VkVideoSessionParametersKHR handles:

// Provided by VK_KHR_video_queue
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkVideoSessionParametersKHR)

To create a video session parameters object, call:

// Provided by VK_KHR_video_queue
VkResult vkCreateVideoSessionParametersKHR(
    VkDevice                                    device,
    const VkVideoSessionParametersCreateInfoKHR* pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkVideoSessionParametersKHR*                pVideoSessionParameters);
  • device is the logical device that was used for the creation of the video session object.

  • pCreateInfo is a pointer to VkVideoSessionParametersCreateInfoKHR structure specifying the video session parameters.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pVideoSessionParameters is the pointer for the video session parameters object returned by the function on VK_SUCCESS.

Valid Usage (Implicit)
  • VUID-vkCreateVideoSessionParametersKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkCreateVideoSessionParametersKHR-pCreateInfo-parameter
    pCreateInfo must be a valid pointer to a valid VkVideoSessionParametersCreateInfoKHR structure

  • VUID-vkCreateVideoSessionParametersKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkCreateVideoSessionParametersKHR-pVideoSessionParameters-parameter
    pVideoSessionParameters must be a valid pointer to a VkVideoSessionParametersKHR handle

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_TOO_MANY_OBJECTS

The VkVideoSessionParametersCreateInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoSessionParametersCreateInfoKHR {
    VkStructureType                sType;
    const void*                    pNext;
    VkVideoSessionParametersKHR    videoSessionParametersTemplate;
    VkVideoSessionKHR              videoSession;
} VkVideoSessionParametersCreateInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • videoSessionParametersTemplate is an optional handle to an existing Video Session Parameters object. If this parameter represents a valid handle, then the underlying Video Session Parameters object will be used as a template for constructing the new video session parameters object. All of the template object’s current parameters will be inherited by the new object in such a case. Optionally, some of the template’s parameters can be updated or new parameters added to the newly constructed object via the extension-specific parameters.

  • videoSession is the video session object against which the video session parameters object is going to be created.

Valid Usage
  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSessionParametersTemplate-04855
    If videoSessionParametersTemplate represents a valid handle, it must have been created against videoSession.

Valid Usage (Implicit)
  • VUID-VkVideoSessionParametersCreateInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_SESSION_PARAMETERS_CREATE_INFO_KHR

  • VUID-VkVideoSessionParametersCreateInfoKHR-pNext-pNext
    Each pNext member of any structure (including this one) in the pNext chain must be either NULL or a pointer to a valid instance of VkVideoDecodeH264SessionParametersCreateInfoEXT, VkVideoDecodeH265SessionParametersCreateInfoEXT, or VkVideoEncodeH264SessionParametersCreateInfoEXT

  • VUID-VkVideoSessionParametersCreateInfoKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSessionParametersTemplate-parameter
    videoSessionParametersTemplate must be a valid VkVideoSessionParametersKHR handle

  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSessionParametersTemplate-parent
    videoSessionParametersTemplate must have been created, allocated, or retrieved from videoSession

  • VUID-VkVideoSessionParametersCreateInfoKHR-commonparent
    Both of videoSession, and videoSessionParametersTemplate must have been created, allocated, or retrieved from the same VkDevice

39.4.6. Updating the parameters of the Video Session Parameters object

To update, add, or remove video session parameters state, call:

// Provided by VK_KHR_video_queue
VkResult vkUpdateVideoSessionParametersKHR(
    VkDevice                                    device,
    VkVideoSessionParametersKHR                 videoSessionParameters,
    const VkVideoSessionParametersUpdateInfoKHR* pUpdateInfo);
  • device is the logical device that was used for the creation of the video session object.

  • videoSessionParameters is the video session object that is going to be updated.

  • pUpdateInfo is a pointer to VkVideoSessionParametersUpdateInfoKHR structure containing the session parameters update info.

Valid Usage (Implicit)
  • VUID-vkUpdateVideoSessionParametersKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkUpdateVideoSessionParametersKHR-videoSessionParameters-parameter
    videoSessionParameters must be a valid VkVideoSessionParametersKHR handle

  • VUID-vkUpdateVideoSessionParametersKHR-pUpdateInfo-parameter
    pUpdateInfo must be a valid pointer to a valid VkVideoSessionParametersUpdateInfoKHR structure

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_TOO_MANY_OBJECTS

The VkVideoSessionParametersUpdateInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoSessionParametersUpdateInfoKHR {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           updateSequenceCount;
} VkVideoSessionParametersUpdateInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • updateSequenceCount is the sequence number of the object update with parameters, starting from "1" and incrementing the value by one with each subsequent update.

Valid Usage (Implicit)

39.4.7. Destroying Video Session Parameters

To destroy a video session object, call:

// Provided by VK_KHR_video_queue
void vkDestroyVideoSessionParametersKHR(
    VkDevice                                    device,
    VkVideoSessionParametersKHR                 videoSessionParameters,
    const VkAllocationCallbacks*                pAllocator);
  • device is the device the video session was created with.

  • videoSessionParameters is the video session parameters object to be destroyed.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage (Implicit)
  • VUID-vkDestroyVideoSessionParametersKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkDestroyVideoSessionParametersKHR-videoSessionParameters-parameter
    videoSessionParameters must be a valid VkVideoSessionParametersKHR handle

  • VUID-vkDestroyVideoSessionParametersKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

39.4.8. Video Encode and Decode commands

To start video decode or encode operations, call:

// Provided by VK_KHR_video_queue
void vkCmdBeginVideoCodingKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoBeginCodingInfoKHR*            pBeginInfo);
  • commandBuffer is the command buffer to be used when recording commands for the video decode and encode operations.

  • pBeginInfo is the pointer to an instance of VkVideoBeginCodingInfoKHR.

Valid Usage (Implicit)
  • VUID-vkCmdBeginVideoCodingKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdBeginVideoCodingKHR-pBeginInfo-parameter
    pBeginInfo must be a valid pointer to a valid VkVideoBeginCodingInfoKHR structure

  • VUID-vkCmdBeginVideoCodingKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdBeginVideoCodingKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode, or encode operations

  • VUID-vkCmdBeginVideoCodingKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdBeginVideoCodingKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Primary

Outside

Decode
Encode

The VkVideoBeginCodingInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoBeginCodingInfoKHR {
    VkStructureType                       sType;
    const void*                           pNext;
    VkVideoBeginCodingFlagsKHR            flags;
    VkVideoCodingQualityPresetFlagsKHR    codecQualityPreset;
    VkVideoSessionKHR                     videoSession;
    VkVideoSessionParametersKHR           videoSessionParameters;
    uint32_t                              referenceSlotCount;
    const VkVideoReferenceSlotKHR*        pReferenceSlots;
} VkVideoBeginCodingInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is reserved for future use.

  • codecQualityPreset is the Video Decode or Encode quality preset value defined in VkVideoCodingQualityPresetFlagBitsKHR.

  • videoSession is video session object to be bound for the processing of the video commands.

  • videoSessionParameters is the video session parameters object to be used for the processing of the video commands. If VK_NULL_HANDLE is provided here, then no video session parameters apply to this command buffer context.

  • referenceSlotCount is the number of reference slot entries provided in pReferenceSlots.

  • pReferenceSlots specifies the reference slots used within the video command context between this vkCmdBeginVideoCodingKHR command and the vkCmdEndVideoCodingKHR commmand that follows. Each reference slot provides a slot index and the VkVideoPictureResourceKHR specifying the reference picture resource bound to this slot index. A slot index must not appear more than once in pReferenceSlots in a given command.

Valid Usage
  • VUID-VkVideoBeginCodingInfoKHR-referenceSlotCount-04856
    VkVideoBeginCodingInfoKHR::referenceSlotCount must not exceed the value specified in VkVideoSessionCreateInfoKHR::maxReferencePicturesSlotsCount when creating the video session object that is being provided in videoSession.

  • VUID-VkVideoBeginCodingInfoKHR-videoSessionParameters-04857
    If videoSessionParameters is not VK_NULL_HANDLE, it must have been created using videoSession as a parent object.

Valid Usage (Implicit)
  • VUID-VkVideoBeginCodingInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_BEGIN_CODING_INFO_KHR

  • VUID-VkVideoBeginCodingInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoBeginCodingInfoKHR-flags-zerobitmask
    flags must be 0

  • VUID-VkVideoBeginCodingInfoKHR-codecQualityPreset-parameter
    codecQualityPreset must be a valid combination of VkVideoCodingQualityPresetFlagBitsKHR values

  • VUID-VkVideoBeginCodingInfoKHR-codecQualityPreset-requiredbitmask
    codecQualityPreset must not be 0

  • VUID-VkVideoBeginCodingInfoKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-VkVideoBeginCodingInfoKHR-videoSessionParameters-parameter
    If videoSessionParameters is not VK_NULL_HANDLE, videoSessionParameters must be a valid VkVideoSessionParametersKHR handle

  • VUID-VkVideoBeginCodingInfoKHR-pReferenceSlots-parameter
    pReferenceSlots must be a valid pointer to an array of referenceSlotCount valid VkVideoReferenceSlotKHR structures

  • VUID-VkVideoBeginCodingInfoKHR-referenceSlotCount-arraylength
    referenceSlotCount must be greater than 0

  • VUID-VkVideoBeginCodingInfoKHR-videoSessionParameters-parent
    If videoSessionParameters is a valid handle, it must have been created, allocated, or retrieved from videoSession

  • VUID-VkVideoBeginCodingInfoKHR-commonparent
    Both of videoSession, and videoSessionParameters that are valid handles of non-ignored parameters must have been created, allocated, or retrieved from the same VkDevice

The decode preset types are defined with the following:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCodingQualityPresetFlagBitsKHR {
    VK_VIDEO_CODING_QUALITY_PRESET_DEFAULT_BIT_KHR = 0,
    VK_VIDEO_CODING_QUALITY_PRESET_NORMAL_BIT_KHR = 0x00000001,
    VK_VIDEO_CODING_QUALITY_PRESET_POWER_BIT_KHR = 0x00000002,
    VK_VIDEO_CODING_QUALITY_PRESET_QUALITY_BIT_KHR = 0x00000004,
} VkVideoCodingQualityPresetFlagBitsKHR;
  • VK_VIDEO_CODING_QUALITY_PRESET_NORMAL_BIT_KHR defines normal decode case.

  • VK_VIDEO_CODING_QUALITY_PRESET_POWER_BIT_KHR defines power efficient case.

  • VK_VIDEO_CODING_QUALITY_PRESET_QUALITY_BIT_KHR defines quality focus case.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCodingQualityPresetFlagsKHR;

VkVideoCodingQualityPresetFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCodingQualityPresetFlagBitsKHR.

The VkVideoReferenceSlotKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoReferenceSlotKHR {
    VkStructureType                     sType;
    const void*                         pNext;
    int8_t                              slotIndex;
    const VkVideoPictureResourceKHR*    pPictureResource;
} VkVideoReferenceSlotKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • slotIndex is the unique reference slot index used for the encode or decode operation.

  • pPictureResource is a pointer to an instance of the VkVideoPictureResourceKHR structure describing the picture resource bound to this slot index.

Valid Usage (Implicit)
  • VUID-VkVideoReferenceSlotKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_REFERENCE_SLOT_KHR

  • VUID-VkVideoReferenceSlotKHR-pNext-pNext
    Each pNext member of any structure (including this one) in the pNext chain must be either NULL or a pointer to a valid instance of VkVideoDecodeH264DpbSlotInfoEXT or VkVideoDecodeH265DpbSlotInfoEXT

  • VUID-VkVideoReferenceSlotKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoReferenceSlotKHR-pPictureResource-parameter
    pPictureResource must be a valid pointer to a valid VkVideoPictureResourceKHR structure

The VkVideoPictureResourceKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoPictureResourceKHR {
    VkStructureType    sType;
    const void*        pNext;
    VkOffset2D         codedOffset;
    VkExtent2D         codedExtent;
    uint32_t           baseArrayLayer;
    VkImageView        imageViewBinding;
} VkVideoPictureResourceKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • codedOffset is the offset to be used for the picture resource.

  • codedExtent is the extent to be used for the picture resource.

  • baseArrayLayer is the first array layer to be accessed for the Decode or Encode Operations.

  • imageViewBinding is the image view representing this picture resource.

Valid Usage (Implicit)
  • VUID-VkVideoPictureResourceKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_PICTURE_RESOURCE_KHR

  • VUID-VkVideoPictureResourceKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoPictureResourceKHR-imageViewBinding-parameter
    imageViewBinding must be a valid VkImageView handle

39.4.9. End of the Video Session

To end video decode or encode operations, call:

// Provided by VK_KHR_video_queue
void vkCmdEndVideoCodingKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoEndCodingInfoKHR*              pEndCodingInfo);
  • commandBuffer is the command buffer to be filled by this function.

  • pEndCodingInfo is a pointer to an instance of the VkVideoEndCodingInfoKHR structure.

Valid Usage (Implicit)
  • VUID-vkCmdEndVideoCodingKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdEndVideoCodingKHR-pEndCodingInfo-parameter
    pEndCodingInfo must be a valid pointer to a valid VkVideoEndCodingInfoKHR structure

  • VUID-vkCmdEndVideoCodingKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdEndVideoCodingKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode, or encode operations

  • VUID-vkCmdEndVideoCodingKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdEndVideoCodingKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Primary

Outside

Decode
Encode

The VkVideoEndCodingInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoEndCodingInfoKHR {
    VkStructureType             sType;
    const void*                 pNext;
    VkVideoEndCodingFlagsKHR    flags;
} VkVideoEndCodingInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is reserved for future use.

Valid Usage (Implicit)
  • VUID-VkVideoEndCodingInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_END_CODING_INFO_KHR

  • VUID-VkVideoEndCodingInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoEndCodingInfoKHR-flags-zerobitmask
    flags must be 0

39.4.10. Video Session Control Command

To apply dynamic controls to video decode or video operations, call:

// Provided by VK_KHR_video_queue
void vkCmdControlVideoCodingKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoCodingControlInfoKHR*          pCodingControlInfo);
  • commandBuffer is the command buffer to be filled by this function for setting encode rate control parameters.

  • pCodingControlInfo is a pointer to VkVideoCodingControlInfoKHR.

Valid Usage (Implicit)
  • VUID-vkCmdControlVideoCodingKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdControlVideoCodingKHR-pCodingControlInfo-parameter
    pCodingControlInfo must be a valid pointer to a valid VkVideoCodingControlInfoKHR structure

  • VUID-vkCmdControlVideoCodingKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdControlVideoCodingKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode, or encode operations

  • VUID-vkCmdControlVideoCodingKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdControlVideoCodingKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types Pipeline Type

Primary

Outside

Decode
Encode

The VkVideoCodingControlInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoCodingControlInfoKHR {
    VkStructureType                 sType;
    const void*                     pNext;
    VkVideoCodingControlFlagsKHR    flags;
} VkVideoCodingControlInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

Valid Usage (Implicit)
  • VUID-VkVideoCodingControlInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_CODING_CONTROL_INFO_KHR

  • VUID-VkVideoCodingControlInfoKHR-pNext-pNext
    pNext must be NULL or a pointer to a valid instance of VkVideoEncodeRateControlInfoKHR

  • VUID-VkVideoCodingControlInfoKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoCodingControlInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoCodingControlFlagBitsKHR values

The vkCmdControlVideoCodingKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCodingControlFlagBitsKHR {
    VK_VIDEO_CODING_CONTROL_DEFAULT_KHR = 0,
    VK_VIDEO_CODING_CONTROL_RESET_BIT_KHR = 0x00000001,
} VkVideoCodingControlFlagBitsKHR;
  • VK_VIDEO_CODING_CONTROL_RESET_BIT_KHR indicates a request for the bound video session context to be reset to init state.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCodingControlFlagsKHR;

VkVideoCodingControlFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCodingControlFlagBitsKHR.

39.5. Video Decode Operations

Before the application can start recording Vulkan command buffers for the Video Decode Operations, it must do the following, beforehand:

  1. Ensure that the implementation can decode the Video Content by querying the supported codec operations and profiles using vkGetPhysicalDeviceQueueFamilyProperties2.

  2. By using vkGetPhysicalDeviceVideoFormatPropertiesKHR and providing one or more video profiles, choose the Vulkan formats supported by the implementation. The formats for output and reference pictures must be queried and chosen separately. Refer to the section on enumeration of supported video formats.

  3. Before creating an image to be used as a video picture resource, obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 and vkGetPhysicalDeviceImageFormatProperties2 using one of the reported formats and adding VkVideoProfilesKHR to the pNext chain of VkFormatProperties2. When querying the parameters with vkGetPhysicalDeviceImageFormatProperties2 for images targeting decoded output and reference (Dpb) pictures, the VkPhysicalDeviceImageFormatInfo2::usage field should contain VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR and VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR, respectively.

  4. Create none, some, or all of the required images for the decoded output and reference pictures. More Video Picture Resources can be created at some later point if needed while processing the decoded content. Also, if the decoded picture size is expected to change, the images can be created based on the maximum decoded content size required.

  5. Create the video session to be used for video decode operations. Before creating the Decode Video Session, the decode capabilities should be queried with vkGetPhysicalDeviceVideoCapabilitiesKHR to obtain the limits of the parameters allowed by the implementation for a particular codec profile.

  6. Bind memory resources with the decode video session by calling vkBindVideoSessionMemoryKHR. The video session cannot be used until memory resources are allocated and bound to it. In order to determine the required memory sizes and heap types of the device memory allocations, vkGetVideoSessionMemoryRequirementsKHR should be called.

  7. Create one or more Session Parameter objects for use across command buffer recording operations, if required by the codec extension in use. These objects must be created against a video session with the parameters required by the codec. Each Session Parameter object created is a child object of the associated Session object and cannot be bound in the command buffer with any other Session Object.

The recording of Video Decode Commands against a Vulkan command buffer consists of the following sequence:

  1. vkCmdBeginVideoCodingKHR starts the recording of one or more Video Decode operations in the command buffer. For each Video Decode Command operation, a Video Session must be bound to the command buffer within this command. This command establishes a Vulkan Video Decode Context that consists of the bound Video Session Object, Session Parameters Object, and the required Video Picture Resources. The established Video Decode Context is in effect until the vkCmdEndVideoCodingKHR command is recorded. If more Video Decode operations are to be required after the vkCmdEndVideoCodingKHR command, another Video Decode Context can be started with the vkCmdBeginVideoCodingKHR command.

  2. vkCmdDecodeVideoKHR specifies one or more compressed data buffers to be decoded. The VkVideoDecodeInfoKHR parameters, and the codec extension structures chained to this, specify the details of the decode operation.

  3. vkCmdControlVideoCodingKHR records operations against the decoded data, decoding device, or the Video Session state.

  4. vkCmdEndVideoCodingKHR signals the end of the recording of the Vulkan Video Decode Context, as established by vkCmdBeginVideoCodingKHR.

In addition to the above, the following commands can be recorded between vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR:

  • Query operations

  • Global Memory Barriers

  • Buffer Memory Barriers

  • Image Memory Barriers (these must be used to transition the Video Picture Resources to the proper VK_IMAGE_LAYOUT_VIDEO_DECODE_DPB_KHR and VK_IMAGE_LAYOUT_VIDEO_DECODE_DST_KHR layouts).

  • Pipeline Barriers

  • Events

  • Timestamps

  • Device Groups (device mask)

The following Video Decode related commands must be recorded outside the Vulkan Video Decode Context established with the vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR commands:

  • Sparse Memory Binding

  • Copy Commands

  • Clear Commands

39.5.1. Video Picture Decode Modes

There are a few ways that the vkCmdDecodeVideoKHR can be configured for the Video Picture Decode Operations, based on:

The most basic Video Picture Decode operation with the vkCmdDecodeVideoKHR command is to output the decoded pixel data without using any Dpb Reference Pictures and without updating any Dpb slot’s indexes.

In this case, the following VkVideoDecodeInfoKHR parameters must be set:

Video Picture Decode operations with the vkCmdDecodeVideoKHR command, requiring one or more Reference Pictures for the predictions of the values of samples for the decoded output picture would require Dpb slots with Valid Picture Reference.

In this case, the following VkVideoDecodeInfoKHR parameters must be set:

After the vkCmdDecodeVideoKHR operation is completed successfully, the VkVideoDecodeInfoKHR::pSetupReferenceSlot->pPictureResource->imageViewBinding pixel data will be updated with the decoded content. The operation will not update any Dpb slot with Reference Pictures data. However, any Dpb slot activation, invalidation, or deactivation operations requested via VkVideoDecodeInfoKHR::pReferenceSlots are still going to be performed.

image/svg+xml0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Dpb Decode dstImageView Bitstream Reference Pictures pReferenceEntries
Figure 28. Decoding a Frame to VkImageView without a slot update for a Reference Picture
Video Picture Decode with a Reference Picture slot update and using optional Reference Pictures

When it is known that the picture to be decoded will be used as a reference picture for subsequent decode operations, one of the available Dpb slots needs to be selected for activation and update operations as part of the vkCmdDecodeVideoKHR command.

Based on whether a decode operation with reference pictures or without reference pictures is required, the vkCmdDecodeVideoKHR should be configured with parameters as described in the previous sections. In addition, one of the available Dpb slots must be selected by the application, activated with resources and then set-up for an update with the decode operation.

In this case, the following VkVideoDecodeInfoKHR parameters must be set:

After the vkCmdDecodeVideoKHR operation has completed successfully, the decoded content will be available in the resource provided for VkVideoDecodeInfoKHR::pSetupReferenceSlot->pPictureResource->imageViewBinding. In addition, this operation will update the selected Dpb slot with Reference Pictures data. Any other Dpb slot activation,invalidation, or deactivation operation requested via the VkVideoDecodeInfoKHR::pReferenceSlots are going to be performed as well.

image/svg+xml0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Dpb Decode dstImageView Bitstream Reference Pictures pReferenceEntries Target Dpb Picture Slot dpbSlotIndex
Figure 29. Decoding a Frame to VkImageView with an update to a Reference Pictures Dpb slot

39.5.2. Video Decode Command Buffer Commands

To decode a frame, call:

// Provided by VK_KHR_video_decode_queue
void vkCmdDecodeVideoKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoDecodeInfoKHR*                 pFrameInfo);
  • commandBuffer is the command buffer to be filled with this function for decode frame command.

  • pFrameInfo is the pointer of VkVideoDecodeInfoKHR.

Valid Usage (Implicit)
  • VUID-vkCmdDecodeVideoKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdDecodeVideoKHR-pFrameInfo-parameter
    pFrameInfo must be a valid pointer to a valid VkVideoDecodeInfoKHR structure

  • VUID-vkCmdDecodeVideoKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdDecodeVideoKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode operations

  • VUID-vkCmdDecodeVideoKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdDecodeVideoKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

The VkVideoDecodeInfoKHR structure is defined as:

// Provided by VK_KHR_video_decode_queue
typedef struct VkVideoDecodeInfoKHR {
    VkStructureType                   sType;
    const void*                       pNext;
    VkVideoDecodeFlagsKHR             flags;
    VkOffset2D                        codedOffset;
    VkExtent2D                        codedExtent;
    VkBuffer                          srcBuffer;
    VkDeviceSize                      srcBufferOffset;
    VkDeviceSize                      srcBufferRange;
    VkVideoPictureResourceKHR         dstPictureResource;
    const VkVideoReferenceSlotKHR*    pSetupReferenceSlot;
    uint32_t                          referenceSlotCount;
    const VkVideoReferenceSlotKHR*    pReferenceSlots;
} VkVideoDecodeInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure. All the codec specific structures related to each frame(picture parameters, quantization matrix, etc.) must be chained here and pass to decode session with the function call vkCmdDecodeVideoKHR.

  • flags are decode flags of type VkVideoDecodeFlagBitsKHR, reserved for future versions of this specification.

  • codedOffset is the codded offset of the decode operations. The purpose of this field is interpreted based on the codec extension. When decoding content in H.264 field mode, the codedOffset specifies the line or picture field’s offset within the image.

  • codedExtent is the codded size of the decode operations.

  • srcBuffer is the source buffer that holds the encoded bitstream.

  • srcBufferOffset is the buffer offset where the valid encoded bitstream starts in srcBuffer. It must meet the alignment requirement minBitstreamBufferOffsetAlignment within VkVideoCapabilitiesKHR queried with the vkGetPhysicalDeviceVideoCapabilitiesKHR function.

  • srcBufferRange is the size of the srcBuffer with valid encoded bitstream, starting from srcBufferOffset. It must meet the alignment requirement minBitstreamBufferSizeAlignment within VkVideoCapabilitiesKHR queried with the vkGetPhysicalDeviceVideoCapabilitiesKHR function.

  • dstPictureResource is the destination Decoded Output Picture Resource.

  • pSetupReferenceSlot is an optional parameter used for generating a Dpb reference slot and Picture Resource. VkVideoReferenceSlotKHR::slotIndex specifies the slot index number to use as a target for producing the Dpb data. slotIndex must reference a valid entry as specified in VkVideoBeginCodingInfoKHR via the pReferenceEntries within the vkCmdBeginVideoCodingKHR command that established the Vulkan Video Decode Context for this command.

  • referenceSlotCount is the number of the Dpb Reference Pictures that will be used when this decoding operation is executing.

  • pReferenceSlots is a list of the Dpb Reference pictures that will be used when this decoding operation is executing.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_INFO_KHR

  • VUID-VkVideoDecodeInfoKHR-pNext-pNext
    Each pNext member of any structure (including this one) in the pNext chain must be either NULL or a pointer to a valid instance of VkVideoDecodeH264PictureInfoEXT or VkVideoDecodeH265PictureInfoEXT

  • VUID-VkVideoDecodeInfoKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoDecodeInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoDecodeFlagBitsKHR values

  • VUID-VkVideoDecodeInfoKHR-srcBuffer-parameter
    srcBuffer must be a valid VkBuffer handle

  • VUID-VkVideoDecodeInfoKHR-dstPictureResource-parameter
    dstPictureResource must be a valid VkVideoPictureResourceKHR structure

  • VUID-VkVideoDecodeInfoKHR-pSetupReferenceSlot-parameter
    pSetupReferenceSlot must be a valid pointer to a valid VkVideoReferenceSlotKHR structure

  • VUID-VkVideoDecodeInfoKHR-pReferenceSlots-parameter
    pReferenceSlots must be a valid pointer to an array of referenceSlotCount valid VkVideoReferenceSlotKHR structures

  • VUID-VkVideoDecodeInfoKHR-referenceSlotCount-arraylength
    referenceSlotCount must be greater than 0

The vkCmdDecodeVideoKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_decode_queue
typedef enum VkVideoDecodeFlagBitsKHR {
    VK_VIDEO_DECODE_DEFAULT_KHR = 0,
    VK_VIDEO_DECODE_RESERVED_0_BIT_KHR = 0x00000001,
} VkVideoDecodeFlagBitsKHR;
  • VK_VIDEO_DECODE_RESERVED_0_BIT_KHR The current version of the specification has reserved this value for future use.

// Provided by VK_KHR_video_decode_queue
typedef VkFlags VkVideoDecodeFlagsKHR;

VkVideoDecodeFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoDecodeFlagBitsKHR.

39.6. Video Decode of AVC (ITU-T H.264)

This extension adds H.264 codec specific structures needed for decode session to execute decode jobs which include H.264 sequence header, picture parameter header and quantization matrix etc. Unless otherwise noted, all references to the H.264 specification are to the 2010 edition published by the ITU-T, dated March 2010. This specification is available at http://www.itu.int/rec/T-REC-H.264.

39.6.1. H.264 decode profile

A H.264 decode profile is specified using VkVideoDecodeH264ProfileEXT chained to VkVideoProfileKHR when the codec-operation in VkVideoProfileKHR is VK_VIDEO_CODEC_OPERATION_DECODE_H264_BIT_EXT.

The VkVideoDecodeH264ProfileEXT structure is defined as:

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264ProfileEXT {
    VkStructureType                         sType;
    const void*                             pNext;
    StdVideoH264ProfileIdc                  stdProfileIdc;
    VkVideoDecodeH264FieldLayoutFlagsEXT    fieldLayout;
} VkVideoDecodeH264ProfileEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • stdProfileIdc is the H.264 codec profile IDC defined in StdVideoH264ProfileIdc for enumeration.

  • fieldLayout is the field layout for interlaced content as defined in VkVideoDecodeH264FieldLayoutFlagBitsEXT for enumeration.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264ProfileEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_PROFILE_EXT

  • VUID-VkVideoDecodeH264ProfileEXT-fieldLayout-parameter
    fieldLayout must be a valid combination of VkVideoDecodeH264FieldLayoutFlagBitsEXT values

  • VUID-VkVideoDecodeH264ProfileEXT-fieldLayout-requiredbitmask
    fieldLayout must not be 0

The H.264 decode interlaced field layout flags are defined with the following enum:

// Provided by VK_EXT_video_decode_h264
typedef enum VkVideoDecodeH264FieldLayoutFlagBitsEXT {
    VK_VIDEO_DECODE_H264_PROGRESSIVE_PICTURES_ONLY_EXT = 0,
    VK_VIDEO_DECODE_H264_FIELD_LAYOUT_LINE_INTERLACED_PLANE_BIT_EXT = 0x00000001,
    VK_VIDEO_DECODE_H264_FIELD_LAYOUT_SEPARATE_INTERLACED_PLANE_BIT_EXT = 0x00000002,
} VkVideoDecodeH264FieldLayoutFlagBitsEXT;
  • VK_VIDEO_DECODE_H264_FIELD_LAYOUT_LINE_INTERLACED_PLANE_BIT_EXT - Interlaced content is supported where each field is decoded within the even/odd line of a picture resource.

  • VK_VIDEO_DECODE_H264_FIELD_LAYOUT_SEPARATE_INTERLACED_PLANE_BIT_EXT

    • Interlaced content is supported where each field is decoded within a separate picture plane or resource.

39.6.2. Selecting a H.264 decode profile

When using vkGetPhysicalDeviceVideoCapabilitiesKHR to query the capabilities for the input pVideoProfile with videoCodecOperation specified as VK_VIDEO_CODEC_OPERATION_DECODE_H264_BIT_EXT, the instance of VkVideoDecodeH264ProfileEXT structure must be chained to VkVideoProfileKHR to select a H.264 decode profile. If supported, the implementation returns the capabilities associated with the specified H.264 decode profile. The requirement is similar when querying supported image formats using vkGetPhysicalDeviceVideoFormatPropertiesKHR.

A supported H.264 decode profile must be selected when creating a video session by chaining VkVideoDecodeH264ProfileEXT to the VkVideoProfileKHR field of VkVideoSessionCreateInfoKHR.

39.6.3. Capabilities

When using vkGetPhysicalDeviceVideoCapabilitiesKHR to query the capabilities for the input pVideoProfile with videoCodecOperation specified as VK_VIDEO_CODEC_OPERATION_DECODE_H264_BIT_EXT, the instance of VkVideoDecodeH264CapabilitiesEXT structure must be chained to VkVideoCapabilitiesKHR to get this H.264 decode profile specific capabilities.

The VkVideoDecodeH264CapabilitiesEXT structure is defined as:

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264CapabilitiesEXT {
    VkStructureType          sType;
    void*                    pNext;
    uint32_t                 maxLevel;
    VkOffset2D               fieldOffsetGranularity;
    VkExtensionProperties    stdExtensionVersion;
} VkVideoDecodeH264CapabilitiesEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • maxLevel is the maximum H.264 level supported by the device.

  • fieldOffsetGranularity - if Interlaced Video Content is suported, the maximum field offset granularity supported for the picture resource.

  • stdExtensionVersion is the specific H.264 extension name and version supported by this implementation.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264CapabilitiesEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_CAPABILITIES_EXT

39.6.4. Create Information

The instance of VkVideoDecodeH264SessionCreateInfoEXT structure can be chained to VkVideoSessionCreateInfoKHR when the function vkCreateVideoSessionKHR is called to create a video session for H.264 decode.

The VkVideoDecodeH264SessionCreateInfoEXT structure is defined as:

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264SessionCreateInfoEXT {
    VkStructureType                    sType;
    const void*                        pNext;
    VkVideoDecodeH264CreateFlagsEXT    flags;
    const VkExtensionProperties*       pStdExtensionVersion;
} VkVideoDecodeH264SessionCreateInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is reserved for future use.

  • pStdExtensionVersion is the specific H.264 codec extensions defined in StdVideoH264Extensions.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264SessionCreateInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_SESSION_CREATE_INFO_EXT

  • VUID-VkVideoDecodeH264SessionCreateInfoEXT-flags-zerobitmask
    flags must be 0

  • VUID-VkVideoDecodeH264SessionCreateInfoEXT-pStdExtensionVersion-parameter
    pStdExtensionVersion must be a valid pointer to a valid VkExtensionProperties structure

39.6.5. Decoder Parameter Sets

To reduce parameter traffic during decoding, the decoder parameter set object supports storing H.264 SPS/PPS parameter sets that may be later referenced during decoding.

An instance of VkVideoDecodeH264SessionParametersCreateInfoEXT holding one H.264 SPS and at least one H.264 PPS paramater set must be chained to VkVideoSessionParametersCreateInfoKHR when calling vkCreateVideoSessionParametersKHR to store these parameter set(s) with the decoder parameter set object for later reference. The provided H.264 SPS/PPS parameters must be within the limits specified during decoder creation for the decoder specified in VkVideoSessionParametersCreateInfoKHR.

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264SessionParametersCreateInfoEXT {
    VkStructureType                                        sType;
    const void*                                            pNext;
    uint32_t                                               maxSpsStdCount;
    uint32_t                                               maxPpsStdCount;
    const VkVideoDecodeH264SessionParametersAddInfoEXT*    pParametersAddInfo;
} VkVideoDecodeH264SessionParametersCreateInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • maxSpsStdCount is the maximum number of SPS parameters that the VkVideoSessionParametersKHR can contain.

  • maxPpsStdCount is the maximum number of PPS parameters that the VkVideoSessionParametersKHR can contain.

  • pParametersAddInfo is a pointer to an optional set of H.264 parameters to add upon object creation.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264SessionParametersCreateInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_SESSION_PARAMETERS_CREATE_INFO_EXT

  • VUID-VkVideoDecodeH264SessionParametersCreateInfoEXT-pParametersAddInfo-parameter
    If pParametersAddInfo is not NULL, pParametersAddInfo must be a valid pointer to a valid VkVideoDecodeH264SessionParametersAddInfoEXT structure

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264SessionParametersAddInfoEXT {
    VkStructureType                            sType;
    const void*                                pNext;
    uint32_t                                   spsStdCount;
    const StdVideoH264SequenceParameterSet*    pSpsStd;
    uint32_t                                   ppsStdCount;
    const StdVideoH264PictureParameterSet*     pPpsStd;
} VkVideoDecodeH264SessionParametersAddInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • spsStdCount is the number of SPS elements in the pSpsStd. Its value must be less than or equal to the value of maxSpsStdCount.

  • pSpsStd is an array of structures representing H.264 sequence parameter sets. Each StdVideoH264SequenceParameterSet entry must have a unique H.264 SPS ID.

  • ppsStdCount is the number of PPS provided in pPpsStd. Its value must be less than or equal to the value of maxPpsStdCount.

  • pPpsStd is an array of structures representing H.264 picture parameter sets. Each StdVideoH264PictureParameterSet entry must have a unique H.264 SPS-PPS ID pair.

Valid Usage
  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-spsStdCount-04822
    The values of spsStdCount and ppsStdCount must be less than or equal to the values of maxSpsStdCount and maxPpsStdCount, respectively.

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-maxSpsStdCount-04823
    When the maxSpsStdCount number of parameters of type StdVideoH264SequenceParameterSet in the Video Session Parameters object is reached, no additional parameters of that type can be added to this object. VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made to add additional data to this object at this point.

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-maxPpsStdCount-04824
    When the maxPpsStdCount number of parameters of type StdVideoH264PictureParameterSet in the Video Session Parameters object is reached, no additional parameters of that type can be added to this object. VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made to add additional data to this object at this point.

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-None-04825
    Each entry to be added must have a unique, to the rest of the parameter array entries and the existing parameters in the Video Session Parameters Object that is being updated, SPS-PPS IDs.

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-None-04826
    Parameter entries that already exist in Video Session Parameters object with a particular SPS-PPS IDs cannot be replaced nor updated.

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-None-04827
    When creating a new object using a Video Session Parameters as a template, the array’s parameters with the same SPS-PPS IDs as the ones from the template take precedence.

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-None-04828
    SPS/PPS parameters must comply with the limits specified in VkVideoSessionCreateInfoKHR during Video Session creation.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_SESSION_PARAMETERS_ADD_INFO_EXT

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-pSpsStd-parameter
    If pSpsStd is not NULL, pSpsStd must be a valid pointer to an array of spsStdCount StdVideoH264SequenceParameterSet values

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-pPpsStd-parameter
    If pPpsStd is not NULL, pPpsStd must be a valid pointer to an array of ppsStdCount StdVideoH264PictureParameterSet values

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-spsStdCount-arraylength
    spsStdCount must be greater than 0

  • VUID-VkVideoDecodeH264SessionParametersAddInfoEXT-ppsStdCount-arraylength
    ppsStdCount must be greater than 0

39.6.6. Picture Decoding

To decode a picture, the structure VkVideoDecodeH264PictureInfoEXT may be chained to VkVideoDecodeInfoKHR when calling vkCmdDecodeVideoKHR.

The structure VkVideoDecodeH264PictureInfoEXT representing a picture decode operation is defined as:

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264PictureInfoEXT {
    VkStructureType                         sType;
    const void*                             pNext;
    const StdVideoDecodeH264PictureInfo*    pStdPictureInfo;
    uint32_t                                slicesCount;
    const uint32_t*                         pSlicesDataOffsets;
} VkVideoDecodeH264PictureInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • pStdPictureInfo is a pointer to the codec standard specific picture information from H.264 spec.

  • slicesCount is the number of slices in this picture.

  • pSlicesDataOffsets is an array of slicesCount offsets indicating the start offset of each slice within the bitstream buffer.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264PictureInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_PICTURE_INFO_EXT

  • VUID-VkVideoDecodeH264PictureInfoEXT-pStdPictureInfo-parameter
    pStdPictureInfo must be a valid pointer to a valid StdVideoDecodeH264PictureInfo value

  • VUID-VkVideoDecodeH264PictureInfoEXT-pSlicesDataOffsets-parameter
    pSlicesDataOffsets must be a valid pointer to an array of slicesCount uint32_t values

  • VUID-VkVideoDecodeH264PictureInfoEXT-slicesCount-arraylength
    slicesCount must be greater than 0

The VkVideoDecodeH264DpbSlotInfoEXT structure correlates a DPB slot index with codec-specific information and is defined as:

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264DpbSlotInfoEXT {
    VkStructureType                           sType;
    const void*                               pNext;
    const StdVideoDecodeH264ReferenceInfo*    pStdReferenceInfo;
} VkVideoDecodeH264DpbSlotInfoEXT;
  • sType is the type of this structure.

  • pStdReferenceInfo is a pointer to the codec standard specific picture reference information from H.264 spec.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264DpbSlotInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_DPB_SLOT_INFO_EXT

  • VUID-VkVideoDecodeH264DpbSlotInfoEXT-pStdReferenceInfo-parameter
    pStdReferenceInfo must be a valid pointer to a valid StdVideoDecodeH264ReferenceInfo value

The VkVideoDecodeH264MvcEXT structure is defined as:

// Provided by VK_EXT_video_decode_h264
typedef struct VkVideoDecodeH264MvcEXT {
    VkStructureType                 sType;
    const void*                     pNext;
    const StdVideoDecodeH264Mvc*    pStdMvc;
} VkVideoDecodeH264MvcEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to an extension specific structure.

  • pStdMvc is a pointer to H.264 codec spec information for MVC.

When the content type is H.264 MVC, an instance of VkVideoDecodeH264MvcEXT must be chained to VkVideoDecodeH264PictureInfoEXT.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH264MvcEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H264_MVC_EXT

  • VUID-VkVideoDecodeH264MvcEXT-pStdMvc-parameter
    pStdMvc must be a valid pointer to a valid StdVideoDecodeH264Mvc value

39.7. Video Decode of HEVC (ITU-T H.265)

This extension adds H.265 codec specific structures needed for decode session to execute decode jobs which include H.265 sequence header, picture parameter header and quantization matrix etc. Unless otherwise noted, all references to the H.265 specification are to the 2013 edition published by the ITU-T, dated April 2013. This specification is available at http://www.itu.int/rec/T-REC-H.265.

39.7.1. H.265 decode profile

A H.265 decode profile is specified using VkVideoDecodeH265ProfileEXT chained to VkVideoProfileKHR when the codec-operation in VkVideoProfileKHR is VK_VIDEO_CODEC_OPERATION_DECODE_H265_BIT_EXT.

The VkVideoDecodeH265ProfileEXT structure is defined as:

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265ProfileEXT {
    VkStructureType           sType;
    const void*               pNext;
    StdVideoH265ProfileIdc    stdProfileIdc;
} VkVideoDecodeH265ProfileEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • stdProfileIdc is the H.265 codec profile IDC defined in StdVideoH265ProfileIdc for enumeration.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265ProfileEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_PROFILE_EXT

39.7.2. Selecting an H.265 Profile

When using vkGetPhysicalDeviceVideoCapabilitiesKHR to query the capabilities for the input pVideoProfile with videoCodecOperation specified as VK_VIDEO_CODEC_OPERATION_DECODE_H265_BIT_EXT, the instance of VkVideoDecodeH265ProfileEXT structure must be chained to VkVideoProfileKHR to select a H.265 decode profile. If supported, the implementation returns the capabilities associated with the specified H.265 decode profile. The requirement is similar when querying supported image formats using vkGetPhysicalDeviceVideoFormatPropertiesKHR.

A supported H.265 decode profile must be selected when creating a video session by chaining VkVideoDecodeH265ProfileEXT to the VkVideoProfileKHR field of VkVideoSessionCreateInfoKHR.

39.7.3. Capabilities

When uses vkGetPhysicalDeviceVideoCapabilitiesKHR to query the capabilities for the parameter videoCodecOperation specified as VK_VIDEO_CODEC_OPERATION_DECODE_H265_BIT_EXT, VkVideoDecodeH265CapabilitiesEXT structure can be chained to VkVideoCapabilitiesKHR to get this H.265 extension specific capabilities.

The VkVideoDecodeH265CapabilitiesEXT structure is defined as:

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265CapabilitiesEXT {
    VkStructureType          sType;
    void*                    pNext;
    uint32_t                 maxLevel;
    VkExtensionProperties    stdExtensionVersion;
} VkVideoDecodeH265CapabilitiesEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • maxLevel is the maximum H.265 level supported by the device.

  • stdExtensionVersion is the specific H.265 extension name and version supported by this implementation.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265CapabilitiesEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_CAPABILITIES_EXT

39.7.4. Create Infomation

The instance of VkVideoDecodeH265SessionCreateInfoEXT structure can be chained to VkVideoSessionCreateInfoKHR when the function vkCreateVideoSessionKHR is called to create a video session for H.265 decode operations.

The VkVideoDecodeH265SessionCreateInfoEXT structure is defined as:

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265SessionCreateInfoEXT {
    VkStructureType                    sType;
    const void*                        pNext;
    VkVideoDecodeH265CreateFlagsEXT    flags;
    const VkExtensionProperties*       pStdExtensionVersion;
} VkVideoDecodeH265SessionCreateInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is reserved for future use.

  • pStdExtensionVersion is the specific H.265 codec extensions defined in StdVideoH265Extensions.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265SessionCreateInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_SESSION_CREATE_INFO_EXT

  • VUID-VkVideoDecodeH265SessionCreateInfoEXT-flags-zerobitmask
    flags must be 0

  • VUID-VkVideoDecodeH265SessionCreateInfoEXT-pStdExtensionVersion-parameter
    pStdExtensionVersion must be a valid pointer to a valid VkExtensionProperties structure

39.7.5. Decoder Parameter Sets

To reduce parameter traffic during decoding, the decoder parameter set object supports storing H.265 SPS/PPS parameter sets that may be later referenced during decoding.

An instance of VkVideoDecodeH265SessionParametersCreateInfoEXT holding one H.265 SPS and at least one H.265 PPS paramater set must be chained to VkVideoSessionParametersCreateInfoKHR when calling vkCreateVideoSessionParametersKHR to store these parameter set(s) with the decoder parameter set object for later reference. The provided H.265 SPS/PPS parameters must be within the limits specified during decoder creation for the decoder specified in VkVideoSessionParametersCreateInfoKHR.

The VkVideoDecodeH265SessionParametersCreateInfoEXT structure is defined as:

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265SessionParametersCreateInfoEXT {
    VkStructureType                                        sType;
    const void*                                            pNext;
    uint32_t                                               maxSpsStdCount;
    uint32_t                                               maxPpsStdCount;
    const VkVideoDecodeH265SessionParametersAddInfoEXT*    pParametersAddInfo;
} VkVideoDecodeH265SessionParametersCreateInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • maxSpsStdCount is the maximum number of SPS parameters that the VkVideoSessionParametersKHR can contain.

  • maxPpsStdCount is the maximum number of PPS parameters that the VkVideoSessionParametersKHR can contain.

  • pParametersAddInfo is a pointer to an optional set of H.265 parameters to add upon object creation.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265SessionParametersCreateInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_SESSION_PARAMETERS_CREATE_INFO_EXT

  • VUID-VkVideoDecodeH265SessionParametersCreateInfoEXT-pParametersAddInfo-parameter
    If pParametersAddInfo is not NULL, pParametersAddInfo must be a valid pointer to a valid VkVideoDecodeH265SessionParametersAddInfoEXT structure

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265SessionParametersAddInfoEXT {
    VkStructureType                            sType;
    const void*                                pNext;
    uint32_t                                   spsStdCount;
    const StdVideoH265SequenceParameterSet*    pSpsStd;
    uint32_t                                   ppsStdCount;
    const StdVideoH265PictureParameterSet*     pPpsStd;
} VkVideoDecodeH265SessionParametersAddInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • spsStdCount is the number of SPS elements in the pSpsStd. Its value must be less than or equal to the value of maxSpsStdCount.

  • pSpsStd is an array of structures representing H.265 sequence parameter sets. Each StdVideoH265SequenceParameterSet entry must have a unique H.265 VPS-SPS ID pair.

  • ppsStdCount is the number of PPS provided in pPpsStd. Its value must be less than or equal to the value of maxPpsStdCount.

  • pPpsStd is an array of structures representing H.265 picture parameter sets. Each StdVideoH265PictureParameterSet entry must have a unique H.265 VPS-SPS-PPS ID tuple.

Valid Usage
  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-vpsStdCount-04829
    The values of vpsStdCount, spsStdCount and ppsStdCount must be less than or equal to the values of maxVpsStdCount, maxSpsStdCount and maxPpsStdCount, respectively.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-maxVpsStdCount-04830
    When the maxVpsStdCount number of parameters of type StdVideoH265VideoParameterSet in the Video Session Parameters object is reached, no additional parameters of that type can be added to the object. VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made to add additional data to this object at this point.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-maxSpsStdCount-04831
    When the maxSpsStdCount number of parameters of type StdVideoH265SequenceParameterSet in the Video Session Parameters object is reached, no additional parameters of that type can be added to the object. VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made to add additional data to this object at this point.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-maxPpsStdCount-04832
    When the maxPpsStdCount number of parameters of type StdVideoH265PictureParameterSet in the Video Session Parameters object is reached, no additional parameters of that type can be added to the object. VK_ERROR_TOO_MANY_OBJECTS will be returned if an attempt is made to add additional data to this object at this point.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-None-04833
    Each entry to be added must have a unique, to the rest of the parameter array entries and the existing parameters in the Video Session Parameters Object that is being updated, VPS-SPS-PPS IDs.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-None-04834
    Parameter entries that already exist in Video Session Parameters object with a particular VPS-SPS-PPS IDs cannot be replaced nor updated.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-None-04835
    When creating a new object using a Video Session Parameters as a template, the array’s parameters with the same VPS-SPS-PPS IDs as the ones from the template take precedence.

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-None-04836
    VPS/SPS/PPS parameters must comply with the limits specified in VkVideoSessionCreateInfoKHR during Video Session creation.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_SESSION_PARAMETERS_ADD_INFO_EXT

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-pSpsStd-parameter
    If pSpsStd is not NULL, pSpsStd must be a valid pointer to an array of spsStdCount StdVideoH265SequenceParameterSet values

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-pPpsStd-parameter
    If pPpsStd is not NULL, pPpsStd must be a valid pointer to an array of ppsStdCount StdVideoH265PictureParameterSet values

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-spsStdCount-arraylength
    spsStdCount must be greater than 0

  • VUID-VkVideoDecodeH265SessionParametersAddInfoEXT-ppsStdCount-arraylength
    ppsStdCount must be greater than 0

39.7.6. Picture Parameters

The VkVideoDecodeH265PictureInfoEXT structure is defined as:

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265PictureInfoEXT {
    VkStructureType                   sType;
    const void*                       pNext;
    StdVideoDecodeH265PictureInfo*    pStdPictureInfo;
    uint32_t                          slicesCount;
    const uint32_t*                   pSlicesDataOffsets;
} VkVideoDecodeH265PictureInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • pStdPictureInfo is the codec standard specific structure from H.265 spec.

  • slicesCount is the number of slices in this picture.

  • pSlicesDataOffsets is an array of slicesCount offsets indicating the start offset of each slice within the bitstream buffer.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265PictureInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_PICTURE_INFO_EXT

  • VUID-VkVideoDecodeH265PictureInfoEXT-pStdPictureInfo-parameter
    pStdPictureInfo must be a valid pointer to a StdVideoDecodeH265PictureInfo value

  • VUID-VkVideoDecodeH265PictureInfoEXT-pSlicesDataOffsets-parameter
    pSlicesDataOffsets must be a valid pointer to an array of slicesCount uint32_t values

  • VUID-VkVideoDecodeH265PictureInfoEXT-slicesCount-arraylength
    slicesCount must be greater than 0

The VkVideoDecodeH265DpbSlotInfoEXT structure is defined as:

// Provided by VK_EXT_video_decode_h265
typedef struct VkVideoDecodeH265DpbSlotInfoEXT {
    VkStructureType                           sType;
    const void*                               pNext;
    const StdVideoDecodeH265ReferenceInfo*    pStdReferenceInfo;
} VkVideoDecodeH265DpbSlotInfoEXT;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • pStdReferenceInfo is a pointer to the codec standard specific picture reference information from H.264 spec.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeH265DpbSlotInfoEXT-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_H265_DPB_SLOT_INFO_EXT

  • VUID-VkVideoDecodeH265DpbSlotInfoEXT-pStdReferenceInfo-parameter
    pStdReferenceInfo must be a valid pointer to a valid StdVideoDecodeH265ReferenceInfo value

39.8. Video Encode Operations

Before the application can start recording Vulkan command buffers for the Video Encode Operations, it must do the following, beforehand:

  1. Ensure that the implementation can encode the Video Content by querying the supported codec operations and profiles using vkGetPhysicalDeviceQueueFamilyProperties2.

  2. By using vkGetPhysicalDeviceVideoFormatPropertiesKHR and providing one or more video profiles, choose the Vulkan formats supported by the implementation. The formats for input and reference pictures must be queried and chosen separately. Refer to the section on enumeration of supported video formats.

  3. Before creating an image to be used as a video picture resource, obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 and vkGetPhysicalDeviceImageFormatProperties2 using one of the reported formats and adding VkVideoProfilesKHR to the pNext chain of VkFormatProperties2. When querying the parameters with vkGetPhysicalDeviceImageFormatProperties2 for images targeting input and reference (Dpb) pictures, the VkPhysicalDeviceImageFormatInfo2::usage field should contain VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR and VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR, respectively.

  4. Create none, some, or all of the required images for the input and reference pictures. More Video Picture Resources can be created at some later point if needed while processing the content to be encoded. Also, if the size of the picture to be encoded is expected to change, the images can be created based on the maximum expected content size.

  5. Create the video session to be used for video encode operations. Before creating the Encode Video Session, the encode capabilities should be queried with vkGetPhysicalDeviceVideoCapabilitiesKHR to obtain the limits of the parameters allowed by the implementation for a particular codec profile.

  6. Bind memory resources with the encode video session by calling vkBindVideoSessionMemoryKHR. The video session cannot be used until memory resources are allocated and bound to it. In order to determine the required memory sizes and heap types of the device memory allocations, vkGetVideoSessionMemoryRequirementsKHR should be called.

  7. Create one or more Session Parameter objects for use across command buffer recording operations, if required by the codec extension in use. These objects must be created against a video session with the parameters required by the codec. Each Session Parameter object created is a child object of the associated Session object and cannot be bound in the command buffer with any other Session Object.

The recording of Video Encode Commands against a Vulkan Command Buffer consists of the following sequence:

  1. vkCmdBeginVideoCodingKHR starts the recording of one or more Video Encode operations in the command buffer. For each Video Encode Command operation, a Video Session must be bound to the command buffer within this command. This command establishes a Vulkan Video Encode Context that consists of the bound Video Session Object, Session Parameters Object, and the required Video Picture Resources. The established Video Encode Context is in effect until the vkCmdEndVideoCodingKHR command is recorded. If more Video Encode operations are to be required after the vkCmdEndVideoCodingKHR command, another Video Encode Context can be started with the vkCmdBeginVideoCodingKHR command.

  2. vkCmdEncodeVideoKHR specifies one or more frames to be encoded. The VkVideoEncodeInfoKHR parameters, and the codec extension structures chained to this, specify the details of the encode operation.

  3. vkCmdControlVideoCodingKHR records operations against the encoded data, encoding device, or the Video Session state.

  4. vkCmdEndVideoCodingKHR signals the end of the recording of the Vulkan Video Encode Context, as established by vkCmdBeginVideoCodingKHR.

In addition to the above, the following commands can be recorded between vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR:

  • Query operations

  • Global Memory Barriers

  • Buffer Memory Barriers

  • Image Memory Barriers (these must be used to transition the Video Picture Resources to the proper VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR and VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layouts).

  • Pipeline Barriers

  • Events

  • Timestamps

  • Device Groups (device mask)

The following Video Encode related commands must be recorded outside the Vulkan Video Encode Context established with the vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR commands:

  • Sparse Memory Binding

  • Copy Commands

  • Clear Commands

39.8.1. Video Encode Vulkan Command Buffer Commands

To launch an encode operation that results in bitstream generation, call:

// Provided by VK_KHR_video_encode_queue
void vkCmdEncodeVideoKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoEncodeInfoKHR*                 pEncodeInfo);
  • commandBuffer is the command buffer to be filled with this function for encoding to generate a bitstream.

  • pEncodeInfo is the pointer of VkVideoEncodeInfoKHR.

Valid Usage (Implicit)
  • VUID-vkCmdEncodeVideoKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdEncodeVideoKHR-pEncodeInfo-parameter
    pEncodeInfo must be a valid pointer to a valid VkVideoEncodeInfoKHR structure

  • VUID-vkCmdEncodeVideoKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdEncodeVideoKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support encode operations

  • VUID-vkCmdEncodeVideoKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdEncodeVideoKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

The VkVideoEncodeInfoKHR structure is defined as:

// Provided by VK_KHR_video_encode_queue
typedef struct VkVideoEncodeInfoKHR {
    VkStructureType                   sType;
    const void*                       pNext;
    VkVideoEncodeFlagsKHR             flags;
    uint32_t                          qualityLevel;
    VkExtent2D                        codedExtent;
    VkBuffer                          dstBitstreamBuffer;
    VkDeviceSize                      dstBitstreamBufferOffset;
    VkDeviceSize                      dstBitstreamBufferMaxRange;
    VkVideoPictureResourceKHR         srcPictureResource;
    const VkVideoReferenceSlotKHR*    pSetupReferenceSlot;
    uint32_t                          referenceSlotCount;
    const VkVideoReferenceSlotKHR*    pReferenceSlots;
} VkVideoEncodeInfoKHR;
  • sType is the type of this structure.

  • pNext is a pointer to a structure extending this structure. A codec-specific extension structure must be chained to specify what bitstream unit to generate with this encode operation.

  • flags is encode flags of type VkVideoEncodeFlagBitsKHR, reserved for future versions of this specification.

  • qualityLevel is the coding quality level of the encoding. It is defined by the codec-specific extensions.

  • codedExtent is the codded size of the encode operations.

  • dstBitstreamBuffer is the buffer where the encoded bitstream output will be produced.

  • dstBitstreamBufferOffset is the offset in the dstBitstreamBuffer where the encoded bitstream output will start. dstBitstreamBufferOffset’s value must be aligned to VkVideoCapabilitiesKHR::minBitstreamBufferOffsetAlignment, as reported by the implementation.

  • dstBitstreamBufferMaxRange is the maximum size of the dstBitstreamBuffer that can be used while the encoded bitstream output is produced. dstBitstreamBufferMaxRange’s value must be aligned to VkVideoCapabilitiesKHR::minBitstreamBufferSizeAlignment, as reported by the implementation.

  • srcPictureResource is the Picture Resource of the Input Picture to be encoded by the operation.

  • pSetupReferenceSlot is an optional parameter used for generating a reconstructed reference slot and Picture Resource. VkVideoReferenceSlotKHR::slotIndex specifies the slot index number to use as a target for producing the Reconstructed (Dpb) data. pSetupReferenceSlot must be one of the entries provided in VkVideoBeginCodingInfoKHR via the pReferenceSlots within the vkCmdBeginVideoCodingKHR command that established the Vulkan Video Encode Context for this command.

  • referenceSlotCount is the number of the Reconstructed Reference Pictures that will be used when this encoding operation is executing.

  • pReferenceSlots is a list of the reference slots that will be used when this encoding operation is executing. Each entry in pReferenceSlots must be one of the entries provided in VkVideoBeginCodingInfoKHR via the pReferenceSlots within the vkCmdBeginVideoCodingKHR command that established the Vulkan Video Encode Context for this command.

Multiple vkCmdEncodeVideoKHR commands may be recorded within a Vulkan Video Encode Context. The execution of each vkCmdEncodeVideoKHR command will result in generating codec-specific bitstream units. These bitstream units are generated consecutively into the bitstream buffer specified in dstBitstreamBuffer of VkVideoEncodeInfoKHR within the vkCmdBeginVideoCodingKHR command. The produced bitstream is the sum of all these bitstream units, including any padding between the bitstream units. Any bitstream padding must be filled with data compliant to the codec standard so as not to cause any syntax errors during decoding of the bitstream units with the padding included. The range of the bitstream buffer written can be queried via video encode bitstream buffer range queries.

Valid Usage (Implicit)
  • VUID-VkVideoEncodeInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_ENCODE_INFO_KHR

  • VUID-VkVideoEncodeInfoKHR-pNext-pNext
    Each pNext member of any structure (including this one) in the pNext chain must be either NULL or a pointer to a valid instance of VkVideoEncodeH264EmitPictureParametersEXT or VkVideoEncodeH264VclFrameInfoEXT

  • VUID-VkVideoEncodeInfoKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoEncodeInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoEncodeFlagBitsKHR values

  • VUID-VkVideoEncodeInfoKHR-dstBitstreamBuffer-parameter
    dstBitstreamBuffer must be a valid VkBuffer handle

  • VUID-VkVideoEncodeInfoKHR-srcPictureResource-parameter
    srcPictureResource must be a valid VkVideoPictureResourceKHR structure

  • VUID-VkVideoEncodeInfoKHR-pSetupReferenceSlot-parameter
    pSetupReferenceSlot must be a valid pointer to a valid VkVideoReferenceSlotKHR structure

  • VUID-VkVideoEncodeInfoKHR-pReferenceSlots-parameter
    pReferenceSlots must be a valid pointer to an array of referenceSlotCount valid VkVideoReferenceSlotKHR structures

  • VUID-VkVideoEncodeInfoKHR-referenceSlotCount-arraylength
    referenceSlotCount must be greater than 0

The vkCmdEncodeVideoKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_encode_queue
typedef enum VkVideoEncodeFlagBitsKHR {
    VK_VIDEO_ENCODE_DEFAULT_KHR = 0,
    VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR = 0x00000001,
} VkVideoEncodeFlagBitsKHR;
  • VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR The current version of the specification has reserved this value for future use.

// Provided by VK_KHR_video_encode_queue
typedef VkFlags VkVideoEncodeFlagsKHR;

VkVideoEncodeFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoEncodeFlagBitsKHR.

The VkVideoEncodeRateControlInfoKHR structure is defined as: