4K Video Processing with Kria SoM-KV260

This tutorial is “foundation” of creating the “Vision ML Pipeline” with 4K resolution in Kria KV260. We already have few Video Demo on “4K Resolution Machine Learning with Kria KV260”.

This tutorial or flow is also compatible to other MPSoC FPGA devices as well as Boards. In most of MPSoC boards the DP (Display Port) is connected to PS, so using this approach you can run the Video Pipeline of 4K(4096×2160) resolution. On Kria KV260 , along with DP, HDMI is also connected to PS, so with Kria KV260 ether of DP or HDMI can be connected with 4K monitor for this/these demo. In Kria KV260, DP and HDMI are connected to PS and signal is spitted to DP/HDMI using splitter, so both interface gives same output. So, we can use either of DP or HDMI in Kria KV260.

  1. 8 channel (4K) Video Streaming on Xilinx Kria KV260: YouTube Video Demo
  2. 8 Channel + 8-ML Model inferencing on Kria KV260 with 4K Output: Video Demo
  3. 4K Video – ML Inferencing on Kria KV260: Video Demo
4K Resolution- Video ML Inferencing on Kria KV260: Demo

Lets jump to the fundamental of “4K Video Processing Pipeline design” for Kria KV260 or MPSoC Boards.

Starting point of 4K video processing is “Using Test Pattern Generator(TPG) IP in VIVADO Hardware Design” and sending that TPG-4K output to DP Port (in case of Kria KV260 DP or HDMI) with Baremetal Flow (VIVADO/Vitis SDK).

LogicTronix’s Nikhil Thapa has detailed the “4K TPG Design with Kria KV260” in his turorial. Nikhil’s original article can be found here: Hackster.IO. This tutorial follows the Vitis 2021.1 version. For starting the project, you have to follow these steps:

  1. Launch VIVADO 2021.1 and create the project with Kria-KV260 starter Kit
  2. Now, create Block Design and add
    • Zynq UltraScale+ MPSoC Processing System (PS), for sending data from PS to DP, follow:
      • Open customization window > PS-PL Configuration > General > Others > Live Video > 1
  1. Now add Clocking Wizard , and customize clocking wizard with following 3 clocks as we needed for 4K TPG streaming design
    • clk_out1 – 100MHz for Axilite interface
    • clk_300 – 300MHz for AXI Stream
    • clk_297 – 297MHz for Video clock
  2. Now add Video TPG and configure following Columns x Rows as per our 4K resolution requirement
  1. Now add Video Timing Controller and configure as follows
  • In Default/Constant tab, we can select any of three video modes to generate their respective video timing. We can also select custom mode to manually set the video timing for custom resolution.
  • Now, enable the AXI-4 Lite on VTC

The timing generation is controlled by Axis2Video for the synchronization between the stream and its timing. So, we need to connect VTC’s gen_clken pin to Axis2Video’s vtg_ce pin.

  1. Now Add AXI4-Stream to Video Out (Axis2Video)
    • This IP takes AXI4 Stream and Video Timing as Input and converts it into Parallel Video. We can then connect this video to the DP video interface in the Zynq PS block. As DP video data width is 36 bit, we also need to set parallel video data width to 36 bit by setting Native Video Output Component Width 12.
    • We also set clock mode Independent so that we can give separate video clock. We let other parameters as it is.
  • On expanding the IP Ports the AXI4-Stream to Video Out (Axis2Video) looks like:
  1. IP Interconnection, connect IP/ports as following
    • (Axis2Video) IP to (Zynq U+ MPSoC PS)
    • vid_active video ——> dp_live_video_in_de
    • vid_data ——> dp_live_video_in_pixel1[35:0]
    • vid_hsync ——> dp_live_video_in_hsync
    • vid_vsync ——> dp_live_video_in_vsync
  2. Now connect Clocking Component
    • m_axis_video (VTPG) ——-> video_in (Axis2Video)
  3. As we are generating three clock outputs from the clock wizard, we need to connect these in the following manner.
  1. Here IP Block Design looks like this:
  1. Now , perform “Run Connection Automation”
  2. Validate the Design
  3. As there is no need of user defined constraint or PIN allocation, no constrainting is needed.
  4. Generate Bitstream
  5. Now you can export the hardware for Vitis Application:
    • Vivado Menu > File > Export > Export Hardware > Next > Include bitstream > Next > Set XSA file name > Finish

Vitis SDK Application Development: We are updating it here soon!

Till then you can follow the “Kria Display Port Initialization in Baremetal Design” tutorial by Adam Taylor. Or also review the “Kria PCAM (Camera) streaming with DP/HDMI” tutorial Vivado and Vitis Source from: Kira_PCAM .

The Credit of this article goes to our Senior Engineer, Nikhil Thapa.