Repeatable Video Testing Tools for Equipment Manufacturers

This paper explores the challenges Video Equipment Manufacturers face when assessing video quality. Assessing video quality ultimately depends on the customer's reaction on their new display (PC, POD, TV, etc.).

Over the next decade, it is expected that digital television penetration into the general television market will increase from less than 5% in 2003 to more than 50% by 2010.

Currently, the FCC requires that 50% of televisions 36 inches or larger have integrated digital capability by July 2005. By July 2007, the FCC will require that all new televisions 13 inches or larger must incorporate a digital tuner. Obviously the FCC's ruling provides additional fuel for the growth of the DTV market.

So testing is increasing important. The test setup simply stated is
* Start with a known video sequence.
* New Video Processing system alters the video sequence.
* Decode the processed video sequence.
* Capture the processed video sequence.
* Display the original and processed video sequences.
* Bring in experts to subjectively vote.

Complexity arises as
* New Video Processing systems may need new equipment to playback the video sequences.
* The original and processed video sequences should be displayed in random orders.
* Expert viewers are expensive and do not produce repeatable results.
* Digital displays scale incoming video sequences based on their native resolution.
* Display manufactures want to check resolutions at 2x the video rate.

Previous Options
Each vendor builds unique test equipment to verify their new algorithms. So the first job is to debug the test equipment before it can be used to verify a new design. Debugging the test equipment can take as long if not longer than debugging the display equipment.

Easier Solution
To streamline the process, equipment for video quality testing needs to be defined, which can capture, play, and analyze any two video sequences. Further, as new input/output modules are continuously under development, the test equipment should use an open-architecture approach to ease upgradeability.

Video Clarity defined the ClearView product line with these objectives in mind.
* Capture video sequences in as many formats as possible.
* Convert all video sequences to user-selectable resolution.
* Translate all video sequences to uncompressed Y'CbCr 4:2:2 or RGB 4:4:4.
* Support 8 and 10-bit data paths with upgradeability to future 12-bit modes.
* Store the video sequences as frames (fields) so that they can be played at any rate.
* Display the video sequences in real time in multiple viewing modes.
* VTR controls - play, shuttle, jog, pause
* Integer-based zoom & pan.
* Apply objective metrics to the video sequences.
* Export pieces of video sequences to further analyze off-line.
* Use a standard operating system so that the operator can run 3rd party analysis applications.

By working in the uncompressed domain, any two video processing algorithms can be compared independent of compression or other processing.

To further simplify the work flow, any video sequence can be played; while capturing another video sequence, thus, combining the video server and capture device into one unit. By doing this, the original source is already inside the test equipment so captured content alignment is easily obtained.

The operator chooses the output video resolution and output rate independent of the input. Video sequences are cropped or centered with black borders to meet the desired resolution, and then played-out at rates up to 120Hz. The video sequence is sent to the display adapter or panel to test desired resolutions and frame rates.

Further the operator can play any stored video sequence, at any speed, for any duration either manually or using automated play lists.

The original and processed video sequences can be displayed - side-by-side, mirrored, or seamless split - on a single display. This eliminates the need to calibrate two separate displays.

ClearView applies various objective metrics to the video sequences, generates graphs, and calculates an objective score.

While development of additional objective algorithms is ongoing, we have built a hybrid system that takes into consideration subjective testing with objective measurements. ClearView can easily be programmed to display video sequences for the expert viewers; while recording the objective metric scores along with the MOS. While the MOS cannot be repeated, the objective metric can, easily and readily.

Since the system is based on an open, Windows-based architecture, any objective measurement algorithm can be modeled off-line using the stored video sequences.

Benefits
* Repeatable tests, quantitative results, and a streamlined setup.
* Analyze 2 video sequences in real-time up to 1080P.
* Input virtually any file type or capture from any digital or analog source.
* Multiple viewing modes are presented on a single display - no need to calibrate 2 separate Television displays to compare video sequences.
* Integrated uncompressed, high definition Video Server and Capture Device.
* Ability to Play 2 fully uncompressed, HD Streams in Real-Time.
* Hybrid Solution with Integrated Objective Metrics and Subjective Viewing Modes.

Case Examples
In all of the following examples, the tests can be performed using software mockups of the actual hardware unit. ClearView exports video sequences as files and accepts video files as inputs.

A Video Processing manufacturer needs to
* Source for their video processing or encoders in SDI, Component, or DVI.
* Capture the output of their unit.
* Capture the output of their unit after it has been decoded.
* Visually inspect the new algorithm compared to the original and/or previous algorithms.
* Generate a Score for repeatability.

A VOD Server manufacture needs to
* A way to bring in multiple file formats into their system through SDI or Component.
* Visually inspect the output of their VOD server compared to the original Source.
* Generate a Score for repeatability.

A Set-top Box (STB) or Decoder manufacture needs to
* Capture a "Golden" video stream or the results of one of their competitors.
* Capture the output of their unit.
* Visually inspect the "Golden" stream compared to their decoded stream.
* Generate a Score for pass/fail.

A Graphic or Display chip manufacture needs to
* Play out "Source" video sequences at various resolutions and rates to their graphic or display chip through DVI, HDMI, or VGA.
* Capture the output of their unit.
* Visually inspect the output of their algorithm compared to the original Source.
* Generate a Score for repeatability.

A Display manufacture needs to
* Play out "Source" video sequences at various resolutions and rates to test their display through DVI, HDMI, or VGA.
* Visually inspect the performance of their Display.
* Generate a count of dropped frames if any occurred.

Summary
ClearView takes advantage of the high-reliability of today's off-the-shelf computer platforms. This ensures that products are made with state-of-the-art hardware, while at the same time avoiding the high cost of custom designs.

ClearView provides broadcasters, video researchers, compression developers with the unique ability to capture, play-out, and analyze video sequences. Objective measurements are generated and logged for repeatable tests.