Video Testing for STB Manufacturers

This paper explores the challenges Video Equipment Manufacturers face when assessing video quality. Assessing video quality ultimately depends on the customer's reaction on their new display (PC, POD, TV, etc.).

Set-top box (STB) functions are continually advancing, but the customers' reactions are usually centered on the quality of the picture. So testing is increasing important. The test setup simply stated is
* Start with a known video sequence.
* Decode the processed video sequence.
* Capture the processed video sequence.
* Display a "gold standard" and processed video sequences.
* Bring in experts to subjectively vote.

Complexity arises as
* New Video Processing systems may need new equipment to playback the video sequences.
* The original and processed video sequences should be displayed in random orders.
* Expert viewers are expensive and do not produce repeatable results.

So a need arises to perform quantitative, repeatable objective scoring of the video.
Previous Options
Each vendor builds unique test equipment to verify their new algorithms. So the first job is to debug the test equipment before it can be used to verify a new design. Debugging the test equipment can take as long if not longer than debugging the display equipment.

Easier Solution
To streamline the process, equipment for video quality testing needs to be defined, which can capture, play, and analyze any two video sequences. Further, as new input/output modules are continuously under development, the test equipment should use an open-architecture approach to ease upgradeability.

* Capture video sequences in as many formats as possible.
* Convert all video sequences to user-selectable resolution.
* Translate all video sequences to uncompressed Y'CbCr 4:2:2 or RGB 4:4:4.
* Support 8 and 10-bit data paths with upgradeability to future 12-bit modes.
* Store the video sequences as frames (fields) so that they can be played at any rate.
* Display the video sequences in real time in multiple viewing modes.
* VTR controls - play, shuttle, jog, pause
* Integer-based zoom & pan.
* Apply objective metrics to the video sequences.
* Export pieces of video sequences to further analyze off-line.
* Use a standard operating system so that the operator can run 3rd party analysis applications.

To further simplify the work flow, any video sequence can be played; while capturing another video sequence, thus, combining the video server and capture device into one unit. By doing this, the original source is already inside the test equipment so captured content alignment is easily obtained.

The original and processed video sequences can be displayed - side-by-side, mirrored, or seamless split - on a single display. This eliminates the need to calibrate two separate displays.

ClearView applies various objective metrics to the video sequences, generates graphs, and calculates an objective score. ClearView includes PSNR, Spatial Information and Temporal Information (as proposed in ITU-T P.9210). These metrics are the basis for more sophisticate metrics like VQM, JND, which are being analyzed by organizations like VQEG, VPQM, and SMPTE.

ClearView displays multiple video sequences, even if they are played at different rates (i.e. mobile phone video compared to TV) for the expert viewers; while recording the objective metric scores. While the MOS cannot be repeated, the objective metric can, easily and readily.

Since the system is based on an open, Windows-based architecture, any objective measurement algorithm can be modeled off-line using the stored video sequences.

Benefits
* Repeatable tests, quantitative results, and a streamlined setup.
* Analyze 2 video sequences in real-time up to 1080P.
* Input virtually any file type or capture from any digital or analog source.
* Multiple viewing modes are presented on a single display - no need to calibrate 2 separate Television displays to compare video sequences.
* Integrated uncompressed, high definition Video Server and Capture Device.
* Ability to Play 2 fully uncompressed, HD Streams in Real-Time.
* Hybrid Solution with Integrated Objective Metrics and Subjective Viewing Modes.

Case Examples
In all of the following examples, the tests can be performed using software mockups of the actual hardware unit. ClearView exports video sequences as files and accepts video files as inputs.

A Set-top Box (STB) or Decoder manufacture needs to
* Capture a "Golden" video stream or the results of one of their competitors.
* Capture the output of their unit.
* Visually inspect the "Golden" stream compared to their decoded stream.
* Generate a Score for pass/fail.

Summary
ClearView takes advantage of the high-reliability of today's off-the-shelf computer platforms. This ensures that products are made with state-of-the-art hardware, while at the same time avoiding the high cost of custom designs.

ClearView provides broadcasters, video researchers, compression developers with the unique ability to capture, play-out, and analyze video sequences. Objective measurements are generated and logged for repeatable tests.