The debate over compression schemes, which began in the early days of digital videotape, continues to rage in today’s high-definition (HD) file-based world.

At issue currently, as it was in the’90s, is file size versus image quality. The smaller the file size, the easier it is to share and transmit. Yet, if you’re a content creator or broadcaster, you still want to acquire the best-looking images possible to ensure they will still look good after transmission.

The strategies for reducing file sizes without compromising picture quality differ from manufacturer to manufacturer, and the issue has intensified during hotly contested exchanges at recent trade shows and engineering gatherings. In the weeks leading up to NAB, both Panasonic and Sony are sending representatives around the country with media players and projectors, trying to convert customers to its file-size reduction strategy. One argues higher bit rates are more important to customers while the other says it’s workflow that matters most to users. Both say the resulting quality of their respective compression systems is superior.

Other camera vendors, such as JVC and Thomson/Grass Valley, have chosen to distance themselves from the controversy, although both feel that workflow is most important to any compression discussion. It’s difficult to cram 100 Mbps material down most infrastructures and keep it intact, they say. JVC, for example, is promoting a 25 Mbps workflow for HD news production, where it says the speed with which new information must be dispatched is a critical part of the workflow.

Thomson also offers customers of its Grass Valley Infinity digital media camcorder the choice of acquiring images using MPEG-2 or the JPEG2000 compression format. Currently, the infrastructure to produce programming in JPEG2000 is not as mature as the ubiquitous MPEG-2 or AVC (MPEG-4) standards, although Thomson says it can offer a complete workflow- including editing software and servers- to support it.

In terms of image quality, MPEG-2 codecs offer 8-bit processing, while AVC and JPEG2000 can leverage a more robust 10-bit architecture. Most agree that the higher processing in 10-bit 4:2:2 recording produces brighter, more colorful images at full raster (1920 x 1080). But original file sizes are larger. Panasonic P2 cameras currently use AVC-I compression to record 50 Mbps material at 10-bit 4:2:0 and 100 Mbps at 10-bit 4:2:2.

Sony contends that its latest-generation algorithms do as good a job at 50 Mbps as AVC-I can do at 100. However, in digital cinema and EFP-style productions, 50 Mbps does not always prove sufficient, in terms of offering the highest quality acquisition. It’s in this environment that many are using 100 Mbps recording (and storing it on Sony’s MPEG-2 compressed HDCAM SR videotape at 4:4:4 or Panasonic’s D-5 tape format, compressed in DVCPRO) to get the job done.

"In today’s world, everything represents a compromise," says Dave Walton, assistant vice president of marketing for JVC Professional Company. JVC’s ProHD cameras use MPEG-2 compression to capture images at 1080i and 720p HD resolution. "When you’re trying to develop the criteria under which your compromises are made, you have to establish a floor level of quality that you’ll be happy with. If you go with the highest quality at the front end, you’ll have to make compromises in other places."

Another issue to consider is that editing footage stored in MPEG-2 as a long group of pictures (long-GOP)- Sony’s preferred format- is not as easy as working with AVC-I’s Intraframe scheme. However, it takes considerably more processing to edit in AVC. What many people don’t realize is that the AVC specification also includes long-GOP tools. Sony has stated its plans to move to an AVC long-GOP scheme by 2010 to facilitate 1080/60p acquisition at 60-75 Mbps. (Not all of Panasonic’s 100 Mbps DVCPRO HD cameras offer AVC-Intra, but the HPX2000 and HPX3000 offer both AVC-Intra and DVCPRO HD.)

"I think Panasonic has found out the hard way that the AVC codec for high definition is not an easy animal to corral," says Jan Eveleens, vice president of image capturing systems at Thomson. "Putting HD images into a small package with reasonable power consumption is tough to do well. Conversely, if Sony or anyone else decides to move to AVC and keep the long-GOP structure, it’s going to be a lot harder. Although it will require less processing power than having to compress every frame [as in Intraframe coding], we still need one more generation of silicon before it’s practical."

The real issue for camera manufacturers is getting the codec (which consumes about 100 watts) to fit into a small body. Thomson offers an MPEG-4 AVC long-GOP chip that’s being used inside its rack-mounted ViBE encoder set-top boxes for transmission. However, that box employs four chips to produce adequate AVC long-GOP compression and uses about 60 watts. That’s too much real estate and not enough power for a shoulder-mounted camera. And it will never work inside a small handheld camcorder, at least not for some time. (One of the first standard-definition digital MPEG-2 camera codec demonstrations from Phillips took place only 14 years ago at an IBC conference. The prototype was the size of a small mini-refrigerator.)

That’s why camera compression systems have traditionally lagged about two or three years behind in codec (silicon) design. For other manufacturers, such as NLE vendors, size is simply not a factor. It’s computing horsepower, either in hardware or software, and advanced algorithms that matter most.

Panasonic is pushing a compression technology called AVC-Intra alongside its DVCPRO-based P2 product line. AVC-Intra uses Intraframe ("single" frame, in other words) transform-based compression algorithms to allow users of its solid-state P2 systems to record HD data as smaller files on a P2 card. This increases recording times in their P2 systems, or alternatively offers much higher quality content at conventional DVCPRO HD bit rates. AVC-Intra comes in two flavors: AVC-Intra 100 (at 100 Mbps), for full raster 1920 x 1080 or 1280 x 720, 10-bit 4:2:2 recordings for high-end production; and AVC-Intra 50 (at 50 Mbps), for DVCPRO HD quality at half the bit rate, thus doubling the record time on a given P2 card.

Some say that AVC-Intra is twice as efficient as interframe-based MPEG-2 long-GOP compression, used by Sony in its tape and optical-disc-based cameras. Others say AVC-Intra does this by throwing away half the original picture information during recording, so it’s really only "half-def" quality.

"If AVC is used with all of its compression tools, such as multi-frame and intra-picture prediction capabilities, then AVC can be twice as efficient as MPEG-2 long-GOP," says Bob Ott, vice president of optical and network systems at Sony. "But this efficiency has to be qualified by the data rate that the algorithm is working at: sub-megabit per second for SD signals or under 10 Mbps for HD signals, approximately. So, yes, AVC long-GOP is about twice as efficient as MPEG-2 long-GOP- but not necessarily at higher data rates."

He adds that when AVC Intraframe is used, especially at 50 Mbps and 100 Mbps for HD signals, the algorithm is not more efficient than MPEG-2 long-GOP at lower data rates.

AVC-Intra should not be confused with the AVCHD format- supported by both Panasonic and Sony- that’s designed for consumer HD camcorders and for DVD authoring of Blu-ray HD discs for home release. Both Panasonic and Sony offer AVCHD cameras for prosumers, and at NAB Panasonic is targeting event videographers with a new shoulder-mount "professional" AVCHD camera (the AG-HMC70)- although this new camera records at a low bit rate of about 6-13 Mbps, separating it from most other professional-level cameras in terms of picture quality.

Sony’s XDCAM HD recording system, its new XDCAM EX and its newest hybrid tape-based digital camcorders are all based on MPEG-2 long-GOP compression, which the company has supported across its entire product line since the introduction of the Z1U HDV camcorder. Sony says its latest algorithms for compressing MPEG-2 material are more efficient than ever.

On the other hand, AVC-Intra and MPEG-2 are not compatible. Material shot with one format must be transcoded to the other when using a competing system. However, AVC is part of the MPEG specification approved by the ISO MPEG committee. (SMPTE approved a recommended practice for using the AVC/H.264/MPEG-4 part-10 standard at 50 and 100 Mbps with specific constraints, such as Intraframe.) Currently, Panasonic’s P2 HD products that can record AVC-Intra can also record in DVCPRO-based formats.

Sony says that in order for AVC-Intra 50 Mbps systems to capture images that are equal in image quality to MPEG-2 long-GOP at 25 Mbps, they have to use a much higher data rate. Sony also says that AVC-Intra requires seven to eight times more CPU power. Thus, using current workstations, someone would have to have four Intel dual-core processors. (Of course, it all depends on whether the algorithm is handling compression or decompression and whether real-time performance has to be guaranteed.) Panasonic says AVC-Intra requires only about twice the processing power of a long-GOP MPEG system, but is much less complex to process than AVCHD (long-GOP H.264/AVC), though both can be easily edited on today’s NLE platforms.

"AVC-Intra at 50 Mbps doesn’t even come close, in terms of picture quality, to MPEG-2 at 50," contends Ott, citing results shown at a number of demos around the country to prove Sony’s case.

"I think you have to go up to 140 Mbps to get the same quality as MPEG-2 at 25 Mbps," Ott says. From a field editing standpoint, he adds, there’s another clear advantage to MPEG-2: "AVC-Intra is not laptop-ready [due to processing requirements]. It’s going to take several years to get to where MPEG-2 is today."

According to Ott, AVC-Intra at 70-75 Mbps will match MPEG-2 long-GOP at 25 Mbps (all other elements being equal, such as 1440×1080 and 4:2:0 color structure). For AVC-I to exceed the quality of MPEG-2 long-GOP at 50 Mbps (1920 x 1080 and 4:2:2 color structure), Ott says, more than 100 Mbps would be needed, especially for demanding picture material and multigeneration work.

Panasonic, which conducted comparison demonstrations throughout 2007 and earlier this year at several trade conferences and exhibitions, counters that AVC-Intra 50 takes advantage of the new H.264 compression tools to offer high picture quality comparable to DVCPRO HD with double the storage efficiency (64 minutes on one 32 GB P2 card). And for 1920 x 1080 24p recordings in AVC-Intra 100, one 32 GB P2 card can hold up to 40 minutes of almost D-5 quality content.

There are laptop editing software packages available from Apple, Avid, Grass Valley and others that do support AVC-Intra material, but the bit rate is lower than 50 Mbps and the resulting picture quality is lacking. Ott says he expects clients will have issues with the quality, especially as a project goes through multiple iterations during editing and transmission.

AVC-Intra, like almost all Intraframe professional HD compression schemes used for acquisition and post-production (such as DVCPRO HD, D-5, HDCAM and HDCAM SR), facilitates more precise editing and images are generally sharper, as there is zero interaction with content in the adjacent video frames, says Steve Mahrer, director of engineering at Panasonic Broadcast’s Business Development Group. This results in dramatically higher image quality, more deterministic performance and easier implementation in hardware.

"AVC-Intra was designed to give professionals the highest level of image quality," Mahrer says. "The use of low-bit-rate MPEG-2 long-GOP compression for acquisition is limited in that it only provides a certain level of image quality, but is sometimes necessary when you’re trying to shoehorn a lot of HD data into a very small bucket."

He adds that long-GOP H.264/AVC, as used in the joint Panasonic/Sony AVCHD camcorder format, can provide a 2:1 or greater compression efficiency increase over older, less efficient MPEG-2 compression schemes. In other words, what MPEG-2 can achieve in terms of video quality at 25 Mbps, AVCHD can achieve at about 12 to 15 Mbps or less.

Mahrer also says that low-bit-rate long-GOP HD is very difficult to edit; even simple editing or manipulation of the video requires a very complex recalculation of the data stream parameters and subsequent recompression of the content. Such processing leads to severe quality degradation at only a few generations.

Both companies say proper comparisons should be done with certified hardware codecs to see the real-world differences. At the end of the day, it’s what’s on the screen that matters most.

"If you have to convert HD-SDI files to baseband, then baseband back for transmission, even though it is digital, there is a bit of a loss in quality," Sony’s Bob Ott says. "We’ve heard from TV networks who say by the time the signal gets to consumers’ homes it’s gone through five conversions. With today’s HDTV sets, you’ve got to be mindful of what compression you’re using. What’s that material going to look like?"

As mentioned earlier, Thomson uses MPEG-2 compression in its LDK HD camera line, but its new Infinity HD camera is designed to use either MPEG-2 or the emerging JPEG2000 format. The camera also has an option slot inside that could accommodate AVC-I or AVC long-GOP in the future as well.

"We have our own native JPEG2000 codec, but we also provide an MPEG-2 solution because we want to give users the choice to use whatever compression format works best for their type of production," says Thomson’s Eveleens.

Like AVC-I, JPEG2000 is an Intraframe format that requires significant horsepower (minimum quad-core CPU) for decoding compressed files. It’s a wavelet-based scheme that doesn’t use compressed blocks of images like MPEG-2 but fractals, which makes compression results more graceful and is better able to handle lots of motion or heavier details than MPEG-2. There are also multiple layers of detail that can be compressed, generally resulting in much better image quality.

Eveleens said that MPEG-2 compression is currently limited to 50 Mbps acquisition because of the media used, in this case optical disc. JPEG2000 can go to 75 and 100 Mbps and this high bit rate data can easily be stored with the Infinity camera on a Iomega REV Pro cartridge or solid-state (Flash) media.

"When choosing a format, or camera, it’s more important to figure out and understand how you will archive and edit the material," Eveleens says. "We don’t want to be placed into any one [compression] camp or another. When people talk about our competitors, they closely identify them with a specific storage media or compression format. At Thomson, we purposely chose to stay neutral with the Infinity camera and give customers the widest choice."

It would be easy to say that compression is just one small part of many important features any video professional must consider when choosing a camera. After all, the way you shoot and the variety of your projects often demand different requirements.

But as file sizes and storage requirements increase as HD projects become more widespread, compression also becomes much more important. Your choice of camera, and how it manages data at acquisition, will be a critical part of the storage equation later on.

The emergence of the Internet has led to the adoption of multiple video codecs. The manufacturers have listened long and hard to their customers, who’ve helped them form compelling reasons about why they believe a particular flavor of compression is best.

In the end, compelling arguments aside, equipment choices should ultimately be based on what the user sees in the resulting images. Without unlimited budgets and unlimited bandwidth, compressed footage is a fact of life for most video pros. If you’re in the market for a new camera, take time to study footage compressed in various formats, whether at trade shows like NAB or at dealer-sponsored demos in your area. Based on the types of projects you produce, which images, after compression, look best to you? The video doesn’t lie. In a compressed world, you should always ask yourself, "How good is‘good enough’?"

For additional information, visit www.projvc.com, www.panasonic.com, bssc.sel.sony.com and www.thomsongrassvalley.com.

Contributing Editor Michael Grotticelli is the editor of HD Studio and regularly covers the production industry.