Media Companies Are Using Tape for Delivery, Conform, Cloud Storage, and More
As media companies look to store ever-growing mountains of digital content for the long term, LTO tapes are becoming a de facto archival standard for media. Widespread adoption of the LTFS format means LTO tape can behave more like a disk drive in digital media workflows, and that's good news for companies whose business plan is to provide flexible and foolproof ways for customers to use LTO tape throughout the production and post process.
"When we started our company, for the most part, Hollywood production and post facilities who needed to move [digital content] off of their SAN would move it to hard drives and put them in a drawer," recalls Mark Ostlund of Cache-A, which was just exhibiting advanced LTO appliances like the Pro-Cache610 and the Simul-Copy system at the Hollywood Post Alliance Tech Retreat in Palm Springs, CA. "Now, everybody who was doing that has lost content, because HDDs are only viable for about five years—and, in fact, the signal can fade in as little as one year. So the industry has gone through self-education.
"But if we can't put it on hard drives, and spinning disk is just too expensive, what is the best long-term, archive-quality media? Everyone has stepped up and said it's LTO."
What's behind the increasing adoption of LTO? Vendors say the LTFS format, introduced in 2010, has been a driver. "Previous generations of LTO were typically written to in tar format," explains Doug Hynes of Storage DNA, which just launched a self-contained Mac-based version of its DNA Evolution platform for nearline and archival use of LTO tape. "Applications couldn't see the files on the tape, so you didn't have accessibility. But we're taking full advantage of LTFS with DNA Evolution."
Hynes notes that, because LTFS makes tapes act like disks, LTO can be treated like a nearline storage medium. "Typically people used tapes for their deep archives. They really never anticipated getting that material back, and if they did, they expected it was going to take a really long time. But people are using our solution day in and day out, moving off of spinning disk to save capacity and space, then bringing them back when they need to quickly and easily."
Scaling the Archive
At XenData, which recently introduced the SXL-5000, a scalable LTO archive that starts at 210 TB and scales to nearly 1.2 PB, Phil Storey says customers are watching their archival needs grow and realizing that they can't just keep scaling out disk-based storage indefinitely. "Just one example is a Miami-based Latino TV station that was initially using 1U 8TB $2,000 NAS devices for their archive. They were buying one or two NAS systems a month, and they realized this just wan't sustainable," he recalls. "Another example is a well-known Hollywood organization that had many millions of files on external FireWire- and USB-attached disks. In both of those cases, they're not offlining their LTO, but keeping it nearline within a Nokia. And there is a realization that disk—whether it be spinning disk or disk sitting on the shelf—is not a good way to go."
Beyond archiving, LTO is increasingly functioning as a deliverable in its own right. Cache-A sees customers eking out new efficiencies with what it calls "hub and spoke" delivery models that have remote content creators providing LTO deliverables to larger media companies at the hub. (Some customers still want tar deliverables, especially if they originally specified their delivery requirements before the arrival of LTFS, but Ostlund said the momentum is with LTFS.) Discovery Communications, for example, has three different specs for broadcast deliverables on LTFS tape, according to Ostlund. One of those is for camera source material, another is for graphics and post-production, and a third is for programs. All of them are legal LTFS tapes, but with different arrangements of files and metadata—such as MD5 checksums for every graphic stored in a special file in the tape's index partition.
"That [arrangement] allows them to load up the programs or specific files they know are going to go into the final program, but also to keep all of the source footage that was generated," Ostlund says. "Many times, they've told us that 100 to 1,000 hours of footage is generated for every hour of program that goes on air. That's a lot of content they're keeping in their facility, and they don't have to actually load it and copy it and spend all that time [archiving it]."
Perfect Copies Can Live in the Cloud
Another benefit of LTO is that it's easy to make a perfect duplicate copy for safe-keeping or transport. "You can make a copy extremely fast in a portable format and use that to populate a data center, deep archive — or the cloud," says Ostlund. "If someone wants or needs cloud archiving capability, LTO can be used to feed that. Up until now, cloud storage has pretty much been seeded with hard drives, but there's quite a bit of chatter about LTO moving into that space."
At XenData, Storey agrees that LTO tape archives will soon find their place in the cloud. "A year ago, it would have been too early for us," he said. "Our typical customer has more than 100 TB of material. They need reasonable bandwidth to get files to the archive and back. There are parts of the world where bandwidth still costs a fortune but, more and more, the cost is going down. That's why we've announced a private, LTO-based cloud solution. If you put a server and an LTO library in the cloud, you can make it very price-competitive for people with large amounts of data and video content."
Storey said the cloud service is being introduced as a hosted disaster recovery system allowing companies to ship their LTO tapes to a cloud-accessible library at XenData's facility in Nevada. Eventually, he said, the program will expand to allow uploads from the customer site, as well.
Why Not Just Conform from LTO?
Another relatively new wrinkle is productions that conform from LTO, rather than card formats like SxS and P2. "As 2K and 4K files are starting to be generated in the field by cameras, there is no more camera master," Hynes says. "People are recording to storage media, but what are you doing with it once it gets into your facility? If it goes to disk, it doesn't give you a physical object to stick on a shelf. LTO is becoming a kind of new version of the old videotape, in some ways."
And Ostlund notes that, sometimes, the use of LTO tape on set is mandated. "A lot of insurance companies that underwrite high-end film productions are requiring that LTOs be produced on set," he says. "Some of them are saying they don't want more than 15 minutes [of footage] in the camera at one time."
If you're still an LTO skeptic—maybe you've been wondering if it's another in the long string of media formats that have fallen by the wayside over the years—Ostlund hopes you'll take some solace in the numbers. He says more than 200 million LTO cartridges have shipped in total, along with over 4 million LTO drives. "We're now using technology in the media and entertainment industry that has economies of scale that increase futureproof security of the format. In other words, you should be able to find an LTO drive 15 years from now."