Going Tapeless with the Omneon MediaGrid and Bridging the Gap to the Tape Archive
It also needed to wrangle a sizable archive of the facility’s past work, and to do it in a way that a staff of just seven people could easily manage. “Bruce was looking into streamlining our process as we search for media,” White adds. “People call for DVDs, saying they need a copy of this or a piece of that, so we do a lot of searches. We needed to streamline those processes internally for our team, because we don’t have the bandwidth to do anything lengthy to acquire the media.”
In its own studio, Covey explains, HDTV acquires SD footage in MPEG IMX at 30 Mbps and HD footage in XDCAM HD at 50 Mbps. The in-house tape standard is SD DV. Meanwhile, material from the field comes in any number of formats. An entry-level MediaGrid and four-channel MediaDeck now handle all studio ingest. Covey knew he would need to use data tapes as the final resting place for footage in the archives, and Omneon recommended the XenData system, with the caveat that it was still under development at the time. So HDTV bit the bullet and moveed files to the tape library manually, all the while looking forward to the XenData solution coming online. “The problem was the Omneon MAS system would lose awareness of the asset,” Covey says. “We had this nice system that would let you start new porjects, track assets, and see proxy views of everything, but when you moved it off to the archive you had to use a completely separate system to manage what you had. We were still able to use the MAS to view proxies, but we couldn’t recall anything from there. It was a little laborious.”
Finally, in November of 2011, a pair of engineers arrived on site with a pre-release version of Omneon’s MAS 3.4, and HDTV got to try out the XenData integration that would let them move projects from online (MediaGrid) storage to offline (Qualstar) storage from within the MAS. With the XenData software and the Omneon software talking to each other, Omneon’s database of project folders keeps track of how different files are related to different projects even after the files move offline. “The beauty of this is we can still view the proxies, and if we identify a clip or even an entire folder that we need – something like our forklift training, which gets redone every year – the producer scrubs through the proxy views, finds the clip he needs, and flags a restore process that will load the tape, recall the clip, and define where in the MediaGrid it’s transferred to.”
The Omneon system was pretty impressive to start with, but Covey is decidedly more enthusiastic now that it’s so seamlessly unified with the tape library. “I felt like I had a piece of cake with no icing until November, and just a few weeks ago we got the final release of MAS 3.4,” Covey says. “Gosh, I probably already laid off a terabyte of data on our new tape storage.”
But using the Omneon/XenData system won’t do your thinking for you. HDTV has developed a specific project structure on the MediaGrid that balances easy access to assets with the time required to deal with them. A project starts with the creation of a uniquely named folder on the MediaGrid. In the edit suite, an individual user will mount that project folder as if it were a disk drive. (Previously, HDTV ran on fibre-channel storage with software from Transoft Networks that split a volume into virtual partitions, which created problems because the size of a partition, once created, was fixed.) All users have read access to the storage, but they can only write to a folder that they’ve mounted.
Managing the endless complexity of proxy files demanded an organizational solution, as well. Creating an individual proxy of every single clip that’s brought into a project would just generate a morass of thousands upon thousands of proxies, the vast majority of them of negligible value in the long run. So only finished, edited pieces get encoded as proxies. “In every project folder, we have an automatically created archive folder,” Covey says. “Any pertinent versions of a show are named with certain conventions and dropped into that folder. This immediately creates an asset with a proxy file. Sometimes, when somebody wants to go back to the well and look for a shot of this or that, they find this giant folder full of P2 clips that nobody ever did anything with. So we ask our producers to make their best effort to scrub through their field B-roll and put together rough sequences and drop those in an archive folder. So a project might have multiple assets and multiple proxies linked to it. When we get ready to move offline, it’s transferred over to the XenData. MediaGrid still has the assets showing, in an offline state, and all the proxies are fully available.”
There are potential pitfalls of moving between the Unix-driven MediaGrid system and the PC-based XenData software – those file names can get you “We run into problems over and over again of people using illegal characters in file names,” Covey says. “We use a little app called NameCleaner. Just before we put something away, we drop a folder on it and let it look through to see if we’ve got any file-name problems.”
Now that the cake has its icing, Covey is looking forward to a promised feature that would allow partial clips to be restored from the archive, rather than full files. That could save a lot of time if someone needs just a few seconds of an 8 GB video file – rather than transcode the entire clip, partial-clip restore would just transcode the relevant bits, making the recall a much faster process.
And management of the archive will become its own adventure. “We have 10,000 tapes in our existing library, and we’re culling those to see what we need to keep,” Covey says. “If we’ve got a series, do we need all 100 episodes of that, or do we need just a few representative episodes? We’re trying to do this intelligently, not convert the whole library in one shot. First, we’ve tried to go to the old Betacam material and find the most important historical material there. The bulk of the library is on DVCAM, so it will be fine for a few years.”
Of course, the new tape library isn’t endlessly capacious – everything ingested to the new tape library is being written to tape pairs, allowing second copies to eventually be stored in a separate location, which reduces its effective capacity to 40 tapes’ worth of footage. What happenes when a proxy refers to a file that’s not just offline, but outside of the robotic library completely? “It’s a little bit tricky to open that robot and stick something in its mail slot, so Xen sold us an external drive we put outside the rack,” Covey explains,. “If the MAS restore process says the tape is offline, someone has to go get that tape off the shelf and shove it in the external drive. But for what we do, things typically have a six-month lifespan and then it gets quiet on a project. And I think the robot is going to have a six-month or one-year window of projects sitting in there.”
The bottom line for HDTV is that Omneon has the knowhow to keep tabs on the chaotic file structure of Final Cut Pro projects. “When I started talking to archive companies, everyone walked in and said, ‘I have a solution for you,'” Covey recalls. “I would make them a disk drive with three Final Cut projects that were just a mess. I said, ‘Here, you make sense of this and come back to me.’ Nobody ever knocked on my door again. They were prepared to keep track of the final output piece, but not the mess that’s inside of it. But on the XenData system, if I recall an entire project folder back, the Omneon puts it back together just as it was to begin with. I was a little leery, but it works.”
For more information: www.homedepot.com; www.omneon.com; www.xendata.com.
Sections: Business Technology
Did you enjoy this article? Sign up to receive the StudioDaily Fix eletter containing the latest stories, including news, videos, interviews, reviews and more.
