The Rules of Sharing

The days of sneakernetting tapes around a post operation or rolling machines into a suite temporarily aren't over, but they're beginning to fade. As more productions move to high-resolution data-centric post processes – HD, 2K or 4K – production and post executives are pelted with alternatives for building IT-based pipelines that will allow their creatives to collaborate more seamlessly. The options for how to store, retrieve, manipulate and, perhaps most important, share media are many. What route you go depends on the complexity, time frames and budgets of your jobs.

Today, there are three primary means for storing media data:
direct-attached storage, network-attached storage and storage area
networks. Each means of storage has an important part to play in
maximizing the handling of any data associated with media; the question
becomes which means are better suited for a given production, whether
for speed (performance), reliability (enterprise versus desktop class
robustness), file sharing (collaboration), determinism (fixed access
latencies and guaranteed bandwidth), affordability, scalability,
interoperability, simplicity of deployment, ease of support or some
combination thereof.

If sharing isn’t a concern but speedy access to media (as data) is,
storage that is directly attached to the workstation (DAS) is
necessary. In this case, there is a one-to-one relationship between the
storage and the applications on that workstation that take advantage of
it. The benefits include speed, determinism and simplicity, while the
disadvantages include a lack of scalability, collaboration and sharing.
Vendors like Discreet, Avid and Quantel, for example, all require DAS
to enable their media applications to perform optimally at the
workstation level.

When files need to be shared across multiple users, network-attached
storage (NAS) offers collaboration but at the expense of speed. Because
NAS uses lower-cost Ethernet and standard IT-based building blocks to
connect storage with workstations and servers, affordability and ease
of management are among the key benefits. The speed, however, is
limited by the throughput of the Ethernet connection (and overhead)
between the workstations and NAS storage pool, be it 10Mb/s (Ethernet),
100Mb/s (Fast Ethernet) or 1000Mb/s (GigE).

Typical reasons for deploying NAS include rendering and cross-platform
file or information sharing, since it can handle network and vendor
variety with ease. Vendors such as Blue Arc, DVS and Maximum Throughput
are focused on providing high-performance NAS that meets production and
post requirements.

Maximum Throughput, for example, takes advantage of the already low
cost and relative ease of use of standard NAS technology while bringing
it up to the performance par of SANs. Mike Hughes, VP of business
development, points out that an extremely fast NAS device like the
company’s Sledgehammer (delivering 200 MB/s sustained throughput at
roughly $12 /GB) can provide enterprise-class performance at a fraction
of the cost of a fibre-channel SAN. For example, PostWorks, based in
NYC, installed a Sledgehammer NAS to move mundane preparatory and
administrative tasks like file distribution, storage and media encoding
away from their more expensive workstations like Discreet’s Inferno and
Quantel’s iQ. This reduced the number of steps it took PostWorks to
complete various post processes, decreasing inefficiencies and
increasing profitability.

Maximum Throughput’s executives saw video as simply another network
protocol over which data is delivered. "Right now, every facility out
there has to contend with two in-house networks- Ethernet and video,"
says Hughes. "But as both carry the same information, we’ve engineered
our Sledgehammers to store the media data once and offer it up
simultaneously, either as files from the NAS side of Sledgehammer or as
video from the DDR side. Essentially we’ve moved the video I/O spigot
into the NAS instead of feeding the data off to a separate stand-alone
DDR for playout to video."

For operations that need both speedy access to a common media pool and
collaborative functionality across multiple workstations, such as NLE
environments, a storage area network (SAN) tends to be the better
solution. SANs aim to combine the speed and performance of DAS with the
sharing, scalability and collaboration of a NAS. For this killer combo,
the complexity of the storage environment increases- you can expect to
pay somewhere between five and 10 times more compared to an equivalent
NAS. Much of this cost is a result of using the more expensive
fibre-channel technology, which offers faster throughput (2Gb/s per
fibre channel port or approximately 180MB/s) supporting up to as much
as 1050MB/s of sustained throughput today. But as technology matures
and new protocols like iSCSI and iFCP come into play, expect the
pricing difference between SAN and NAS to narrow.

As technology matures and new protocols like iSCSI and iFCP come into
play, expect the pricing difference between SAN and NAS to narrow.

Even so, in environments where a lot of data is manipulated and shared
across a range of production or post workstations, SAN systems offer
significant economic benefits, such as reducing the need to replicate
or move media among numerous workstations that require simultaneous
access. That can, among other things, reduce cycle times between each
step of the workflow. But for a SAN to be most effective, conventional
wisdom demands a homogeneous environment where all the parts to a SAN
are assembled from a single or limited set of vendors. It is not
uncommon for fibre-channel hardware from different vendors to be
incompatible with one another, and as different SAN vendors offer
incompatible network file systems, mixing and matching SAN vendor
components- unlike NAS- is asking for trouble.

Knowing this, Apple entered the SAN arena this spring with the
introduction of its own clustering software technology called Xsan.
Deploying the Xsan alongside its Xserve RAID, HBA (host bus adapter)
hardware and NAS-based metadata controllers, Apple seeks to take
advantage of the strong homogeneous environment it has already
developed with its complement of workstations, servers, OS and
applications. Additionally, Apple realizes that it needs to play well
with others and designed Xsan to communicate with Linux, UNIX and
Window workstations via a third-party network file system.

But what is really intriguing, according to Tom Goguen, Apple’s
director of server and storage software, is that "for about $70,000 a
customer could assemble five G5 workstations, all running OS X, all of
our professional media applications and all tied to a common Xsan,
enterprise-class fibre channel switch and 3.5 TB of Xserve RAID." When
Apple releases Xsan later this year, when they do, it expects to
deliver two streams of uncompressed HD (168MB/s) or 10 streams of
DVCPRO HD (15MB/s) shared storage per Xserve RAID when properly
configured.

Studio Network Solutions (SNS) provides SAN solutions that are
optimized for media environments typically found at customer locales
like Technicolor, Vivendi Universal and Abbey Road. SNS believes that
proprietary shared file systems are not necessary, instead relying on
native HFS+ ( Apple) and NTFS ( Microsoft) file systems and
partition-level access to storage (as opposed to file-level access) to
deliver the benefits of a SAN without the cost and complexity. Typical
file-level SANs like Xsan, CXFS and StorNext all require two networks
to operate effectively: one for managing the media (typically atop a
fibre-channel network) and another to manage the metadata (typically
atop an Ethernet network). SNS SANs can handle both the media and
metadata across a common fibre-channel network. An entry-level SNS
SANmp 3TB SAN, complete with all the necessary components to enable
shared storage between two workstations, starts at just under $20,000.

Even so, many companies cannot afford the expense associated with
implementing a dedicated fibre-channel SAN. SNS made a strategic
decision to support the recently ratified iSCSI protocol (SCSI over
TCP/IP), which enables Ethernet networks to act as a foundation for
delivering SAN functionality at an estimated 40 to 60 percent of the
cost of equivalent fibrechannel infrastructure, but with transfer rates
nearing those of fibre channel-350MB/s and up.

SGI, long a provider of DAS, SAN and NAS storage subsystems to power
users like Efilm, Universal Studios and The Orphanage, has staked out
the high ground where throughput and real-time handling of large files
are absolute requirements. The company offers a proprietary network
file system, XFS, for implementing very high-performance SANs. Jim
Farney, senior marketing manager for media broadcast and production and
a onetime facility owner, says, "XFS is a 64-bit file system that in
theory is able to massively scale files sizes up to 9 million TB and
file systems of up to 18 million TB."

To put that into perspective, a typical movie at 2K resolution is 2 TB,
meaning a single XFS file system could handle 9 million feature films
at the highest quality level used today. Although these astronomically
large numbers do not reflect real-world needs, SGI believes it has
created an architecture, called InfiniteStorage, capable of meeting any
current and future SAN needs. "The movement to a data-centric
environment is an arguable eventuality for all media facilities,"
Farney says.

A typical movie at 2K resolution is 2TB, meaning a single XFS file
system could handle 9 million feature films at the highest quality
level used today.

By using XFS and CXFS, SGI offers media companies a means to migrate
from legacy video-centric workflows into the rapidly growing IT-based
data-centric environment. An entry-level SGI SAN environment for five
to seven workstations with 3 TB of storage would typically start at
under $100K and can be expanded to handle a number of simultaneous
real-time uncompressed HD streams. Customers like Efilm, however, are
routinely working in higher-than-HD resolution, handling multiple
streams of uncompressed film from multiple film scanners simultaneously.

But the line between NAS and SAN performance and other benefits are
beginning to blur. New network protocols are being introduced, like
iSCSI, FCIP (Fibre Channel over TCP/IP) and iFCP (Internet Fibre
Channel Protocol), existing DAS storage technologies are being
optimized, like parallel ATA into serial ATA (SATA) or parallel SCSI
into serial SCSI (SAS), and Ethernet is being power-boosted with
next-generation 4GigE and 10GigE Ethernet NICs. It really pays to do
your homework before you settle on a solution.

Vendors like Bright Systems and Isilon take things a step further by
offering interesting alternatives for mixing and matching SAN and NAS
functionality. Bright Systems, a vendor for shared storage subsystems,
is also an R&D company that works with vendors like DVS, MTI Film,
Nucoda, Quantel, and Discreet to validate and certify interoperability
within the data world while maintaining interfaces to the legacy video
environment. Bright uses StorNext as a network file system for enabling
interoperability but goes beyond that by having manufacturers of RAID
equipment alter their firmware to better handle the real-time nature of
media as it is transported and stored on data-centric infrastructures.
Bright’s goal is to remove the inefficiencies associated with
shoehorning media into data-centric processes. Bright offers an
entry-level uncompressed 2K-capable SAN with two workstations that
starts at $55,000 with 8 TB of storage and includes professional
on-site installation. Bright’s goal is to deliver SAN systems and
functionality at NAS pricing.

Isilon uses industry-standard commodity components to minimize costs
for deploying its IQ clustered storage systems. "We believe that SANs
were never properly architected from the ground up to handle some very
key digital media workflows," says Sujal Patel, Isilon founder and CTO,
"so we created our own distributed file system called OneFS to merge
NAS and SAN functionality into a common storage pool." An entry level
three-node IQ 1440 system with 4.3 TB of raw capacity providing 1.7
Gb/s of throughput has a list price of just under $90,000, which works
out to be $20 /GB, including all the hardware and software.

Isilon’s customers include large facilities like Ascent Media,
Technicolor and Fotokem- facilities that need scalable, reliable and
high-performance network storage capable of handling multiple jobs and
multiple applications while accessing more than 10,000 media-sized
files within a single directory. Traditional SANs are generally unable
to handle these types of operational requirements.

No matter what your workflow requirements are, there is bound to be a
shared storage solution that will fit your bill. The process of moving
from sharing media across a traditional video infrastructure to one
based on an IT-based infrastructure will bring many benefits that are
only beginning to be fully understood. And while it will be a challenge
to implement, many vendors are already doing everything possible to
bring video functionality and cost benefits to equipment not normally
associated with moving media. It wouldn’t be surprising to find some
combination of SAN and NAS shared storage solutions in many future
production and post workflows.