How Ethernet Hybrids are Blurring the Lines Between SAN and NAS Systems

NAS v. SAN. Ethernet v. fibre-channel. Decisions in the shared-storage
space aren't getting much easier to make as the options available
multiply and the power and performance relationships among technologies
shift. Fibre-channel used to be the power-users' connection of choice,
but the imminent arrival of 10 Gigabit Ethernet means it won't be the
fastest cable on the block anymore. And while your gut may say you need
a SAN for heavy-duty post work, there are NAS systems on the market
that demonstrate, convincingly, that's not always the case. In the
weeks leading up to NAB, and at the show itself, Film &
Video talked to a sampling of storage vendors about how they
help their customers sort through the issues. We asked power users at
Post Works in New York City and Post Logic in Hollywood, CA, to talk
about the moving target that is shared-storage nirvana. Finally, we got
engineer Ari Zohar Klingman to write an article that frames the debate
and details three different approaches to shared storage in three
different environments: Nice Shoes, Guava, and Freestyle Collective.
Read
his piece here.

One of the splashiest launches in recent memory was Avid's roll-out of
the Unity ISIS system last year. Unlike SANs, which are generally built
around fibre-channel technology, the ISIS is Ethernet-based. Each ISIS
engine comprises 16 ISIS storage blades (that's 8 TB of storage) and
two Ethernet "switch blades" that connect to a rack-mounted server as
well as to individual clients. More than two ISIS engines are
interconnected using what Avid calls an integrated expansion switch
blade, or iXS, which creates a network that routes any client to data
stored on any storage blade in the system with real-time efficiency.
The system is entirely built on Ethernet networking, which makes it a
better fit in existing IT infrastructures.

Avid's Andy Dale says the arrival of the ISIS creates a two-tiered
storage strategy for Avid, with the ISIS addressing users who need
unflappable, rock-solid performance. "The ISIS is really going after
that extreme level of reliability and availability, with very high
levels of scalability," he says. "It goes up to 64 TB right now, and
will be significantly higher than that by mid-2006." (At NAB, Avid said
ISIS will scale up to 192 TB and up to 150 active users, offering 300
50 Mbps streams in real time.)

The intelligence behind this stuff is apparent – yank out a few storage
blades and the system's performance won't even hiccup as it calmly
starts recovering from the failure by redistributing the data across
the rest of the system. But even the mighty ISIS is bandwidth-limited.
The IT-style flexibility that comes with Ethernet networking means it's
constrained, for the time being, by the bandwidth of Gigabit-Ethernet
connectivity. That means that if you want to work in HD, you're either
going to need compression or something like an Avid Unity MediaNetwork,
which runs over four-gigabit fibre-channel. Dale, of course, recommends
using Avid's high-performance DNxHD codec to save money. "When you're
talking about 27.5 MB/sec per stream rather than 156 MB/sec per stream,
the overall required investment in storage and networking falls
dramatically," Dale says. "It's a very good tool even if you're working
in lower resolutions – DNxHD 115 is extremely good quality for dailies
and things like that."

So in terms of the user experience, the Isis is part Ethernet-connected
NAS and part robust, powerful SAN. And Avid isn't the only vendor
blurring the lines. Bright Systems, for instance, has a new line of
servers called BrightDrive. They allow facilities to get bandwidth
where they need it by connecting users who need high-resolution
real-time capabilities via fibre-channel (they'll see the BrightDrive
server as a local drive), and hooking up the rest of the facility via
less expensive Ethernet connections (which will see the server as a
network drive). The idea is to make a scalable system that leverages
the cost-efficiency of Ethernet with the need for fibre bandwidth.
"It's no more expensive than a NAS-only solution," says Bright's Peter
Lambert. "There's nothing that you throw away, no penalty along the
line [as you increase performance]."

Wayne Veitschegger, chief engineer at Post Logic in Hollywood, CA,
subscribes to similar logic, combining online and nearline volumes from
Bright to keep storage and bandwidth capacities right where he needs
them. “We have three fibre-channel volumes, which provide us with nine
theoretical 2K streams, and then one volume of SATA storage for
cost-effective nearline storage, which is non-real-time,” he explained.
“Each fibre-channel volume is easily expandable to add more spindles,
adding storage and speed as we ramp up for 2K, which is what it’s
optimized for. We have certain islands that are capable of real-time
4K, and, by adding another array, each volume can be moved up to
real-time 4K streams using 4 gigabit fibre. The main infrastructure is
running 2k, but with very little infrastructure improvement or
additions, we can ramp up to 4K really quick.”

Here's how Post Logic does DI work: First, the film is scanned directly
into the BrightSystems SAN, and a quick QC of the raw scan is performed
using such tools as a directly attached DVS Clipster that lets the
scans be previewed sequentially in real time. Depending on the project,
Post Logic may next take the scans into the iQ system for a conform
check, depending on whether the material has already been conformed
offline. At the same time, files will be copied to local storage on
Post Logic's Baselight systems, where the dirt-removal process starts
using MTI Film's DRS system. When that's finished, the files are placed
back onto the SAN. Color-correction can be started at any time.
Screenings often take place out of the Clipster or from a Baselight
system. "Our digital projector is on the video router, and we can
screen from any source in the facility," Veitschegger says. Screenings
take place on a Barco DP100 running at up to 2048×1080 in 4:4:4 or
4:2:2 color space, depending on the project.

"Post Logic is set," Veitschegger says. "Knowing our action plans for
the future, we're good – unless something drastic changes."

System designer Studio Network Solutions, meanwhile, is a big believer
in iSCSI, designing systems that behave like fibre-channel SANs but
work over Gigabit ethernet. SNS uses iSCSI interfaces to scale up
through SD workloads, and then bring in fibre-channel only when
necessary to handle HD. It makes sense that SNS might have a different
approach to shared storage – the company came from the audio space,
where it gained a reputation by building custom fibre-channel systems
for recording studios. About two years ago, SNS started deploying iSCSI
systems for sound editing applications, and is now looking to extend
its success into video post.

"We think the hybrid of fibre-channel and iSCSI is really promising,"
says SNS Director of Operations Eric Newbauer. "It's cost-effective,
and everyone has CAT5e ethernet or CAT6 in their walls. They don't all
have fibre-channel or optical cable. And we can now do things over
iSCSI that used to only be possible with fibre-channel." So you can
leverage your existing Ethernet experience and infrastructure in your
SAN. And the company believes that iSCSI over 10-Gigabit Ethernet could
make 4-gigabit fibre obsolete purely on the basis of speed. (Newbauer
thinks affordable 10-Gigabit Ethernet solutions will start apearing "in
about 24 months.")

For offline editorial applications, Newbauer stacks the GlobalSAN
directly up against Avid's Unity and Apple's Xsan. "Spending $45,000 on
a fibre-channel SAN to do offline video is totally unnecessary," he
says. "A GlobalSAN solution from SNS starts at $7,000, and it'll
effortlessly handle multiple streams of DV." A 9.6 TB eight-user SAN
goes for less than $30,000.

But others disagree about the benefits of systems that are built around
IT-friendly technologies like Ethernet and iSCSI. "We've been shipping
four-gigabit fibre-channel for well over a year, and that's the best
bang for the buck right now," says James McKenna, VP of sales and
marketing for Facilis Technologies. "Treating a data pipe like a coax
cable and not having to worry about the networking complexities is the
way to go for the post customer."

Facilis is targeting users who need a SAN, but see systems like Avid's
Unity as an expensive way to get there. "We start out at the
lowest-priced SAN solution available: under $10,000 for a couple of
terabytes of capacity with decent performance," says McKenna. From
there, it's a question of when to goose bandwidth by adding more
servers. Once you get up to as many as 24 drives for a multistream
uncompressed-HD-capable SAN, you're looking at an investment of
somewhere between $16,000 and $26,000.

"For the bulk of our customers, having six, eight or 10 users running
SD uncompressed shouldn't be a problem on a single server, as long as
you have 24-drive, or multiple 12-drive, units," McKenna explains. "30
or 40 TB is not uncommon – Post Works has a 43 TB installation that
does a mix of HD dual-stream connectivity from Symphony Nitris and 2K
DI work with Assimilate Scratch and the Quantel iQ. It's a big
environment that we're the hub of as a storage solution. That's what a
lot of facilities will need as 2K and 4K become the formats of choice.
You don't want to be stuck without the ability to hold onto a couple of
reels in 2K and access them in real time. We offer 2K accessible
storage even with the 12-drive unit."

Post Works has Avid Unity systems that take care of offline and online
SD editorial work, but uses local storage for its HD and 2K work to
gurarantee performance at those resolutions. The Facilis SAN uses SATA
storage and shares out on fibre-channel, enabling real-time 2K
playback. "We can use that to preview some of the frames we scan at
real time, which is nice, but we primarily use it as fast copy space,"
explains Corey Stewart, director of systems engineering. "Our film
scanner has local fibre-channel storage, so once we fill up that
volume, we copy the media up onto a [Facilis] Terrablock, and the media
will live on that one volume for the duration of that job. We'll keep
it there until we're finally signed off on it and we can delete those
original scans."

For 2K projects, Stewart says, the scans are imported to an iQ from the
Terrablock, which has its own disk set and file system. When the
picture is locked, the reel is archived to another Terrablock
volume.

"We've been shying away from buying into any big SAN based on
fibre-channel storage, like a Bright Systems solution, at the moment,"
Stewart says. "The file-system limitation based on performance is
really the issue. You have one file system per grouping of
fibre-channel RAID controllers, or a JBOD set, and you can't really go
above that. If you need to use the same media that's on that file
system across three or four workstations reading and writing 2K at the
same time, it's just not possible. That's the limiting factor – and
obviously the cost, as well."

There's another category of shared-storage provider that specializes in
optimizing the various solutions that exist. Archion, for instance, has
a RAID product called Synergy Plus that basically piggybacks on an Avid
Unity system, offering fully redundant storage at a cost savings over
fully mirroring data on the Unity. "It's transparent to Unity," says
Archion CTO Jim Tucci, "and it takes over drive management from Unity,
looking at bad blocks, etcetera. Three drives can fail in one unit with
no data loss."

And Exavio was at NAB showing its Examax SAN acceleration system – one
demo ran four simultaneous 2K streams out of a single CXFS file system.
Exavio gets extra performance out of SANs partly by striping data
across multiple RAIDs in hardware, not software, and aggressively
pre-loading material. "On the back end, we're coalescing the reads and
writes – when we do writes, we do very big writes, and then we do it
opportunistically" by storing data in a RAM cache, explains Jim Farney,
the company's director of market development. "We're working with
file-system vendors and with applications people to get project
awareness. Customers ask if we can connect to ScheduAll, or to Xytech
[workflow and asset management systems]. We'll be able to in the
future."

Of course, some in the industry argue most users don't really need a
SAN at all. “The biggest misconception among our customers is that they
need a SAN,” says Maximum Throughput’s Giovanni Tagliamonti. “There’s a
lot of evangelizing about SANs. Customers say, ‘I need a SAN.’ And,
when you break down their needs, they don’t need
one.”

What’s Tagliamonti’s angle? For one thing, his company makes the
Sledgehammer, which is a network-attached storage device that’s
designed to make the traditional digital disk recorder redundant. It’s
a purpose-built system for post applications, running software that
handles video IO and format conversions. You can scan film directly
into a Sledgehammer and make it available immediately over your
network. And with the new dual-stream play and record capabilities
added at NAB, dropping a Sledgehammer into a traditional
color-correction workflow has interesting implications – for instance,
a Da Vinci system can control the Sledgehammer as if it were a playback
and recording deck. “The Da Vinci 2K owns that market, and it’s still a
linear system,” Tagliamonti says. “You take one of our boxes and slap
it in the middle of that process, and you’ve now converted it to a
nonlinear process. You’ve also made the files available on your
network.”

Tagliamonti argues that only about five percent of post-production
clients really need a SAN rather than a NAS system. “If you need
massive bandwidth to a single client, a NAS can’t do it today,” he
says. “You can only talk to a NAS over a NIC, and only at 120 MB/sec.
If you need four-gigabit HBA, you need a SAN. Where I need a big pipe
to a single client is on my video network, and as far as our box is
concerned, you come in to your video card over a big pipe. You may need
a lot of aggregate bandwidth – but why do you need a big pipe to a
single client?”

And, when asked about the future of shared storage, Tagliamonti echoes
some of Newbauer’s arguments against the primacy of fibre-channel SANs.
“I am from the school that says Moore’s Law is about to take care of
all your [bandwidth] requirements,” he says. “10 Gigabit Ethernet is
here, and the line between what a SAN is and what a NAS is is so
blurred today. A SAN is designed to fit behind data-based servers, and
there you absolutely need one. Sitting behind an Oracle database or any
ERP [enterprise resource planning] system, with two or three servers
talking to a SAN? Yeah. But in our customers’ environment, you don’t.
You want clustered NAS.”

“The important stuff,” says McKenna of Facilis Technologies, “is going
to come with the ability of a central-storage system to service
multiple clients without the need to measure bandwidth across different
drive sets and watch your client loads. Avid’s ISIS product uses that
clustering mentality, with servers acting together to up the bandwidth.
We’re going to be selling a lot more smaller-capacity servers in the
future, because, spanning them together, you end up with larger
bandwidth than you can get even with our largest, 24-drive server. Not
having to deal with load-balancing across different drive sets, or
putting clients in different places to avoid stepping on toes, will
become important.”

“The Holy Grail is some sort of clustered system like the Avid ISIS,”
agrees Stewart of Post Works. “That is the be-all and end-all.
Something that will scale linearly. You’ll be able to throw extra nodes
onto it and have it spread data across all the nodes, and [speed won't
be an issue] as long as one node will have the connectivity to be able
to do the bandwidth you require. Right now, we don’t ask for much, just
2K. But down the line, 4K will be possible. It will be interesting to
see where those clustered SANs come from and how we can use them in our
environment.

“The connectivity to that cluster is really key. What is going to be
the format for everybody to play with? And if no one picks a format, is
this cluster going to be able to share out multiple formats –
fibre-channel, InfiniBand, 10-Gigabit Ethernet, or whatever the new
thing is six months from now?”