The classic trade-off in storage architecture, especially when it comes to media, is between performance and capacity. With the advent of Flash arrays, you can crank up all the performance you need for a demanding film project. Whether it makes sense to pay for it, or whether you can afford to wait while files transfer from your existing infrastructure to the faster system, is another story. 

Those questions were facing VFX studio Atomic Fiction last year, when it had to quickly build out its storage infrastructure to solve sluggishness related to rendering and other demanding tasks on the VFX-heavy feature project The Walk. Initially, the obvious solution seemd to be simple: buy faster storage. But it turned out to be more important to get better performance in place right away, rather than waiting for 60 TB of files to be moved from slower to faster storage devices.

That's where NAS acceleration came in — the studio needed better performance without completely replacing its storage infrastructure.

The Walk

"Atomic Fiction had an existing storage array but they were looking to upgrade to the next generation," said Avere VP of Marketing Rebecca Thompson. "They found that getting the performance they needed would require a huge outlay. A smarter choice on a project deadline was to put a bunch of edge filers in front of their existing arrays."

Avere FXT edge filers

An edge filer is a high-performance storage appliance that sits between NAS storage infrastructure and the client workstations making demands on those drives, monitoring what data is being requested and how often. Data is automatically moved to and from the edge filer, depending on how  high the demand is from users. And once the data has been relocated from existing NAS, it's part of a smaller storage pool that can be accessed much more quickly. Think of it as a scalable caching system for NAS hardware.

"The idea is that out of any given workset, including data for visual effects rendering, there's only a certain percentage of data active at one time — the working set — and all of that can be held in the Avere devices,"  Thompson told StudioDaily. "It's not only an edge filer but a clustering architecture, so you can keep adding [devices] until you get the high-performance capacity you need to hold that active working set, and it takes all the load off the capacity that it's attached to."

The Avere system gave Atomic Fiction an immediate advantage, decreasing response times and increasing overall performance by three to five times, the studio reported. It's efficient enough that, after The Walk​, Atomic Fiction also used it to finish this year's Deadpool. Today, the company has clustered Avere FXT edge filers acting as front ends to more than 90 TB of NAS at its Oakland and Montreal studios, serving data to rendering as well as 155 artist workstations.

Deadpool VFX breakdowns courtesy Atomic Fiction

The Avere FXT edge filers talk to clients via standard NFS or SMB protocols. In other words, workstations see them as core filers, rather than devices existing separately from the facilities' main storage capacity. They can also work as a fast gateway to object-based storage on premise or in the cloud. Scalability is immense — up to 50 devices can be connected to create a huge pool of flash storage (to a maximum of 450 TB) in a single Avere cluster, which can connect to up to 25 core filers.

The FXT comes in three different models, starting with the FXT 3200 (96 GB of DRAM, 2 GB of NVRAM, and 4.8 TB of SAS HDD storage), and scaling up through the FXT 5400 (256 GB of DRAM, 4 GB of NVRAM, and 4.8 TB of flash SSD storage) to the FXT 5600 (384 GB DRAM, 4 GB NVRAM and 9.6 TB of flash SSD storage). The more DRAM and NVRAM is on board, the better the read/write performance.

While Avere serves a "huge contingent" of VFX customers who need the devices to accelerate rendering performance, Thompson said the current trend toward 4K is making edge filers more popular as additional pressure is put on existing storage hardware. She said facilities that do a lot of transcoding, or need a flexible archive solution, can benefit, too.

"A number of folks use us more for file-to-object translation, replacing their traditional NAS with [Amazon Web Services] S3-type storage, making a dense and inexpensive object store for media archival use," she explained. "If you're a studio and you just have dozens of years of media files you need to store, you don't need everyday access to them, but you still want them presented as a file system and you want a cheaper storage system. And our lower-cost models are sometimes used by small studios in house, or out in remote locations for WAN usage when they don't need a ton of flash storage and the issue is more WAN latency."