This is part 2 of a larger article intending to introduce the reader to Avid’s NEXIS shared storage system. Read part 1 here.
So, where does NEXIS improve on Avid’s previous generation storage systems?
Avid|NEXIS takes several significant steps to improve on its previous stable mates. Here is an overview of what has been done to warrant the Next Generation moniker:
- A new series of storage enclosures (or “Engines”) has been introduced by Avid giving the customer choices over the size and number of drives hosted by an Engine, the level of redundancy offered by the Engine (in terms of power supply & cooling modules, Controller modules, and Ethernet ports), and whether hot spare drives can be hosted by the Engine.
- Drives are managed by the NEXIS system in groups of ten drives referred to as a Media Pack. The NEXIS Engine hosts one or more Media Packs and is charged with the responsibility of tracking the contents of each Media Pack it hosts using a carefully maintained set of data referred to as the Media Pack metadata.
- To increase the client access bandwidth possible to the Media Drives, NEXIS separates the storage of the Media Pack metadata from the Media Data stored on the Media Drives. Mirror protected Solid State Drive (SSD) technology is used as the storage tier to hold the Media Pack metadata thus increasing the client access bandwidth capability of the Media Drives by some margin (compared to previous technologies employed by Avid).
- Enterprise class rated 7200 RPM Serial Attached SCSI (SAS) drives are used as the Media data storage drives to take advantage of having two possible paths to the drive controller (SATA drives used in previous products only have one path). This enables the storage chassis to optionally have two system controllers fitted for redundancy purposes. Media drive capacities of 2 TBs and 6 TBs are being offered by Avid.
- By employing a Software Defined Storage strategy NEXIS is not reliant on manufacturer specific or hardware implemented RAID schemes to protect the data stored on the Media Packs. The new Avid designed NEXIS file system offers a multi-faceted approach to transacting data to and from the component drives in the Media Pack which in turn offers several key advantages in terms of data redundancy and performance over previous technologies used both by Avid and by other vendors:
- The data redundancy scheme used by NEXIS offers two distinct protection levels to ensure that data stored on the Media Pack is protected against one drive failure or two drive failures inside the Media Pack, as well as a third mechanism called Media Mirroring which protects against a storage chassis becoming un-available. Users can also choose not to protect the storage if the data is duplicated on storage elsewhere and the simple employment of raw storage capacity is required (the data is backed-up on LTO tape for example). The levels of protection offered by NEXIS can be selected on a Workspace by Workspace basis. As with Avid’s previous shared storage products, Workspace is the name Avid give to the user accessible logical volume in NEXIS. (The ability to dynamically modify the Workspace Protection level as production needs change over the life cycle of the media is thought to be a feature that Avid may add in a future version of NEXIS).
- The NEXIS data redundancy scheme takes a new dynamic approach. When writing data to the media drives, the quantity of blocks of data into which files are broken into and then grouped, prior to actually being written to the storage, can effectively be throttled depending on the size of the files being written to the storage. This dynamic method consumes data capacity more efficiently when there are quantities of smaller files to be stored but also allows the system to optimise the input/output (I/O) performance of the drives for larger contiguous data transactions typical of larger media files.
- A further advantage of the new data redundancy scheme employed by NEXIS is the shortening of recovery times in the event of drive failure. The time taken to rebuild the storage in the event of drive failure can be a problem for large workgroups since the storage is vulnerable to further drive failure during drive rebuilds. Also the process to rebuild the missing data will inevitably have an impact on the performance of the storage which can potentially affect the number of users which can access the storage at normal levels of usage. With traditional hardware implemented drive data protection systems, the system controlling the drives typically needs to process and rebuild all of the drive storage capacity of the failed drive in the background onto the incoming replacement while I/O operations continue from users. This is the case even if areas of the failed drive contain no actual data at all – even that potential but empty data capacity structure needs to be rebuilt which effectively extends the time required for the rebuild. With these systems a failed but half-filled data drive takes as long to rebuild as a failed drive which is full! With the NEXIS data redundancy scheme, a faster recovery is possible in the event of a drive failure because the system recreates and redistributes only the missing data from used data blocks onto the replacement drive thus reducing the potential vulnerability of the system and the duration of any potential impact to users.
- Like its predecessors, NEXIS uses a service known as the System Director to manage the storage environment. This service acts as the file system’s “front of house” and is the service which performs the fundamental file system role as well as the system housekeeping and monitoring. Compared to other file systems, the System Director’s concierge-like functionality enables ISIS and NEXIS to work in what Avid refer to as a Server Assisted mode of operation for the underlying storage. In this way storage clients never find the System Director a bottle-neck to accessing the storage and the system is capable of servicing access to storage for many storage clients in very time sensitive high performance scenarios. A new feature for NEXIS is that NEXIS offers a flexible approach for running the System Director service.
- For small systems of up to four Media Packs, one of the NEXIS Engines can be used to perform the System Director functionality for the system hosted from that Engine’s Controller module. In this mode, the system is said to be running in “Foundation” mode and is limited to up to 40 users, to storage environments up to 240 TBs in raw capacity (4 x Media Pack x 10 drives each pack at up to 6 TBs in capacity), and with a total number of 8 million files being trackable in the system.
- For larger systems a separate platform can be dedicated to the role of running the System Director service. In this case the NEXIS System Director Appliance (or SDA) is offered to increase the scale of the system. The SDA enclosure can have up to two controller modules fitted for full redundancy.
- When deployed, the SDA places the NEXIS system into “Extended File System” mode where up to 165 users can use the system simultaneously, up to 24 Media Packs are manageable (with a raw capacity of 1440 TBs of storage), and up to 20 million files are trackable in the system.
- A further license can be purchased for the SDA to increase the scale of the NEXIS system beyond Extended mode. The NEXIS Advanced File System mode enables the possibility of the System Director to manage up to 48 Media Packs (with a potential raw capacity of up to 2.88 PBs of raw storage capacity) for an environment where up to 330 clients can use the storage simultaneously.
Please see part 3 of this article to continue reading.