This is part 4 of a larger article intending to introduce the reader to Avid’s NEXIS shared storage system. Read part 3 here.
In the earlier parts of this article I discussed some of the key aspects of NEXIS, discussed the NEXIS Pro in some detail and introduced the NEXIS Enterprise Engines. In this part I will examine the NEXIS Enterprise Engines in more detail.
A common theme which runs across the NEXIS Enterprise Engines relates to the level of hardware redundancy that is possible.
Each E Series Engine supports being fitted with a second Storage Controller module as an option. When an Engine is configured with two units, the Storage Controller modules work in an Active / Standby style of mode of operation. The module taking the standby role will monitor the health of the active module and take on the active Storage Controller role if required in the case of failure of the initially active module.
Each Storage Controller module connects to the Ethernet media network using one, and optionally, two transceiver modules located in the rear facing ports on each Storage Controller module. Any Controller with dual configured Ethernet connections uses Link Aggregation Control Protocol (or LACP) to negotiate the two network paths both to load balance traffic over the links and provide link redundancy. If configured with dual Storage Controllers, each E Series Engine can be effectively configured with up to four Ethernet links for a highly redundant setup that will survive the failure of a switch port, Transceiver module or NEXIS Controller module, or accidental disconnection or breakage of a network link.
Like the NEXIS Pro Engine each E Series Engine is also fitted with a pair of Solid State Drives to store the system state and system metadata at differing levels depending on the usage mode of the Engine. These SSDs are very high performance units tested for long life cycle Enterprise level performance and are configured in an Avid proprietary configuration to mirror the data across the two drives to ensure that the data can survive failure of one of the drives.
The media Drives used by NEXIS are SAS connected and the NEXIS Engines benefit from this by being able to provide dual paths between each drive and the electronics in each Storage Controller module. This means that the Engine itself is a passive “container” for the modular components and ensures that failure of one printed circuit traces on the mid-plane assembly where each module merges does not become a point of failure for the system.
And finally each NEXIS Engine has redundant power supplies at various levels meaning that with the correctly configured external power routing, and use of a Uninterruptable Power Supply (UPS) the system can survive blown fuses, power outages, failed power supply modules and inadvertent power cable removal.
NEXIS | System Director Appliance (SDA):
The NEXIS System Director Appliance or SDA is very similar in form to the NEXIS Pro and NEXIS E2 Engine. All three are 12 drive slot 2 Rack Unit (2 RU) Engines with slots at the rear for the two fitted Power Supply and Cooling (PSC) modules, and for the Storage Controller modules. As with the other 2RU NEXIS Enclosures the SDA Engine can run on one PSC module in the event of a failure. To improve on the redundancy provided in the NEXIS Pro, the SDA can be fitted with a second Storage Controller 20 module as an option. Each module connects to the Ethernet Media Network using SFP+ module type transceivers running at 10 Gb/s and each Controller can also have two SFP+ transceivers connected to the media network for up to four Ethernet links in total per Engine.
For the SDA, that’s where the similarity to other NEXIS Engine enclosures ends. The SDA does not use 10 of the 12 drive slots – its role in a NEXIS Enterprise system is to host the NEXIS System Director service and not to host or directly manage storage itself. It uses a combination of the two SAS attached 400 GB Solid State Drives (SSDs) and the NEXIS type 20 Controller module to host the NEXIS file system, to provide the management and monitoring of up to 48 Media Packs and Engines hosting them, as well as user management access rights and administration for up to 330 simultaneous users of the NEXIS system as well as accommodating account details for up to 5560 user accounts.
The SDA Engine will be needed by facilities wishing to have more than 40 simultaneously connected users in the NEXIS environments, those who wish to operate more than four NEXIS Media Packs or those who wish to use the NEXIS E5 Engine.
NEXIS | E2 Engine:
The E2 Engine is also a 12 drive slot, 2 Rack Unit enclosure with slots at the rear for two fitted Power Supply and Cooling (PSC) modules and for the Storage Controller modules. As with the other 2RU NEXIS Enclosures the E2 Engine can run on one PSC module in the event of a failure. To improve on the redundancy provided in the NEXIS Pro, the E2 Engine can be fitted with a second Storage Controller 10 module as an option. Each module connects to the Ethernet Media Network using SFP+ module type transceivers running at 10 Gb/s and each Controller can also have two SFP+ transceivers connected to the media network for up to four Ethernet links in total per Engine.
The E2 Engine can host either a single 20 Terabyte (TB) Media Pack (made from ten x 2 TB SAS HDDs) or a single 60 TB Media Pack (made from ten x 6 TB SAS HDDs). The Engine also hosts two SAS attached 400 GB Solid State Drives (SSDs) for the Media Pack metadata and which also host the System Director Metadata where the Engine is used in Foundation Mode to host the service. The Engine is capable of delivering up to 400 Megabytes per second (MB/s) storage access bandwidth to clients connected to the NEXIS system.
NEXIS | E4 Engine:
The NEXIS E4 Engine increases the drive concentration capability of the NEXIS system by being able to host up to 24 drives in 4 Rack Units of rack height. This enclosure also has slots at the rear for four fitted Power Supply and Cooling (PSC) modules and for the Storage Controller modules. The E4 Engine can run on two PSC modules in the event of a failure the Engine can be fitted with a second Storage Controller 20 module as an option. Each module connects to the Ethernet Media Network using SFP+ module type transceivers running at 10 Gb/s and each Controller can also have two SFP+ transceivers connected to the media network for up to four Ethernet links in total per Engine.
The E4 Engine can host up to two Media Packs of either 2 TB SAS drives or 6 TB SAS drives, and up to two Hot Spare drives. The Engine also hosts two SAS attached 400 GB Solid State Drives (SSDs) for the Media Pack metadata and which also host the System Director Metadata where the Engine is used in Foundation Mode to host the service. The Engine is capable of delivering up to 800 MB/s storage access bandwidth to clients connected to the NEXIS system when loaded with two Media Packs.
The E4 Engine will appeal to facilities who are looking to increase drive density and hardware redundancy where system up-time is vital. While it is potentially a more expensive Engine compared to the NEXIS Pro or E2 Engine, it offers the ability to connect two Media Packs with a single set of Ethernet ports which will reduce connectivity costs in terms of transceiver, cable and Ethernet switch port consumption, and so may be a better choice where a larger system is planned from the outset.
NEXIS | E5 Engine:
The NEXIS E5 Engine was introduced after the main release of NEXIS at the IBC Trade Show in September 2016 and delivered with the release of NEXIS version 6.2 software that month. According to Avid Engineers Kevin Usher and Raul Lobo on the IBC show floor, the design aim behind the NEXIS | E5 Engine is to deliver a very high density of high performance shared storage for use in busy media production environments. The unit is an enterprise facility class enclosure and it is intended to concentrate NEXIS storage bandwidth performance as efficiently as possible into a given rack space.
- The E5 Engine is an 84 drive slot enclosure consisting of two drive drawer assemblies capable of accommodating 42 3.5 inch SAS attached hard drive units in each drawer. The enclosure itself fits into 5 units of rack height (5 RU) and at a glance it looks very similar to the Avid | ISIS 2000/2500 Engine (but it is a different design in some small but important ways).
- Up to 80 (active) Media Drives are hostable in this engine, of either 2 TB or 6 TB capacity per drive, along with up to 2 hot spare drives. In the NEXIS storage management nomenclature, that equates to four to eight Media Packs and this allows for a lot of flexibility in terms of how the Engine can be deployed. Configurations from 80 TBs raw storage capacity (consisting of 40 x 2 TB Drives) up to 480 TBs (consisting 80 x 6 TB drives) of raw storage capacity per NEXIS | E5 Engine are possible depending on the drive capacity and the number of Media Packs loaded into the chassis. If starting with fewer than eight Media Packs, the configuration can be expanded easily by adding more Media Packs later, after the initial purchase of the system.
- The Engine has two System Drives consisting of 800 GB High Performance Solid State Drives (SSDs) which are used to manage the housekeeping and Media Pack level system meta-data for the Engine.
- The Engine is fitted with a NEXIS Type 80 Controller module, offering a QSFP+ type connection to the NEXIS system Ethernet Switch and which runs at the full 40 Gb/s speed. This is a new Controller module (and is not the same as the controller module fitted to the Avid ISIS | 2500 Engine). An optional Type 80 controller module can be fitted to the NEXIS | E5 Engine for Controller redundancy.
- The Engine is fitted with two 2200 Watt Power Supply Modules (the Engine can run on one PSU module in the case of failure) and five fan modules (the enclosure can run safely with a minimum of two fan modules).
The NEXIS | E5 Engine is sold in several configurations but in its base form it is supplied with 40 Media Drives (four Media Packs). You can then expand that by adding up to four more Media Packs per enclosure. You can also order an optional redundant Controller, and decide whether to purchase up to two spare media drives as options, too.
A key difference between the NEXIS | E5 Engine and the other Avid NEXIS E Series stable mates is that you must use a System Director Appliance (SDA) with this Engine from the outset. In other words, unlike other E series Engines, the NEXIS | E5 cannot host its own System Director service to enable it to run in Foundation Mode at all (even with the nominal limit for Foundation Mode of 4 Media Packs loaded into the Engine). For new systems, an SDA will need to be ordered along with the NEXIS | E5 Engine as part of the kit list. If a NEXIS | E5 is being ordered to augment an existing NEXIS system based on E-series Engines, and the system currently runs in Foundation Mode (up to four Media Packs) and does not already have an SDA, then again, a NEXIS | SDA will need to be ordered. For existing NEXIS systems which already use an SDA, then the E5 can be used alongside the existing SDA and the other NEXIS storage Engines.
So the NEXIS | E5 Engine is likely to appeal to organisations who perhaps want to start out with a larger capacity of storage from the outset (compared to that possible with other NEXIS Engines). It is also for those who need to optimize physical space usage in the data centre by concentrating a larger quantity of storage and or bandwidth capability into a smaller space. When fully loaded with Media Packs, the E5 Engine is capable of delivering up to 3200 MB/s aggregated read bandwidth performance to NEXIS storage clients using its 40 Gbit QSFP+ uplink into the Ethernet switch (yes that is Megabytes per second – not megabits!). To put that into perspective, four E4 Engines would be required to match the capacity of one E5 Engine but would consume up to 20 RU of rack space. Alternatively, three E5 Engines fully loaded would accommodate the current NEXIS | FS Extended limit of 24 Media Packs (where 24 E2s or 12 E4s would have been required). The NEXIS | E5 Engine will almost certainly appeal to larger enterprise organisations and facilities perhaps wanting to replace existing older ISIS 7000 systems or wishing to augment existing Avid ISIS Media Network based workgroups.
Note that the E5 Engine requires an available 40 Gb/s port on Ethernet switches. Avid have qualified the Arista 7280 SE-64, Dell F10 S4810, Dell S4048-ON and Cisco NEXUS 9372 Ethernet Switches in appropriate configurations to connect the NEXIS E5 Engine using its 40 Gb/s QSFP+ port(s) to the network. There is the possibility that some degree of network re-design will be necessary in order to accommodate this new enclosure in any existing media network.
While the NEXIS Pro and NEXIS E2 Engines allow Administrators to configure NEXIS Workspaces with levels of increasingly capable storage data protection, the fact that manual intervention in the form of a drive replacement will be required in order to trigger a repair, means that the Media Pack is potentially more vulnerable with these models to performance degradation while waiting for that intervention. The excellent performance monitoring provided by the NEXIS Administration system will warn users of when the system is running less than optimally and if carefully monitored will ensure that the impact of any degradation is kept to a minimum.
The value of being able to host spare drives in the NEXIS E4 and NEXIS E5 Engines then will not be lost on facilities who run time sensitive workflows. To ensure that data in Protected Workspaces is maintained with the highest possible level of data redundancy, the system will detect storage performance issues related to drive failure and automatically use an available spare drive to repair Media Packs hosted in the Engine in the background.