INFINIBAND HYBRiB STORAGE
Finally our HYBRiD solution that offers ultra fast flash speeds at a portion of Fibre Channel, Combine the speed of flash with the cost effectiveness of spinning hard disks with the speed of Infiniband, Add in the robust and resilient file system of ZFS and you have one of the best shared storage networks for Post Production and Creative Content.
Today’s flash far outperforms hard drives for throughput, latency, IOPS, power consumption, and reliability. With fast storage, your physical network and your network protocol must have high bandwidth and low latency, otherwise you’re wasting much of the value of flash. Tomorrow’s NVMe devices will support up to 2-3GB/s (16-24Gb/s) each with latencies <50 us (that’s <0.05 milliseconds vs. 2-5 milliseconds for hard drives). Modern Ethernet supports speeds of 100Gb/s per link, with latencies of several microseconds, and combined with the hardware-accelerated iSER block protocol, it’s perfect for supporting maximum performance on non-volatile memory (NVM), whether today’s flash or tomorrow’s next-gen solid state storage.
The “old” Ethernet network ran at 1Gb/s or 10Gb/s speeds and relied on TCP to deliver data, which was reliable but somewhat unpredictable. But today’s Ethernet runs at 25, 40, 50, or 100Gb/s speeds, is lossless, and no longer dependent on TCP alone. It supports RDMA connections which lower latency and frees up CPU cycles to run applications (or storage features). Ethernet easily supports multiple storage protocols—block, file, object, etc.—simultaneously, and allows client, server, and storage traffic to share the same network, using with traffic prioritization and QoS.
What is our HYBRiD Solution?
OS – REHL 7.2 File System – ZFS
Networking – Infiniband EDR 100Gb/s / FDR 56Gb/s
Ethernet 1GBe / 10GBe / 40GBe
Infiniband Sharing Protocol – RoCE – SMB Direct / NFS over RDMA
Infiniband Block Level – iSER (Infiniband iSCSI with RDMA)
Ethernet Sharing Protocol – SMB / NFS / CIFS / AFP
Ethernet Block Level – iSCSI
Proprietary Hardware/Software – NONE
Hybrid Storage is to provide high performance at an affordable price. It address’ the dollar-per-IOPS vs dollar-per-GB decision that you have to grapple with. In addition it best leverages flash by using flash as a large caching area, automatically moving the most active data into memory-based storage.
But How Fast is Fast?
Using 8 NVMe Flash Drives as a Cache returns over 20,000 MB/s & 112,000 IOPS, Thats not a type error. Up to 16 drives can be installed giving you a total of over 40,000 MB/s & 224,000 IOPS of read cache that can be utilised for 5-6 UHD DPX streams simultaneously. Can FC do that?
Sounds Expensive, How Does FC compare?
At roughly have the price of a 32GB/s FC switch, Infiniband wins the price battle with over 3 times the performance.
Can we add our Mac’s & Some other hardware?
Of course, See the diagram below on our recommended configuration. The storage can be accessed through 1GBe or 10GBe just as if its been through Infiniband.
RDMA over Converged Ethernet
Direct Memory Access (DMA) is an ability of a device to access host memory directly, without the intervention of the CPU.
Remote Direct Memory Access (RDMA) is the ability of accessing (read, write) memory on a remote machine without interrupting the processing of the CPU(s) on that system.
Zero-copy – applications can perform data transfers without the involvement of the network software stack. Data is sent and received directly to the buffers without being copied between the network layers.
Kernel bypass – applications can perform data transfers directly from user-space without kernel involvement.
No CPU involvement – applications can access remote memory without consuming any CPU time in the remote server. The remote memory server will be read without any intervention from the remote process (or processor). Moreover, the caches of the remote CPU will not be filled with the accessed memory content.
Because it is an enterprise-ready open source file system and volume manager with unprecedented flexibility and an uncompromising commitment to data integrity. ZFS is a truly next-generation file system that eliminates most, if not all of the shortcomings found in legacy file systems and hardware RAID devices. Once you go ZFS, you will never want to go back.
Uncompromising Data Integrity
Unprecedented flexibility is pointless without dependability. ZFS is uncompromising about data integrity and data redundancy. ZFS creates a chain of trust for your data by checksumming data when it is written and verifying the checksums when it is read. It even checksums the metadata that describes the file system and allows you to periodically verify all checksums to determine if your infrequently-used data or backups are suffering silent data corruption a.k.a. “bit rot”. By taking these precautions, ZFS will never return you corrupt data from disk.
Knowing your data is intact is one thing but combating disk unreliability is quite another. ZFS can maintain data redundancy through a sophisticated system of multiple disk strategies. These strategies include mirroring and the striping of mirrors equvalent to traditional RAID 1 and 10 arrays but also includes “RaidZ” configurations that tolerate the failure of one, two or three member disks of a given set of member disks.
ZFS helps you avoid most storage planning mistakes by pooling together the blocks provided by all of your disks and allowing you to divide the available space into highly-configurable file systems. ZFS includes the permissions and quotas of traditional file systems but also includes transparent compression levels, capacity reservations, and clonable snapshots. Of these features, snapshots will probably be the one that forever changes the way you think about storage. Imagine if you could bring “what if?” style experimentation to your storage…