16GFC Fibre Channel is 16-Fold Better Than 4GFC? Well, All Things Considered, Yes. Part I.

16GFC == 16X
If someone walked up to you on the street and said, “Hey, guess what, 16GFC is twice as fast as 8GFC! It’s even 8-fold faster than what we commonly used in 2007” you’d yawn and walk away.  In complex (e.g., database) systems there’s more to it than line rate. Much more.

EMC’s press release about 16GFC support effectively means 16-fold improvement over 2007 technology. Allow me to explain (with the tiniest amount of hand-waving).

When I joined the Oracle Exadata development organization in 2007  to focus on performance architecture, the state of the art in enterprise storage for Fibre Channel was 4GFC. However, all too many data centers of that era were slogging along with 2GFC connectivity (more on that in Part II). With HBAs plugged into FSB-challenged systems via PCIe it was uncommon to see a Linux commodity system configured to handle more than about 400 MB/s (e.g., 2 x 2GFC or a single active 4GFC path). I know more was possible, but for database servers that was pretty close to the norm.

We no longer have front-side bus systems holding us back*. Now we have QPI-based systems with large, fast main memory, PCI 2.0 and lots of slots.

Today I’m happy to see that 16GFC is quickly becoming a reality and I think balance will be easy to achieve with modern ingredients (i.e., switches, HBAs). Even 2U systems can handily process data flow via several paths of 16GFC (1600 MB/s). In fact,  I see no reason to shy away from plumbing 4 paths of 16GFC to two-socket Xeon systems for low-end DW/BI. That’s 6400 MB/s…and that is 16X better than where we were even as recently as 2007.

Be that as it may, I’m still an Ethernet sort of guy.  I’m also still an NFS sort of guy but no offense to Manly Men intended.

A Fresh Perspective
The following are words I’ve waited several years to put into my blog, “Let’s let customers choose.” There, that felt good.

In closing, I’ll remind folks that regardless of how your disks connect to your system, you need to know this:

Hard Drives Are Arcane Technology. So Why Can’t I Realize Their Full Bandwidth Potential?

* I do, of course, know that AMD Opteron-based servers were never bottlenecked by Front Side Bus. I’m trying to make a short blog entry. You can easily google “kevin +opteron” to see related content.

Filed under: oracle