It is not a Good Plan to Follow-up PCIe 4.0

The PCI-SIG (PCI Special Interest Group) announced PCIe 4.0 will double PCIe 3.0 performance and--sit down for the next part--it will arrive in 2015 or 2016. Given how late PCIe 3.0 was, we thought that 2016 would be optimistic at best. So from 2004 with PCIe 1.0 to 2016, we will go from 250 MB/sec per lane to 2 GB/sec per lane or a factor of 8x in 12 years. 

Moore's Law is certainly not in play for PCIe. During that time, everything else has increased far more than 8x. In 2004, we had LTO-2 at 35 MB/sec; today, we have tape drives well over 240 MB/sec. Surely, we will have another generation faster. We have faster memory, faster CPUs faster communication with 10 GbE and 40 GbE coming (100 GbE depends on PCIe 3.0). The only thing in the stack that is not that much faster is spinning disk, but we of course have flash SSDs that are far more than 8x faster.

The message from the PCI-SIG is basically saying, "Look, we are doing just a great job getting you 2x performance by 2015 or 2016." We see this as a major problem. PCIe is required for all communications, and it just is not fast enough as things scale out, whether for storage or for network communication. It is time for the PCI-SIG to realize we need something much faster. 2x every four years is not fast enough, and we are already pretty far behind since PCIe 3.0 was so late that it is not even here yet.

We want to do a bit more ranting about PCIe 4.0 plans. Having a doubling of performance between now and 2015 will stifle innovation. Maybe that is the plan--have everyone think you are happily moving down the PCIe yellow brick road and then do something different. Something must be done to improve communications between CPUs. It has become abundantly clear that building memory systems with more than 4 sockets get very expensive, given the engineering design costs.

However, there are many problems that require many CPUs. Things like search and index have used Hadoop and connected lots of cheap CPUs. But many complex scientific and engineering problems cannot decompose problems such that communications is not a barrier to getting the work done. You are not going to run a complex earthquake, atmospheric or other simulation today without high-speed communications. Doubling performance with IB by 2015 (my bet is at least 2016) is not doing to do it for advancing science, unless of course everyone makes some breakthroughs in the area of algorithms that do not require communications. We are thinking that some vendor has something up its sleeve, and presto--we will have high-speed communications outside of the PCIe framework. While search engines do not depend on this, scientific advance does.

We are hard pressed to believe that some of the major vendors have not realized that and are holding back to see if they can gain some market advantage. Since one of the Chinese supercomputers had its own interconnect not requiring PCIe, it is clear to me that someone will get it. Sopto can't believe that the major CPU vendors do not. The new IBM P775 and current Cray XE/6 systems have addressed this problem.

Sopto is professional in manufacturing and suppling PCI-E Server Network Interface Cards. For more information, please contact a Sopto representative by calling 86-755-36946668, or by sending an email to info@sopto.com.