AMD Unveils EPYC 7000 Series Processors And Platform To Take On Intel In the Data Center
AMD EPYC 7000 Series Infinity Fabric
Infinity Fabric is the name AMD is giving to the interconnect technologies it is using to link not only the individual pieces of IP within a processor, but between the dies, sockets, and other I/O.
AMD created Infinity Fabric because it needed a new approach to scale its products, beyond moving to new process nodes. AMD needed to increase performance and efficiency, scale bandwidth, improve latency and QoS, and it required flexible, coherent interfaces across CPU and GPU cores, and the other functional blocks within the chips.
Infinity Fabric is comprised of two key elements: a scalable control fabric and a scalable data fabric. The scalable control fabric has all of the central control elements, with small remote elements that are dispersed in each different block of the SoC. And feeding into the control elements across the fabric is the data from a myriad of sensors embedded across the SoC. The scalable data fabric is much like a high performance network pathway. It features a common bus, with low latency, and a coherent Hyper Transport Plus bus, that’s multi-socket and multi die-ready. Infinity Fabric is the key element that AMD uses to link the four, 8-core dies together on a single EPYC CPU package, and is also how it links the sockets.
The individual dies on each EPYC 7000 series processors, and the sockets in a 2P system, are all linked via Infinity Fabric. The connections between the dies is fully coherent within the socket, and is optimized for low power and low latency. The connections between the sockets leverage Infinity Control Fabric. There is 42GB/s of bi-directional bandwidth between the on-die links, for a total of 170GB/s. And the bi-directional links between the sockets offer up to 38GB/s, for a grand total of up to 152GB/s. Because each CPU die is connected to its peer in the second socket, there is a 2-hop maximum between the processors. As we've seen, there is higher latency involved when threads bounce between the quad-core CCXes in on-die, so it is safe to assume there will be even higher latency when bouncing between CCXes in different sockets, but AMD is claiming they have optimized these links for low latency.
The eight total memory channels on an EPYC 7000 series processors come by way of dual memory channels per die; there are four dies per MCM, two channels per die, hence eight total memory channels. Infinity Fabric connects all of the dies and the memory together into a coherent memory pool, with up to 171GB/s of aggregate bandwidth.
All told, there are four bi-directional links between the dies on each processor, and another four bi-directional links between the sockets. This arrangement is what allowed AMD to leverage what is essentially the exact same 8-core die across every Ryzen 5, Ryzen 7, Ryzen ThreadRipper, and EPYC 7000 series processor.
AMD created Infinity Fabric because it needed a new approach to scale its products, beyond moving to new process nodes. AMD needed to increase performance and efficiency, scale bandwidth, improve latency and QoS, and it required flexible, coherent interfaces across CPU and GPU cores, and the other functional blocks within the chips.
Infinity Fabric is comprised of two key elements: a scalable control fabric and a scalable data fabric. The scalable control fabric has all of the central control elements, with small remote elements that are dispersed in each different block of the SoC. And feeding into the control elements across the fabric is the data from a myriad of sensors embedded across the SoC. The scalable data fabric is much like a high performance network pathway. It features a common bus, with low latency, and a coherent Hyper Transport Plus bus, that’s multi-socket and multi die-ready. Infinity Fabric is the key element that AMD uses to link the four, 8-core dies together on a single EPYC CPU package, and is also how it links the sockets.
The individual dies on each EPYC 7000 series processors, and the sockets in a 2P system, are all linked via Infinity Fabric. The connections between the dies is fully coherent within the socket, and is optimized for low power and low latency. The connections between the sockets leverage Infinity Control Fabric. There is 42GB/s of bi-directional bandwidth between the on-die links, for a total of 170GB/s. And the bi-directional links between the sockets offer up to 38GB/s, for a grand total of up to 152GB/s. Because each CPU die is connected to its peer in the second socket, there is a 2-hop maximum between the processors. As we've seen, there is higher latency involved when threads bounce between the quad-core CCXes in on-die, so it is safe to assume there will be even higher latency when bouncing between CCXes in different sockets, but AMD is claiming they have optimized these links for low latency.
The eight total memory channels on an EPYC 7000 series processors come by way of dual memory channels per die; there are four dies per MCM, two channels per die, hence eight total memory channels. Infinity Fabric connects all of the dies and the memory together into a coherent memory pool, with up to 171GB/s of aggregate bandwidth.
All told, there are four bi-directional links between the dies on each processor, and another four bi-directional links between the sockets. This arrangement is what allowed AMD to leverage what is essentially the exact same 8-core die across every Ryzen 5, Ryzen 7, Ryzen ThreadRipper, and EPYC 7000 series processor.
