User Manual for GIGABYTE models including: AMD Epyc, 7003 Series, Processors, AMD Epyc 7003 Series Processors
ation Page | IT Creations
File Info : application/pdf, 16 Pages, 761.44KB
DocumentDocumentMemory Population Guidelines AMD EPYCTM 7003 Series Processors User Manual Rev. 1.0 Copyright © 2021 GIGA-BYTE TECHNOLOGY CO., LTD. All rights reserved. The trademarks mentioned in this manual are legally registered to their respective owners. Disclaimer Information in this manual is protected by copyright laws and is the property of GIGABYTE. Changes to the specifications and features in this manual may be made by GIGABYTE without prior notice. No part of this manual may be reproduced, copied, translated, transmitted, or published in any form or by any means without GIGABYTE's prior written permission. Documentation Classifications User Guide: detailed information about the installation & use of an add-on hardware or software component (e.g. BMC firmware, rail-kit, memory population) compatible with this product. For More Information For related product specifications, the latest firmware and software, and other information, please visit our website at: http://www.gigabyte.com. For GIGABYTE distributors and resellers, additional sales & marketing materials are available from our reseller portal: http://reseller.b2b.gigabyte.com For further technical assistance, please contact your GIGABYTE representative or visit http://esupport.gigabyte.com/ to create a new support ticket. For any general sales or marketing enquires, you may message GIGABYTE server directly by email: server.grp@gigabyte.com. Table of Contents Introduction ..................................................................................................................................................................... 5 Memory Bandwidth.........................................................................................................................................................5 Choosing the Right Configuration...................................................................................................................................7 One DIMM Configuration (Not Recommended) ................................................................................................................................. 7 Two DIMM Configuration (Not Recommended) ................................................................................................................................. 8 Four DIMM Configuration (Conditionally recommended only with EPYC processors that have 128MB L3 or less).......................... 9 Six DIMM Configuration (Conditionally recommended if only 6 channels can be populated).......................................................... 10 Eight DIMM Configuration (Recommended) .................................................................................................................................... 11 Ten DIMM Configuration (Recommended)....................................................................................................................................... 12 Fourteen DIMM Configuration (Recommended) .............................................................................................................................. 12 Sixteen DIMM Configuration (Recommended) ................................................................................................................................ 13 Appendix A: DIMM Population Rules............................................................................................................................14 - 3 - Revision History Date March 2021 Revision 1.0 Initial Release Description - 4 - Introduction The introduction of AMD newest EPYCTM newest processor, the EPYC 7003 generation is designed to build on last generation's industry leading eight channels of DDR4 memory EPYC 7002 generation processor. While the EPYC 7003 brings additional performance capability, the memory subsystem is similar to EPYC 7002 apart from a new, 6-way interleave mode. EPYC 7003 processors have been designed to be a drop-in upgrade to EPYC 7002 systems with type 1, enhanced motherboards. However, systems with type 0 motherboards cannot be upgraded to use EPYC 7003. Supported DIMM types include registered DIMMs (RDIMMs) built with x4 and x8 devices, load-reduced DIMMs (LRDIMMs) built with dual die and stacked packages, threedimensional stacked DIMMs (3DS DIMMs), and nonvolatile DIMMs (NVDIMMs, type N only). Supported Standard DIMM Types on AMD EPYC Processors* DIMM Type RDIMM RDIMM LRDIMM LRDIMM 3DS 3DS Ranks 1 (SR) 2 (DR) 4 8 4 8 Capacity ** 8 GB, 16 GB or 32 GB 16 GB, 32 GB or 64 GB 64 GB or 128 GB 128 GB or 256 GB 64 GB or 128 GB 128 GB or 256 GB *This table represents a listing of DIMMs available on AMD EPYC processors at the time of writing. While the AMD EPYC 7003 family of processors is compatible with the listed DIMM configurations, you should consult with your platform vendor for a list of supported DIMMs. ** See Appendix A "DIMM Population Rules" for guidance on mixing different DIMMs Memory Bandwidth EPYC 7003 processors have eight memory channels designated A, B, C, D, E, F, G, and H. Each channel supports up to two DIMMs. Systems can be built with one DIMM per channel (1 DPC), two DIMMs per channel (2 DPC), or a combination thereof. The operating speed of memory will depend on the number and types of DIMMs in the system. For EPYC 7002 processors, there were two types of motherboards defined: Motherboard Type-0 and Motherboard Type-1. While Type-0 motherboards cannot be upgraded to use EPYC 7003, Type-1 motherboards can be upgraded with no degradation of EPYC 7003 memory and I/O capability. While a decreased operational frequency with two DIMMs populated may not seem ideal for memory-intensive workloads, the additional chip selects being used, or ranks of memory, can outweigh the change in operating memory speed in certain workloads. - 5 - Memory Population EPYC Memory Speed based on DIMM Population (One DIMM per Channel) DIMM Type RDIMM LRDIMM 3DS DIMM Population DIMM 0 1R (1 Rank) 2R or 2DR (2 Ranks) 4DR (4 Ranks) 2S2R (4 Ranks) 2S4R (8 Ranks) 2S2R (4 Ranks) 2S4R (8 Ranks) Max EPYC 7003 DDR Frequency (MHz) 3200 3200 3200 3200 3200 3200 3200 EPYC Memory Speed based on DIMM Population (Two DIMM per Channel) DIMM Type RDIMM LRDIMM 3DS DIMM Population DIMM 0 DIMM 1 -- 1R 1R 1R -- 2R or 2DR 1R 2R or 2DR 2R or 2DR 2R or 2DR -- 4DR 4DR 4DR -- 2S2R (4 Ranks) -- 2S4R (8 Ranks) 2S2R (4 Ranks) 2S2R (4 Ranks) 2S4R (8 Ranks) 2S4R (8 Ranks) -- 2S2R (4 Ranks) 2S2R (4 Ranks) 2S2R (4 Ranks) -- 2S4R (8 Ranks) 2S4R (8 Ranks) 2S4R (8 Ranks) Max EPYC 7003 DDR Frequency (MHz) 3200 2933 3200 2933 2933 3200 2933 3200 3200 2933 2933 2933 2666 2933 2666 Memory Population - 6 - Choosing the Right Configuration AMD recommends that all eight memory channels per CPU socket be populated with all channels having equal capacity. This enables the memory subsystem to operate in eight-way interleaving mode, which should provide the best performance in most cases. For a given processor model number, memory population, and NUMA node per socket (NPS) configuration, the preBIOS firmware chooses the optimal memory interleaving option. There are three NPS options available: NPS=1, NPS=2, and NPS=4. The following diagrams are examples of supported DIMM configurations in a system. Notice, however, that most configurations populating fewer than eight channels are supported, but not recommended. For a full list of population rules, consult Appendix A: DIMM Population Rules. One DIMM Configuration (Not Recommended) Interleave: None (NPS=1, 2 or 4) All DIMM have the same capacity DIMM are unpopulated H A G B F C E D Figure 1. One DIMM Population in 1 DPC Configuration H A G B F C E D Figure 2. One DIMM Population in 2 DPC Configuration - 7 - Memory Population Two DIMM Configuration (Not Recommended) Interleave: CD, (NPS=1, 2 or 4) All DIMM have the same capacity DIMM are unpopulated H A G B F C E D Figure 3. Two DIMM Population in 1 DPC Configuration H A G B F C E D Figure 4. Two DIMM Population in 2 DPC Configuration Memory Population - 8 - Four DIMM Configuration (Conditionally recommended only with EPYC processors that have 128MB L3 or less ) Note1 Interleave: CDGH, (NPS=1; default and preferred) Other interleave options: CD, GH (NPS=2 or 4) All DIMM have the same capacity DIMM are unpopulated H A G B F C E D Figure 5. Four DIMM Population in 1 DPC Configuration H A G B F C E D Figure 6. Four DIMM Population in 2 DPC Configuration Note1: Recommended only if eight channels cannot be populated, and only with processors that have 128MB L3 or less - 9 - Memory Population Six DIMM Configuration (Conditionally recommended if only 6 channels can be populated ) Note2 Interleave: ACDEGH, (NPS=1; default and preferred) Other interleave options: CD, GH (NPS=2 or 4) Channels ACDEGH are the only channels capable of six-way interleaving. No other channels may be populated. All DIMM have the same capacity, and only with 256GB per channel DIMM are unpopulated H A G B F C E D Figure 7. Six DIMM Population in 1 DPC Configuration H A G B F C E D Figure 8. Six DIMM Population in 2 DPC Configuration Note2: Recommended only if 8 channels cannot be populated, and only with <= 256GB per channel, all channels with equal capacity Memory Population - 10 - Eight DIMM Configuration (Recommended) Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) All DIMM have the same capacity DIMM are unpopulated H A G B F C E D Figure 9. Eight DIMM Population in 1 DPC Configuration H A G B F C E D Figure 10. Eight DIMM Population in 2 DPC Configuration - 11 - Memory Population Ten DIMM Configuration (Recommended) Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) DIMM = 2x capacity of DIMM All DIMM have the same capacity DIMM are unpopulated Two DIMMs occupying the same channel are of the same type H A G B F C E D Figure 11. Ten DIMM Population in 2 DPC Configuration Twelve DIMM Configuration (Recommended) Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) DIMM = 2x capacity of DIMM All DIMM have the same capacity DIMM are unpopulated Two DIMMs occupying the same channel are of the same type H A G B F C E D Figure 12. Twelve DIMM Population in 2 DPC Configuration Memory Population - 12 - Fourteen DIMM Configuration (Recommended) Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) DIMM = 2x capacity of DIMM All DIMM have the same capacity DIMM are unpopulated Two DIMMs occupying the same channel are of the same type H A G B F C E D Figure 13. Fourteen DIMM Population in 2 DPC Configuration Sixteen DIMM Configuration (Recommended) Interleave: ABCDEFGH, (NPS=1; default and preferred) Other interleave options: ABCD, EFGH (NPS=2) and AB, CD, EF, GH (NPS=4) All DIMM have the same capacity Two DIMMs occupying the same channel are of the same type H A G B F C E D Figure 14. Sixteen DIMM Population in 2 DPC Configuration - 13 - Memory Population Appendix A: DIMM Population Rules Supported device technology: - DDR4 8 Gbit, 16 Gbit. Supported DIMM typesNote1: - 1R and 2R RDIMM, built with x4 and x8 DDR4 devices. - 4R and 8R LRDIMM, built with x4 DDR4 devices (4DR, 2S2R, 2S4R devices). - 2S2R (= 4R) and 2S4R (= 8R) 3DS built with x4 DDR4 devices. - NVDIMMs, type N only. Not supported - UDIMMs, 4R RDIMMs. General population order guidelines - Populate open channels before populating two DIMMs on a given channel - In 2 DPC configurations where only one DIMM is populated on a channel, populate the DIMM socket physically farthest away from the processor - Balance memory capacity per channel pair on a given CPU - Balance memory capacity per CPU socket in a two-socket system - Though a system can be populated with a single DIMM as a minimum configuration, full memory bandwidth requires one DIMM per channel (A-H) be populated For best performance, AMD recommends populating all eight memory channels per socket, with every channel having the same capacity If a customer chooses to populate only four channels in the system, AMD recommends limiting the processors in that system to those with 128MB or less of L3 and populating channels CDGH identically. This will enable four-way interleaving, which will generally provide the best performance with a four DIMM population If a customer chooses to populate only six channels, AMD recommends populating channels ACDEGH with equal capacity, where all channels are less than 256GB each. This will enable sixway interleaving, which will generally provide the best performance with a six-channel population. Note that the memory interleaving size must be set to 2kB or 4kB for six-way interleaving operation. General population rules - RDIMM, LRDIMM, 3DS, and NVDIMM-N DIMM types can coexist in the system with restrictions. - DIMMs within the same channel must be of the same base DIMM module type (all RDIMM, LRDIMM, or 3DS). NVDIMM-N can be mixed with RDIMM - DIMMs within the same channel must be of the same DRAM width. No mixing DIMMs with x4 and x8 DIMMs on a channel. - DIMMs within the same channel must be of the same DRAM density. No mixing DIMMs with 8Gb and 16Gb DRAMs on a channel. - While DIMMs with different manufacturers can be populated on the same channel, platform developers are expected to validate these configurations. Note1: 4DR = Four ranks of dual die packaged DRAM. 2S2R = Two ranks of dual die packaged DRAM. 2S4R = Two ranks of dual die packaged DRAM. Memory Population - 14 - - All memory channels operate at the same frequency. The system will use the highest common supported frequency when populated with different speed DIMMs. The highest common supported speed is the rated speed of the slowest DIMM in the system while also applying the population speed limits for the configuration (1 of 1, 1 of 2, 2 of 2). General interleaving rules - Memory channels may be organized in four pairs: A+B, C+D, E+F, and G+H. These pairs may be interleaved together (two-way interleaving), a specific set of two pairs may be interleaved together (fourway interleaving like ABCD, CDGH, or EFGH), six specific channels may be interleaved together (six-way interleaving, ACDEGH), or all channels may be interleaved together (ABCDEFGH). Eight-way interleaving is only possible when all eight channels are populated with equal size memory and NPS=1. Six-way interleaving is only possible when channels ACDEGH are populated with equal size memory, all channels are <= 256GB, no other channels are populated, memory interleaving size is set to 2kB or 4kB, and NPS=1. Four-way interleaving when NPS=1 is only possible when channels ABCD, EFGH, or CDGH are populated with equal size memory, and no other channels are populated. Four-way interleaving when NPS=2 is only possible when channels ABCD, EFGH, or CDGH are populated with equal size memory, and no other channels are populated. It is possible to have 2 four-way interleaves of ABCD and EFGH simultaneously if channels ABCD are all equal size memory, and channels EFGH are all equal size memory. The memory sizes of ABCD and EFGH are independent of each other in this case and can be different. Two-way interleaving is only possible for channels AB, CD, EF, and GH for all NPS values. For an optimized configuration, each channel should have identical capacity. While supported, unequal capacity configurations are suboptimal because the larger DIMM's excess capacity will not be interleaved, resulting in only the matched memory capacity between the DIMMs being interleaved. - The resulting interleave is based on the processor model number, how the DIMMs are populated in the system, and the chosen NPS value. The highest allowable interleave is chosen by firmware for a given NUMA domain. - Volatile (standard DIMMs) and non-volatile (NVDIMM-N) memory types cannot be interleaved. If a channel is shared between a standard DIMM and an NVDIMM-N, that channel cannot be interleaved. NVDIMM-N population rules - Contact your AMD representative for the BIOS version required for NVDIMM-N enablement and general support - All DIMM population rules in Appendix A still apply except where noted in this section - There must be at least one channel populated without NVDIMM-N in the system. A system with 100% NVDIMM-N is not supported. - For a given configuration, the populated channel with the lowest numbered UMC on the boot processor must contain only standard DIMMs. For a single processor example, if only channels E and F are populated on the boot processor, channel E must be populated with only standard DIMMs. For a multiprocessor example, if both processors have only channel B populated then the boot processor must be populated with only standard DIMMs. - 15 - Memory Population Instance to Package Channel Mapping on an Ethanol-X board Package SP3 Instance UMC0 UMC1 UMC2 UMC3 UMC4 UMC5 UMC6 UMC7 Channel A B D C H G E F Memory Population - 16 -