AMD unveils Instinct MI300X GPU and MI300A APU, claims up to 1.6X lead over Nvidia’s competing GPUs

AI MD fo HPC d its MI3 AMD lineup using MI300 most advanced production technologies ever pressed into mass production, employing new techniques like its 3.5D’ packaging to 3.5D uce two multi-chip behe two hs that it says provide Nvidia-beating perf Nvidia e in a wide range of AI workloads.

AMD AI sn’t sharing AMD icing for its new exotic silicon, but the products are now shipping to a range of OEM partners.

AMD In OEM nct MI300A is the world’s first data center APU that combines both a CPU a APU GPU in the same pac 13 ge.

AMD culled the memory capacity by using eight 8Hi HB twenty-four nstea 4 CPU the eight 12Hi stacks use 3 for the MI300X, thus eight ing capacit HBM3 om 192GB to 128GB.

AMD also claim 146 billion vantage over the H100 in FP64 Matrix/DGEM AMD nd FP64/FP32 Vector TFLOPS.

AMD leverages Instinct MI300X GPU onding SoIC (sil 1.6X -on-integrated-circuit) tech to 3D-stack AI e various compute elements.

Each I/O die houses a 32-channel HBM3 memory controller to host two of the eight HBM stacks, resulting in 128 total 16-bi Nvidia ry channels for the device.

AMD has also added 256MB of total Infinity HBM3 he capacity, spread ac Nvidia ll four of the I/ O Dies 192 o GB ache data traffic via a prefetcher. US Nvidia’s H100 HGX > The AMD Instinct MI300A ed_highLightText__NxlGi">Paul Alcorn CPU ss="summaryFeed_highLightText__NxlGi">AI GPU an class AMD ummaryFeed_highLightText__NxlGi">25 years Nvidia ummaryFeed_highLightText__NxlGi">Tom's Hardware AMD Instinct xlGi">3D V-Cache AMD _highLightText__NxlGi">AMD eight mmaryFeed_highLightText__NxlGi">X eight pan> 5 lass="su 3 m GPU Feed_highLightText XCD xlGi" four AMD i">CoWos 750W Gi">3.5D 304 ="summaryFeed_hi 192 GB htText__NxlGi">first5.3 > TB 256 ">MI30 Infinity Cache n class="summaryFeed_high L3 ghtText__NxlGi">AMD AMD __NxlGi">MI300 3.5D Gi">3D V-Cache GPU Feed_highLightText__NxlGi">millions 2.5D mmaryFeed_highLightText__NxlGi">AMD NUMAness EPYCeight span AMD ass="summaryFe AI _highLightText__NxlGi">NPS 896 p GB > Infinity Fabric > 1.5 ghtText__NxlGi">GB 10.4 Feed_highLightText__NxlGi">665 the OCP Universal Baseboard /span> AMD >GB 2.4X mmaryFeed_highLightText__NxlGi">641.3X > Nvidia’s H100 HGX 1.5 TBAMD 400GbE _NxlGi">1.2 Nvidia Gi">NoC Mellanox third 60% AMD class="summaryFeed_highLightText__NxlGi">three 2.4X ummaryFeed_highLightText__NxlGi">30% H100 ghLig HPC ext__NxlGi">two1.3X span class="summaryFeed_highLightText__NxlGi">three AI ummaryFeed_highLightText__NxlGi">IOD two IOD AMD an class 176-billion ed_highLigh Flash Attention 2 o 1.6X "summaryFeed_highLightText__ the Nvidia H100 n> 70-billion t__NxlGi">I Llama 2 n> 1.4X ed_highLightText__NxlGi">HBM 2 ghtText__NxlGi">AMD AMD mmaryFeed_highLightText__NxlGi">two 30-billion _NxlGi">first H100 e HGX ighLightText__NxlGi">Infinity Fabric one ryFeed_highLightText__NxlGi">XCD eight Text__NxlGi">two GPU AMD __NxlGi">HBM3 two 30B _highLightText__NxlGi">Two70B H100 NxlGi">3 70B IOD 290B class="summaryFeed_highLightText__NxlGi">XCD H100 i">KB< Naturally sp Nvidia ss="summaryFeed H200 hLightText__NxlGi">16x 256 L2 Nvidia ext__NxlGi">4 next year eed_highLightText__NxlGi">38 AMD Instinct nly 38The AMD Instinct MI300A maryFeed_highLig first t__NxlGi">CDN APU /span> CPU Light GPU t__NxlGi">40 Nvidia xlG Grace Hopper Superchips ass="summaryFee CPU ighLi GPU Text__NxlGi">GPU MI300A tText__NxlGi">one El Capitan highLightText__NxlGi AMD I300 MI300A NxlGi">two MI200 three 5 s="summaryFeed_highLi CCD Text__N eight >GPU< 4 s CPU > EPYC xlGi" Ryzen /span> two _highLig XCD GPU _NxlGi">5 MI300A mmaryFeed_hi 24 LightText_ CPU lGi">two228 > 3 n class="summaryFeed_highLigh six x XCD GPU i">GMI3 AMD four class="summaryFeed_hig IOD ghtText__NxlGi">2.5D 3 AMD t__NxlGi">Global Memory Interconnect< eight 8Hi span class="summaryFeed_high eight Text__NxlGi">GMI3 192 GB NxlG 128 GB YC 5.3 g TB Text AMD xlGi">CCD CPU AMD HPC s="su AI aryFeed_highLightText__NxlGi 128 GB /span> 5.3 " TB mmaryFeed_highLightText__NxlGi"> 1.6X /span> Nvidia’s H100 SXM GPU xt__Nx MI300A MD up to 760W. P AMD span> CPU t__Nx GPU ">MI300A CPU CPU I300s GPU pan> the Infinity Fabric x16 four 256 MB Infinity Cache lGi">Infinity Fabric AMD < 1.8X class="summaryFeed_ H100 LightText__NxlGi">5.0 x16 H100 class="summaryFeed_highLightText__Nxl AMD >four MI300A n class="summaryFee Nvidia’s H100 xt__NxlGi">OneHPC n> two H100 an> < GPU n class="summaryFeed CPU ghLig GPU ext__NxlGi">5.0< MI300A 128 HBM the Nvidia Grace Hopper GH200 Superchip i">Advanced Package) I CPU rconn GPU L3 AMD ss="summaryFeed_highLightText__NxlGi">L4 Nvidia ighLightT AMD __NxlGi">all four Nvidia’s GH200 ext__NxlGi">Infi 2X ty Cache 256 H200 ss AMD ummaryFeed_highLightText__NxlGi">AMD< H100 n> HPCG Feed_h Gromacs tText__NxlGi">GB 1.2X s="summaryFe 1.1X ighLightText__NxlGi">128 GH200 ummaryFeed_highLightText__NxlGi">8Hi AMD Instinct maryFeed_highLightText__NxlGi">MI300A GB 192AMD > TSMC "summaryFeed_highLightText__NxlGi">16 128 CPU mmaryFeed_highLightText__NxlGi"> GPU four ="summaryFeed_highLightText__NxlGi">eight two class="s three yFeed_highLi CCD Text__NxlGi">two EPYC ss="summaryFeed_ eight ightText__NxlGi">32 4 /span> Three "summaryFeed_highLi six Text__NxlGi">2.5D MI300A ss="summaryFeed_highLigh eight __NxlGi">AM HBM span> 2.5D