High-end AI server shipments are forecast to reach 337,000 units in 2024, with the generative AI market reaching US$109 billion by 2030, at a compound annual growth rate (CAGR) of 35%, according to DIGITIMES Research. High-end server shipments in 2023 will total an estimated 167,000 units.
Large amounts of data created by the cloud and hardware have significantly increased the amount of data being transmitted, which not only opens up opportunities for AI but for the whole ICT industry, according to DIGITIMES VP Eric Huang. The next step is to enter the era of edge AI.
Based on observations during the first quarter of 2023, it was expected that the economy would pick up in the second half of the year, according to Huang. However, demand in China, the US, and Germany has been disappointing. Following the demand boom during the pandemic, demand for PCs and mobile phones has stagnated, with general-purpose servers now also seeing negative growth.
While the online fervor surrounding AI has died down a bit, Huang believes the momentum is still there. Among the main customers in the AI server field, including Microsoft, Amazon Web Services (AWS), Google, Core Weave, Meta, Oracle, and Twitter (now called X), Microsoft has garnered the most attention recently.
Based on the bill of materials of high-end servers, challenges have arisen for several items, according to Huang. While the demand is large, there are bottlenecks, with the most critical being the capacity of TSMC's advanced packaging CoWoS technology. Based on TSMC's expansion plans, the company will begin production at its new packaging plant in Tongluo, Taiwan, in 2027. The plant is expected to have a monthly advanced packaging capacity of more than 110,000 units.
The supply of high bandwidth memory (HBM) is another challenge. Previously, SK Hynix was the only source for HBM3. It has since unveiled its HBM3E standard. Beginning in the third quarter, Samsung Electronics has entered the ranks of HBM3 makers. Micron Technology is also expected to be a major participant in the HBM market in 2024.
In analyzing the trends among tier-1 semiconductor manufacturers, Intel continues to invest in its own backend packaging. Samsung also continues to heavily invest and poach talent, hoping to win some advanced packaging customers once CoWoS capacity is filled.
The next wave of opportunities for data centers and AI servers will focus on edge computing. Along with the gradual development of hardware, cloud, and software companies, it can be expected that the content and applications generated by generative AI will increase significantly, such as the launch of PCs and mobile phones with integrated AI functions.
