After announcing its foray into the LLM (Large Language Model) IP market, Skymizer recently unveiled a series of hardware and software solutions centered around LLM IP.
These offerings are designed to bring more imagination to the LLM application service market. Skymizer's IP solution series is codenamed EdgeThought, with the first market-ready solution named ET2, capable of efficiently handling all current edge devices requiring LLM, including the recently released Llama3 with a parameter scale of up to 8 billion.
Before introducing the IP solution, Skymizer's primary solutions were in the compiler space, bridging the gap between chips and software. This background has endowed the company with extensive experience in overall system hardware and software integration and optimization. As the demand for LLM rises, Skymizer, leveraging its solid market foundation, has entered the IP market to cater to diverse vertical application needs.
Launch of integrated hardware-software platform: ET2 features edge computing, LLM, and AI inference
According to Skymizer Executive Vice President William Wei, ET2 encompasses three key elements: edge computing, LLM, and AI inference. Besides accommodating various LLMs in the market, ET2 can flexibly expand computing resources to meet client needs.
If the parameter scale of the LLM to be processed is too large, expansion can achieve the required computing power, naturally increasing memory capacity and power consumption. In addition to the existing IP solutions, Skymizer also launched the SkyGenie SDK (Software Development Kit).
The SDK can address various categories of LLMs, including general, domain-specific, and private, assisting various industry applications. This enables software developers to create corresponding applications based on different LLM types, optimizing overall system performance. During COMPUTEX 2024, Skymizer will further demonstrate applications such as smart factories' Autonomous Mobile Robots (AMRs), Drive-thru ordering smart assistants, and smart automotive scenarios using ET2 and other hardware-software solutions.
Wei emphasized that Skymizer's comprehensive hardware-software system development experience allows the broad tech industry ecosystem to benefit from Skymizer's complete platform solutions. He revealed that while the market currently sees edge AI GPUs performing at about 20 tokens per second, Skymizer's ET2 tests show around 32 tokens per second, with implementation costs at just 1/100 of Edge AI GPUs. This high cost-performance ratio makes ET2 an ideal choice for cost-sensitive end applications.
First Chip with ET2 solution to debut at CES 2025
In the semiconductor domain, Wei candidly shared that Skymizer is open-minded and actively collaborating with domestic and international design services and IP firms. He also revealed that ET2 is highly expandable, from small IoT MCUs to high-performance Edge Servers.
When paired with higher bandwidth memory interfaces, it can function as a server-level inference engine for multi-user, multi-batch processing. The first chip adopting ET2 is expected to debut at CES 2025, positioning ET2 as a game-changer for edge device LLM inference.
Conclusion
Skymizer's launch of its first IP solution, ET2, marks a new milestone for the company. With rich hardware-software integration experience, Skymizer not only provides high-performance, cost-effective edge computing solutions but also equips developers with powerful tools through the SkyGenie SDK to meet diverse LLM application needs.
These innovations enhance the performance of different vertical market applications and create new possibilities for smart factories, autonomous mobile robots, smart automotive, and more. Skymizer's comprehensive platform solutions are poised to be a significant driving force in the future LLM application market.
Skymizer not only provides LLM silicon intellectual property solutions but also offers the SkyGenie SDK, supporting various types of LLMs. This makes it easier and faster for AI application developers to create applications, enabling hardware chip partners to achieve higher integration and better meet market demands