Exploring Opportunities for Nvidia Competitors in AI Workloads

AI encompasses a diverse range of disciplines, leaving room for various chips beyond Nvidia GPUs, even though market consolidation seems inevitable. This is the finding of a research report by the consultancy J.Gold Associates.

The processor market for AI-enabled systems is extensive and varied, given the numerous applications and work conditions for AI, spanning data centers, cloud, and edge environments. No single vendor can dominate the entire market, and specialized vendors will be entering the scene in the next year or two to further diversify solution offerings, states Jack Gold, president of the consultancy.

The vendor landscape is expected to broaden beyond the few currently in leadership roles, according to Gold. Changing needs and substantial growth in AI-enabled systems will present significant opportunities for various vendors focused on specific AI system types and processing areas, he forecasts.

“There is enough width and breadth in the marketplace to float all boats, so to speak,” Gold noted. “Looking two or three years ahead, the majority of AI workflows will not be running on [Nvidia] H100s – the high-end machine learning units. Instead, they will involve inference loads, primarily at the edge but also on PCs and mobile devices. IoT will play a role as well. Thus, there will be a wide range of AI workloads deployed on a variety of chips, not just high-end Nvidia ones.”

Breaking down the different markets, and starting with cloud and hyperscalers, Gold sees AWS and company offering near-Nvidia levels of performance to their customers with their own custom chip technologies but at a lower price than the expensive Nvidia chips.

“We expect the hyperscaler market to offer a wide array of processors to meet the growing diversity of AI training needs, as well as some higher end inference based workloads. While in the short term, Nvidia’s dominance in this segment is likely protected, longer term (2+ years) we expect a significant dilution of its market share,” he wrote.

In the data center, he expects to see more traditional data center servers running AI workloads as they move toward inference-based workloads, as well as fine tuning and RAG optimizing of existing models. Inferencing is much less process-intensive than training and can be done on traditional CPUs instead of more expensive GPUs.

This is opening up an opportunity for AI as a service, provided by major cloud service providers, where a company can have the AI training done on the expensive hardware without having to make a major capital investment in hardware they only need once and then do the updates or inferencing with their own gear.

“It’s also likely that as newer, more efficient modeling methods are developed, they will increasingly be run on traditional servers, both from a cost/performance advantage perspective as well as for greater compute availability. This will benefit the traditional players who have a well-established data center business,” Gold wrote.

On the edge, Gold expects the vast majority of AI workloads to migrate to edge-based systems over the next two or three years. What qualifies as the edge is a wide range of systems and processing capabilities – from small internal processing in sensor arrays to heavy machinery, autonomous vehicles and medical diagnostics, just to name a few.

Gold predicts that open-source platforms and development environments will play a key role in this space as opposed to proprietary solutions like Nvidia’s CUDA. “Open and compatible ecosystems like Arm and x86 from will have significant advantages as they create compatibility from small to large computing needs. They allow up scaling or down scaling as the processing requires as well as ease of porting solutions and reuse,” he wrote.

The IoT space has a lot of overlap with edge computing, and therefore there is a need for an open ecosystem to provide scalable solutions, much like the edge. It’s just that with IOT, the devices tend to be smaller and lower power, but there are plenty of players in that field.

With so much hype surrounding AI, there has been a significant number of startups in the AI processor market in the last few years, with more to come in the next few years. But since they are relatively new players with a lack of established market presence and proven capability, it’s difficult to position them effectively as they stake out their particular niches, he said.

“We do expect a few of the new entrants to ultimately be successful, while many of the others will either fade away or be acquired in the next two to three years,” he wrote. He cited Cerebras as a notable startup, with its wafer scale technology that is positioned at the high end of the market and challenging the dominance of Nvidia.

---

Welcome to DediRock, your trusted partner in high-performance hosting solutions. At DediRock, we specialize in providing dedicated servers, VPS hosting, and cloud services tailored to meet the unique needs of businesses and individuals alike. Our mission is to deliver reliable, scalable, and secure hosting solutions that empower our clients to achieve their digital goals. With a commitment to exceptional customer support, cutting-edge technology, and robust infrastructure, DediRock stands out as a leader in the hosting industry. Join us and experience the difference that dedicated service and unwavering reliability can make for your online presence. Launch our website.