According to Fujitsu, their new middleware has accomplished more than double the computational efficiency of GPUs for artificial intelligence (AI) workloads during trials. This innovation was developed to address the challenges posed by GPU limitations and shortages, which have become increasingly relevant due to the growing computing demands of AI.

The middleware, which has been launched globally for customers, aims to enhance resource allocation and memory management across a variety of platforms and applications that utilize AI, as stated in a recent press release. Fujitsu has already partnered with several organizations to pilot the technology, with additional trials set to commence this month.

Fujitsu started testing this middleware with partners such as AWL, Xtreme-D, and Morgenrot in May. The results indicated an impressive increase of 2.25 times in computational efficiency when handling AI workloads. Furthermore, the partners experienced a significant rise in the number of AI processes that could be managed concurrently across different cloud environments and servers while utilizing the middleware.

Hisashi Ito, CTO of Morgenrot, noted in a press statement, “By enabling GPU sharing between multiple jobs, we achieved a remarkable near 10% reduction in overall execution time compared to running jobs sequentially on two GPUs. This parallel processing capability allowed for simultaneous execution of long training sessions for model building and shorter inference/testing tasks, all within constrained resources.”

This month, Tradom is set to commence trials of its new product, while Sakura Internet is conducting a feasibility study regarding the application of this technology in its data center operations, as reported by Fujitsu.

GPUs have proven to be more effective for AI processing compared to CPUs, leading to a significant uptick in their utilization. This rise, however, has resulted in a marked increase in power consumption across data centers and has sparked a GPU shortage. As a result, companies are exploring alternative solutions to enhance their AI workloads.

“The swift growth of computational infrastructure necessary for training generative AI poses a significant challenge regarding electrical power availability,” stated a Gartner research note on emerging technologies aimed at energy-efficient generative AI compute systems authored by researchers Gaurav Gupta, Menglin Cao, Alan Priestley, Akhil Singh, and Joseph Unsworth.

This situation compels operators of AI data centers to seek immediate solutions to address issues such as rising costs, inadequate power resources, and declining sustainability performance. “All of these challenges will eventually be passed on to the customers and end users of data center operators,” the researchers emphasized.

Eckhardt Fischer, a senior research analyst at IDC, pointed out that data centers are facing challenges due to the performance bottlenecks caused by the growing demand for GPU-assisted AI. He remarked, “Any enhancement in the computer system to alleviate this bottleneck will typically lead to a proportional increase in output.”

Gartner’s Gupta emphasized that the bottlenecks linked to AI and generative AI computational needs are primarily in memory and networking. He noted, “Even the current advancements of Moore’s Law are struggling to keep pace with the rapidly increasing computational demands.”

To address these challenges, Fujitsu has introduced its AI computing broker middleware, which combines a cutting-edge adaptive GPU allocator technology developed by the company in November 2023, along with AI-processing optimization technologies. According to the company, this middleware can automatically recognize and enhance CPU and GPU resource allocation for AI applications across multiple programs, prioritizing those that deliver high execution efficiency.

Instead of relying on traditional resource allocation methods that operate on a per-job basis, Fujitsu’s AI computing broker dynamically manages resources on a per-GPU level. This approach is designed to boost availability rates and facilitate the simultaneous execution of numerous AI processes without concerns regarding GPU memory consumption or physical capacity.

Gupta highlighted that the idea of middleware is logical, especially considering that GPU power consumption is a significant issue, making energy efficiency a priority.

He explained that while this approach does not directly address the shortage of resources, it enhances utilization and operational efficiency. In essence, this technology allows for accomplishing more with less, provided that the technology performs as expected, which remains uncertain at this stage.

Moreover, Gupta mentioned that if Fujitsu’s AI-centric middleware can yield improvements in memory and GPU utilization, it would be worth monitoring its development, adoption, and the future competitive dynamics surrounding similar offerings.

Welcome to DediRock, your trusted partner in high-performance hosting solutions. At DediRock, we specialize in providing dedicated servers, VPS hosting, and cloud services tailored to meet the unique needs of businesses and individuals alike. Our mission is to deliver reliable, scalable, and secure hosting solutions that empower our clients to achieve their digital goals. With a commitment to exceptional customer support, cutting-edge technology, and robust infrastructure, DediRock stands out as a leader in the hosting industry. Join us and experience the difference that dedicated service and unwavering reliability can make for your online presence. Launch our website.