Amazon Web Services (AWS) has recently introduced a new feature called Amazon Elastic Compute Cloud (EC2) Capacity Blocks for machine learning (ML). This offering allows users to reserve Nvidia GPUs for specified durations, catering to tasks such as training machine learning models and conducting experiments. In a blog post by Channy Yun, a principal developer advocate for AWS, the innovative nature of this feature was highlighted. EC2 Capacity Blocks allows users to reserve GPU instances for future use, tailored to their specific time requirements.
The demand for GPU capacity in machine learning has been growing, surpassing the industry’s ability to supply it. This is especially true as more companies are utilizing large language models that require access to GPUs. Nvidia GPUs are the most popular choice, but they are expensive and often in short supply. As a result, GPUs have become a limited resource, causing challenges for customers with varying capacity needs in their research and development phases.
To address this challenge, Amazon has introduced EC2 Capacity Blocks for ML. This offering simplifies access to GPU instances for training and deploying ML and generative AI models. Users can reserve hundreds of GPUs located in EC2 UltraClusters optimized for high-performance ML workloads. The Elastic Fabric Adapter (EFA) networking ensures top-tier network performance within Amazon EC2.
The process of renting EC2 Capacity Blocks is likened to reserving a hotel room. Users can specify the date, duration, and size of their reservation, similar to choosing a bed when booking a hotel room. On the designated start date, users can access their reserved EC2 Capacity Block and initiate their P5 instances. At the end of the reservation period, any running instances are automatically terminated.
This service is especially useful for scenarios where users require capacity assurance for ML model training, experiments, or anticipating future spikes in demand for ML applications. For other workload types that demand compute capacity assurance, such as critical business applications, On-Demand Capacity Reservations are still available.
To reserve an EC2 Capacity Block, users can navigate to the Capacity Reservations section on the Amazon EC2 console in the US East (Ohio) Region. Two capacity reservation options are presented, and users can select “Purchase Capacity Blocks for ML” to initiate the process.
Overall, Amazon’s EC2 Capacity Blocks for ML is an innovative solution to address the growing demand for GPU capacity in machine learning. It provides users with the flexibility to reserve and utilize GPUs based on their specific time requirements, ensuring optimal performance for their ML workloads.