Anthropic has launched its Message Batches API, which offers developers a cost-effective solution for processing large volumes of Claude queries asynchronously.
Through the Message Batches API, developers can now send batches of up to 10,000 queries. These batches are processed in less than 24 hours and come at a 50% reduction in cost compared to standard API calls—representing a significant advancement in handling non-time-sensitive tasks more efficiently.
The API is currently available in public beta and supports Claude 3.5 Sonnet, Claude 3 Opus, and Claude 3 Haiku on the Anthropic API. Additionally, customers using Claude in Amazon Bedrock can already utilise batch inference, and support for Google Cloud’s Vertex AI is on the horizon.
The need for high throughput processing is critical for developers utilising Claude for tasks such as customer feedback analysis and language translation, where immediate responses are not paramount.
Rather than managing complex queuing systems or worrying about rate limits, the API allows developers to submit up to 10,000 queries simultaneously, ensuring smooth processing at a 50% discount. Importantly, batches are often processed much quicker than the 24-hour mark.
This API offers enhanced throughput and higher rate limits, making it possible to handle extensive volumes of requests without affecting standard API usage. Moreover, it provides scalability for large-scale data processing, taking care of tasks such as dataset analysis and model evaluations without infrastructure concerns.
The API unlocks opportunities for large-scale data processing, which were once either impractical or too costly. By leveraging the batching discount, analysing extensive corporate document repositories, potentially consisting of millions of files, becomes an economically feasible endeavour.
Claude Message Batches API pricing structure
The Message Batches API enables users to benefit from infrastructure cost savings with a 50% discount on both input and output tokens. Here’s a breakdown of the pricing structure:
Claude 3.5 Sonnet
- Input: $1.50 / MTok
- Output: $7.50 / MTok
Claude 3 Opus
- Input: $7.50 / MTok
- Output: $37.50 / MTok
Claude 3 Haiku
- Input: $0.125 / MTok
- Output: $0.625 / MTok
User-based Q&A platform Quora is taking full advantage of Anthropic’s Batches API for tasks such as summarisation and highlight extraction, thereby enhancing end-user features.
Andy Edmonds, Product Manager at Quora, said: “Anthropic’s Batches API provides cost savings while also reducing the complexity of running a large number of queries that don’t need to be processed in real time.
“It’s very convenient to submit a batch and download the results within 24 hours, instead of having to deal with the complexity of running many parallel live queries to get the same result. This frees up time for our engineers to work on more interesting problems.”
Developers eager to start using the Message Batches API in public beta can explore the documentation here.
See also: PostgreSQL 17 delivers a leap forward for open-source databases
Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
Tags: AI, anthropic, api, artificial intelligence, claude