AWS adds usage-based metering for Amazon SageMaker AI bidirectional streaming, with Deepgram as launch partner

AWS added usage-based metering for Amazon SageMaker AI bidirectional streaming. A model server opts in with the X-Amzn-SageMaker-Metadata-Stream-Supported header, then reports usage over a companion WebSocket (/invocations-bidirectional-stream-metadata). It sends JSON frames (inference.count dimension, ConsumedUnits, ClientToken) at up to 1 message per second and 512 bytes each. Until now, streaming models listed on AWS Marketplace could only use hourly pricing. Deepgram is the launch partner, with usage-based speech-to-text and text-to-speech billing. Endpoint infrastructure charges still apply on top.

The main audience is companies that sell streaming models, especially real-time speech models, through AWS Marketplace. They can now bill by consumption instead of a flat hourly rate. Buyers of those models get one AWS bill and can use private-offer rates, but they still pay for the underlying endpoint separately. That means the total cost of a model may be harder to compare, not easier. For most AWS customers and for sellers outside streaming speech, little changes.

The claim that this aligns billing with real consumption is fair, but the benefit goes mainly to sellers and depends on how each one defines a billable unit. The metering is self-reported by the model server, and the coverage says nothing about how buyers can audit it. The Deepgram tie-in reads partly as promotion, and one launch partner is thin evidence of wider uptake.

1. 01How many Marketplace sellers beyond Deepgram adopt the metering stream, and in which model categories (speech, video, other)
2. 02What Deepgram's usage-based Marketplace prices look like next to its own direct rates and the old hourly option, including the extra endpoint costs
3. 03Whether AWS publishes buyer-side tools to check usage, such as billing dispute handling, audit logs or cost dashboards for the reported inference.count units