From the source

Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2

Capability change

Amazon Web Services, NVIDIA, and Heidi Health announce a solution that uses NVIDIA CUDA Multi-Process Service and NVIDIA Triton Inference Server on Amazon EC2 GPU instances to reduce GPU infrastructure requirements for ASR inference by 75 percent, from 16 instances to 4, while maintaining sub-second latency at 92.1 requests per second per GPU, enabling Heidi Health to process over 2.4 million clinical consultations per week.

From the source
In this post, we focus on what comes after fine-tuning: serving that model efficiently. We demonstrate how NVIDIA CUDA Multi-Process Service (MPS), combined with NVIDIA Triton Inference Server™ on Amazon EC2 GPU instances, reduces GPU infrastructure requirements by 75 percent (from 16 instances to 4).
aws.amazon.com

Most covered

Meta Muse Wonder: A Frontier Model For Personal & Professional AI Agents
SiliconANGLE: “Meta releases Muse Wonder AI models built to run personal and professional agents”
Reporting · 29 outlets
Meta· 12 Oct
Muse Wonder: A Frontier Model For Personal & Professional AI Agents
Google Antigravity Gemini 4 Argon in Google Antigravity
9to5Google: “Google Antigravity is now built into Gemini app for macOS”
Reporting · 29 outlets
Image from antigravity.google
Google Antigravity· 13 Oct
Gemini 4 Argon in Google Antigravity
Microsoft Deep dive into Azure Maia 300 architecture
Axios: “Every major Nvidia customer is now a competitor as Microsoft reveals new Maia chip”
Reporting · 14 outlets
Image from azure.microsoft.com
Microsoft· 12 Oct
Deep dive into Azure Maia 300 architecture
xAI Introducing Grok Voice 3.0
SiliconANGLE: “Grok Voice 3.0 from SpaceXSI outscores OpenAI and Google on speech-to-speech index”
Reporting · 8 outlets
Image from x.ai
xAI· 14 Oct
Introducing Grok Voice 3.0
Anthropic Anthropic opens Singapore office and signs MOU with the Government of Singapore
Unite.AI: “Anthropic Signs Singapore Government MOU as New Office Opens”
Reporting · 2 outlets
Image from anthropic.com
Anthropic· 14 Oct
Anthropic opens Singapore office and signs MOU with the Government of Singapore

Other stories

Other stories

Other stories

Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2

From the source

Other stories

Build self-improving document processing using Amazon Bedrock AgentCore

Generate presentations with Amazon Bedrock AgentCore

How Tamnoon built AI-powered cloud security remediation with Amazon Bedrock

Building agents that transact with Anchor Browser and AgentCore payments

Manage private pricing offers for Amazon Bedrock with managed entitlements