Cohere is launching the public beta of Confidential Computing on Cohere Model Vault, bringing hardware-backed protection and independently verifiable security to managed enterprise AI inference on Microsoft Azure and NVIDIA hardware.
The feature is built for some of an organization's most sensitive workloads, including financial information, clinical records, privileged documents, proprietary source code and intellectual property, where the goal is to use managed AI without giving the operating infrastructure access to plaintext data.
PitchBook data connector arrives in Cohere's North
Inference runs inside a hardware-backed confidential environment spanning CPU and GPU, so hardware-enforced isolation shields workload memory from privileged infrastructure software and operators, and traffic between CPU and GPU is encrypted while the model runs.
Application payloads are encrypted end-to-end to the attested environment, so requests can pass through intermediary infrastructure without exposing their contents and only that environment can decrypt them, with responses similarly protected on the return path.
Remote attestation lets customer-side software cryptographically check signed evidence about hardware, firmware and software against an approved security policy before sending data, and the client refuses to send the request if the policy is not met.
The beta runs on NVIDIA H100 GPUs on Microsoft Azure, with Cohere operating the serving platform, and customers keep using the Model Vault API without redesigning payloads.
Customers can start deploying encrypted Model Vault workloads today.
From the source
Now in public beta with NVIDIA Confidential Computing with NVIDIA GPUs on Microsoft Azure, bringing hardware-backed protection and verifiable security to sensitive enterprise AI.
Key takeaways
Protect data in use: Model Vault extends protection beyond data at rest and in transit to prompts, documents, responses, and model execution while inference is running.
Encrypt sensitive request contents end-to-end: Application payloads remain encrypted through intermediary infrastructure and can only be decrypted inside the attested confidential environment.
Verify before you trust: Customers can cryptographically verify the hardware and software environment before sensitive data leaves their infrastructure.
Use confidential AI as a managed service: Cohere operates the serving platform on Microsoft Azure infrastructure with NVIDIA H100 GPU Confidential Computing capabilities.
Start today in public beta: Customers can start deploying encrypted Model Vault workloads today.
Today, we are announcing the public beta of Confidential Computing on Cohere Model Vault , bringing hardware-backed protection and independently verifiable security to managed enterprise AI.
