From the source
Cohere has opened a public beta of Confidential Computing on Cohere Model Vault, its fully managed inference platform, for enterprises that process sensitive data.
- The service runs on
NVIDIA H100 GPUs on
Microsoft Azure and extends Model Vault to workloads such as clinical records, financial information, privileged documents, intellectual property and proprietary source code.
- Cohere says the goal is to give enterprises managed AI's capabilities and operational simplicity while keeping sensitive plaintext out of reach of the infrastructure running the service.
- The model-serving stack, including the operating system, drivers and inference server, runs inside a confidential virtual machine, and NVIDIA Confidential Computing protects GPU memory and encrypts CPU-to-GPU traffic.
- Requests are encrypted with Oblivious HTTP to a public key tied to the attested environment, so intermediaries route traffic without seeing plaintext payloads.
- Before each protected request, the environment produces signed evidence that customer-side software checks against an approved policy; if a property does not match, the client refuses to transmit the payload.
- Cohere says the trusted computing base still includes the hardware and firmware, the measured software, the reference measurements and the attestation infrastructure, and that physical and side-channel attacks remain outside the boundary.