What we know

3 model running on Cloudflare’s Workers AI platform caused a bug in the SGLang inference engine that exposed key-value (KV) cache data across different user requests. Cloudflare has addressed the issue and shared details about the incident and their remediation efforts. The exposure involved cached data being accessible beyond its intended scope, potentially affecting user data confidentiality. 3 model’s configuration mismatch triggered an SGLang bug that led to KV cache exposure. Cloudflare remediated the issue to prevent further data leakage. Cached data was unintentionally shared between requests, raising concerns about user data exposure.

Why it matters

  1. The bug caused by the configuration mismatch allowed cached data to leak between user sessions, which could compromise user privacy and data security. Cloudflare’s response highlights the challenges of securely deploying AI models at scale and the importance of thorough configuration and testing to prevent data leaks.

What is still unknown

  • It remains unclear if any sensitive or user-specific data was actually accessed or compromised due to the KV cache exposure. Details about the scope and duration of the exposure have not been disclosed. No independent verification of the remediation’s effectiveness has been reported.