Security researchers at Adversa uncovered a vulnerability in xAI's Grok that forces the LLM to exfiltrate user chats and personal data by exploiting its inability to differentiate between legitimate commands and malicious instructions hidden in encrypted form.
Researcher Rony Utevsky's technique embeds encrypted malicious instructions within a webpage alongside plaintext decryption keys. When prompted to summarize the page, Grok executes the embedded decryption command. Instead of generating the expected output, the decrypted instructions cause the model to extract the user's name, location, and full chat history, then append this data to a URL parameter and transmit it to an attacker-controlled server.
Adversa's analysis suggests Grok's filtering guardrails inspect inputs and outputs but fail to monitor the results of internal code execution. Instructions for processing ciphertext using PBKDF2 and AES-256-GCM pass through as routine requests, evading detection.
xAI was notified of the vulnerability in June and has not patched it. The flaw is not unique to Grok. Earlier this week, researchers demonstrated a similar attack against Microsoft 365 Copilot for enterprise, using hidden instructions to exfiltrate a password from a user's inbox.
Both incidents illustrate a fundamental weakness in LLM architecture: these models are trained to comply with user requests, making them inherently vulnerable to prompt injection attacks—malicious instructions smuggled into emails, documents, or webpages the model is asked to process. Unlike traditional software vulnerabilities that can be patched by fixing code, this weakness is structural. Developers must implement external guardrails to steer models away from harmful actions rather than relying on the model itself to refuse malicious requests.
For enterprises evaluating large-scale deployment of AI assistants like Grok 4.5 Fast or Copilot, unpatched vulnerabilities that persist months after disclosure create immediate risk to data integrity and privacy. More broadly, the recurring nature of prompt injection attacks signals that companies racing to capture AI revenue are prioritizing feature velocity over security hardening. The ability to guarantee data security is not a secondary concern—it directly influences enterprise adoption rates and, by extension, the revenue and valuation of AI product companies competing for mission-critical workloads.
