A security researcher bypassed ChatGPT 4.0’s safety measures by framing a prompt as a guessing game, leading the AI to disclose sensitive Windows product keys, including one owned by Wells Fargo bank. This highlights the vulnerabilities in AI content filtering systems and the risk of trained-in sensitive data leaks. #ChatGPT #WindowsProductKeys
Keypoints
- The researcher used a game-based prompt to trick ChatGPT into revealing secret information.
- ChatGPT’s safety guardrails were bypassed by framing queries creatively, such as through HTML tags and game rules.
- Sensitive data like Windows serial numbers and private keys were extracted, including a Wells Fargo key.
- The training data’s inclusion of sensitive information poses security risks for organizations and AI providers.
- Strengthened contextual awareness and multi-layer validation are recommended to prevent such jailbreaks.
Read More: https://www.theregister.com/2025/07/09/chatgpt_jailbreak_windows_keys/