ModeLeak: Escalating Privileges for LLM Model Exfiltration in Vertex AI

hendryadrian.com

hendryadrian.com

Organizations increasingly train AI models on sensitive datasets, but that convenience can open new attack vectors. Palo Alto Networks analyzed Google’s Vertex AI and identified two distinct vulnerabilities: one that enables privilege escalation through custom jobs and another that permits model-to-model exfiltration when a poisoned model is deployed. After disclosure, Google patched the specific issues on Vertex AI, but the findings highlight broader risks around model deployment and permissions.

The first vulnerability centers on how Vertex AI runs custom training jobs. Custom jobs execute under a service agent identity in a tenant project, and that service agent had broad access to source-project resources during testing. By injecting arbitrary code into the container spec or by supplying a custom image that launched a reverse shell, researchers ran code as the AI Platform Custom Code Service Agent (service-<PROJECT_NUMBER>@gcp-sa-aiplatform-cc.iam.gserviceaccount[.]com). That identity inherited permissions that allowed listing and reading Cloud Storage buckets and BigQuery datasets in the source project, effectively letting the attacker access data and artifacts beyond the intended scope.

To exploit this, the team either modified the container specification JSON to run commands or built an image that opened a remote shell. Running that custom job revealed the service agent identity and exposed metadata and user-data scripts for the VM instances. Those scripts pointed to internal GCP artifactory repositories, which they could then query and download images from despite original account restrictions. In short, a single permission— aiplatform.customJobs.create—combined with excessive service agent privileges enabled classic privilege escalation and broader data access.

The second vulnerability is a model exfiltration scenario that begins with a poisoned model uploaded to a public repository. If a data scientist imports and deploys that model into Vertex AI, the malicious model can execute code in the deployment environment. The researchers demonstrated this by exporting a model, altering the container image to include a reverse shell, reimporting the modified model into the Vertex AI Model Registry, and then deploying it. When the altered model ran, it gave the team an interactive shell running as the custom-online-prediction service account.

Once running as that service account, the researchers enumerated project resources and discovered multiple Kubernetes clusters tied to the environment. Using gcloud to fetch cluster credentials, they switched contexts and used kubectl to list namespaces, pods, and images. The malicious model had visibility into other deployments: pods referenced prediction images stored in tenant-project artifact registries. By describing an image to get its digest and using the service account token with crictl, they pulled container images out of the environment and exported them as .tar files using containerd tools. This process allowed them to copy deployed model images from different clusters and move them to remote environments for analysis.

The attack did not stop with ML model binaries. Vertex AI’s deployed LLMs include a fine-tuning adapter layer (an adapter file containing additional weights). Although LLM exports are more restricted in the GCP console, researchers found that deployed models and their adapter files were present in Cloud Storage buckets associated with the tenant project. Those buckets used predictable caip-prefixed paths and contained identifiers that linked back to model IDs in the source project model registry. By listing and copying these buckets—permitted by the service account’s viewer-level access—the team retrieved adapter files containing the fine-tuned weights, which are highly sensitive because they capture proprietary tuning and potentially embedded data.

Across this chain, IAM Workload Identity Federation played a key role: GCP service accounts were bound to Kubernetes service accounts, enabling lateral movement from GCP APIs into GKE clusters. Although the service accounts often had read-only permissions in the tenant project, read access combined with container runtime tools and token-based authentication permitted retrieval of images and stored model artifacts. The researchers confirmed they could access deployed model containers across clusters and download images for all models deployed in the project, including fine-tuned adapters for LLMs.

The practical impact is severe. A poisoned model published publicly could be imported and deployed by an unwitting user, at which point the model itself becomes a foothold to enumerate project resources, extract container images for models, and copy fine-tuning adapters. That chain exposes intellectual property and sensitive training artifacts, enabling model theft and replication of proprietary behaviors. Palo Alto Networks illustrated the full attack chain from a poisoned public model through deployment, gaining access to tenant-repositories, pulling images, and stealing both ML models and LLM adapter layers.

Palo Alto Networks reported the findings to Google, which implemented fixes that address the specific issues discovered in Vertex AI on GCP. The researchers recommend enforcing strict controls on model imports and deployments, minimizing service agent permissions, auditing custom job capabilities, and monitoring for unusual container images or network activity that could indicate a reverse shell or data exfiltration attempt. Palo Alto Networks also notes that their Prisma Cloud offerings help protect customers from similar cloud threats, and they encourage organizations with suspected compromises to contact the Unit 42 Incident Response team.

Read more: https://unit42.paloaltonetworks.com/privilege-escalation-llm-model-exfil-vertex-ai/