The article explains how to correlate Kubernetes audit logs with Elastic Defend for Containers data to investigate suspicious service account activity, pod creation, and attempted container escape behavior. It highlights two main join methods—pod caller identity and service account plus objectRef pod names—using an EKS lab scenario that includes a breakout pod, nsenter, chroot, and decoded exec requestURIs. #ElasticDefendforContainers #Kubernetes #EKS #nsenter #chroot
Keypoints
- Kubernetes audit logs and Defend for Containers telemetry each show different parts of the same attack, so both sources are needed for full visibility.
- The article presents two primary correlation methods: direct pod caller identity when the audit event includes the pod-name extra, and service account plus objectRef pod names when it does not.
- Shared join fields such as namespace, cluster, pod name, and time window help align audit events with runtime activity.
- In the EKS lab, a compromised workload service account performed discovery, secret access, token minting, privileged pod creation, exec activity, and cloud metadata probing.
- The breakout pod creation included sensitive settings such as hostPID, hostIPC, privileged security context, and a hostPath mount to /.
- Kubernetes audit decoded requestURI data revealed nsenter and chroot in exec activity, while runtime telemetry showed only the final executed command, not the wrappers.
- Context fields like user_agent.original, requestObject.spec, and process.args were essential for understanding what happened, even though they were not join keys themselves.
MITRE Techniques
- [T1078 ] Valid Accounts – The compromised workload service account was used to perform API activity and access secrets, suggesting misuse of legitimate credentials. [“a compromised workload service account ran discovery, read secrets, minted a token, created a privileged pod, and execed into it”]
- [T1528 ] Steal Application Access Token – The article describes reading the projected service account token from the workload, indicating token theft or harvesting. [“a projected-token read” / “service account token”]
- [T1098 ] Account Manipulation – The service account minted a token and interacted with Kubernetes identities and access paths in ways that changed or extended access. [“minted a token” / “TokenRequest”]
- [T1611 ] Escape to Host – The breakout pod used nsenter and chroot in an attempt to move into the host environment. [“nsenter and chroot” / “attempt to container escape”]
- [T1610 ] Deploy Container – A privileged pod was created as part of the escape workflow and breakout path. [“created a privileged pod” / “breakout pod create”]
- [T1059 ] Command and Scripting Interpreter – Commands were executed inside the container via exec, including shell-like breakout commands and client tools. [“execed into it” / “nsenter -t1 -m -u -n -i true and chroot /host true”]
- [T1082 ] System Information Discovery – The compromised service account performed discovery activity before escalation and breakout attempts. [“ran discovery”]
- [T1005 ] Data from Local System – The workload accessed secrets from Kubernetes, which is consistent with local data collection from the environment. [“read secrets”]
- [T1613 ] Container and Resource Discovery – The investigation focused on pods, namespaces, and container runtime events to map attacker activity across Kubernetes objects. [“look for matching orchestrator.namespace and orchestrator.resource.name”]
Indicators of Compromise
- [File paths ] service account token and CA certificate access in container runtime – /var/run/secrets/kubernetes.io/serviceaccount/token, /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- [Command names ] breakout and exec wrapper activity referenced in audit/runtime correlation – nsenter, chroot
- [Kubernetes audit fields / objects ] suspicious workload and breakout pod activity in the lab – emu-satoken-0297d6, worker-sa, breakout-0297d6
- [Process/client user agents ] suspicious API client behavior from the workload service account – python-requests, curl
- [Cloud/metadata indicators ] metadata probing from the breakout pod – cloud instance-metadata URLs or headers
Read more: https://www.elastic.co/security-labs/threat-command/kubernetes-audit-logs-container-escape