Keypoints
- Run an isolated virtual cluster with vcluster and deploy an intentionally vulnerable SSH pod (securecodebox/dummy-ssh) inside it to contain attacks.
- Install Falco via Helm (with Falcosidekick enabled) on the host cluster to detect kernel-level events like reads of /etc/shadow.
- Suppress noisy Falco rules with a rules override (disable “Redirect STDOUT/STDIN to Network Connection in Container”) to reduce false positives.
- Install Falco Talon (from the falco-talon GitHub repo) and configure Talon rules to match specific Falco alerts and take Kubernetes actions (terminate pods).
- Test the chain by port-forwarding the SSH service, using sshpass + ssh to run “cat /etc/shadow”, and verifying Falco logs and Talon actions terminate the compromised pod and a new pod is recreated.
- Required local/cloud prerequisites and tools include Docker, Minikube (docker driver), Helm, kubectl, sshpass, curl; an Ubuntu EC2 instance is used as an example host.
- An automation script (minhoney.sh) is provided to build (–buildit) and tear down (–burnit) the entire honeypot stack.
MITRE Techniques
- [T1021.004] Remote Services: SSH – Attacker used SSH to access the honeypot and run commands (example command: ‘ssh -p 5555 [email protected] “cat /etc/shadow”‘).
- [T1059.004] Command and Scripting Interpreter: Unix Shell – The adversary executed shell commands inside the container (Falco output shows proc.cmdline=’cat /etc/shadow’).
- [T1555] Credentials from Password Stores – The attacker read the system password file to collect credential data (Falco warning: ‘Warning Sensitive file opened for reading by non-trusted program (file=/etc/shadow …)’).
- [T1071] Application Layer Protocol – Example Talon rules reference outbound C2-style connections (‘Outbound Connection to C2 Servers’) indicating monitoring/detection of application-layer network traffic for C2.
Indicators of Compromise
- [File path] sensitive file read in honeypot – /etc/shadow
- [Container image] vulnerable target deployed – securecodebox/dummy-ssh:v1.0.0
- [Pod name] affected honeypot pods in vcluster – my-dummy-ssh-7955bc99c8-mwqxg-x-ssh-x-ssh, my-dummy-ssh-7955bc99c8-k8jgl-x-ssh-x-ssh
- [Container ID] seen in Falco event output – 0f044393375b, 1536aa9c45c2
- [Port/Forwarding] local access used for testing – 127.0.0.1:5555 -> 22 (kubectl port-forward svc/… 5555:22)
- [Repositories/URLs] installation sources referenced – https://falcosecurity.github.io/charts, https://github.com/Issif/falco-talon.git (and securecodebox chart repo)
Setup: Provision a Linux host (the author used Ubuntu Server 22.04 on an EC2 t3.xlarge) and ensure Docker, Minikube (use the docker driver), Helm, kubectl, sshpass, and curl are installed. Start Minikube with the Docker driver and enable the ingress addon. Add the falcosecurity Helm repo, create a falco namespace, and install Falco via Helm with falcosidekick enabled and its webhook configured to point at Falco Talon (set –set falcosidekick.config.webhook.address=”http://falco-talon:2803″).
Integration and configuration: Disable noisy Falco rules by creating a custom override.yaml and helm-upgrading Falco to avoid spamming alerts (the example disables “Redirect STDOUT/STDIN to Network Connection in Container”). Clone the Falco Talon repository (git clone https://github.com/Issif/falco-talon.git), install the Talon Helm chart into the same falco namespace, and edit Talon’s rules.yaml to add a rule that matches the Falco rule name (e.g., “Read sensitive file untrusted”) with an action like kubernetes:terminate. Also comment out unused outputs in values.yaml (e.g., slack) before upgrading Talon and restarting its pods to pick up changes.
Isolated honeypot and test: Install vcluster (download latest vcluster binary), create a virtual cluster called ssh inside a vcluster namespace, switch context to the vcluster, create an ssh namespace, and deploy the dummy-ssh Helm chart (securecodebox/dummy-ssh). Disconnect to return to Minikube, port-forward the SSH service (kubectl port-forward svc/ 5555:22 -n vcluster), then SSH using sshpass and run “cat /etc/shadow”. Falco will emit a “Warning Sensitive file opened for reading by non-trusted program (file=/etc/shadow …)” event; Falcosidekick forwards that event to Talon, which matches the Talon rule and performs kubernetes:terminate on the offending pod. The cluster events show the old pod being killed and a new my-dummy-ssh pod being scheduled. An automation script (minhoney.sh) is available to build (–buildit) and tear down (–burnit) the full environment.
Read more: https://sysdig.com/blog/honeypots-vcluster-and-falco-episode-ii