Перейти до вмісту

Module 4.5: Tetragon - eBPF Runtime Security

Цей контент ще не доступний вашою мовою.

Time to Complete: 90 minutes Prerequisites: Module 4.3 (Falco basics), eBPF Fundamentals, understanding of Linux syscalls, and basic Kubernetes security concepts Learning Objectives:

  • Understand eBPF-based runtime security
  • Deploy Tetragon for kernel-level threat detection
  • Write TracingPolicies for custom security rules
  • Block malicious actions at the kernel level before they complete

After completing this module, you will be able to:

  • Deploy Tetragon for eBPF-based runtime security enforcement with kernel-level visibility
  • Configure TracingPolicy resources to monitor and block process execution, file access, and network connections
  • Implement Tetragon’s real-time enforcement to kill processes violating security policies at the kernel level
  • Compare Tetragon’s eBPF enforcement approach against Falco’s detection-only model for defense-in-depth

Traditional runtime security tools like Falco detect threats by watching syscalls from userspace and alerting after the fact. By the time you see the alert, the malicious command has already executed. You’re often one step behind the attacker.

Tetragon changes the game by operating inside the kernel.

Using eBPF, Tetragon can observe and enforce security policies at the kernel level. It doesn’t just detect a malicious process starting; it can enforce policy during the kernel operation itself. It doesn’t just log suspicious egress; it can enforce policy on matching network-related operations.

“With Falco, you detect the attack and respond. With Tetragon, you prevent the attack from happening.”


  • Tetragon can kill a process mid-syscall, before the syscall completes
  • A single Tetragon policy can block entire attack chains—from binary execution to network exfiltration
  • Tetragon can enforce policy before an encrypted connection leaves the process, based on kernel-level context such as the operation and workload identity.
  • Tetragon is part of the Cilium ecosystem.
  • Tetragon is designed for structured, precise eBPF-based security enforcement.
  • Tetragon can enforce security without ever touching the application—no sidecars, no code changes

Falco vs Tetragon: Understanding the Difference

Section titled “Falco vs Tetragon: Understanding the Difference”
┌─────────────────────────────────────────────────────────────────────────┐
│ Falco Architecture │
│ │
│ ┌────────────┐ ┌────────────┐ ┌────────────┐ ┌────────────┐ │
│ │ Application│───▶│ Kernel │───▶│ Falco │───▶│ Alert │ │
│ │ executes │ │ (syscall) │ │ (userspace)│ │ (after) │ │
│ │ malware │ │ │ │ detects │ │ │ │
│ └────────────┘ └────────────┘ └────────────┘ └────────────┘ │
│ │
│ Timeline: [malware executes] ──────▶ [detection] ──────▶ [response] │
│ (too late) │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│ Tetragon Architecture │
│ │
│ ┌────────────┐ ┌────────────────────────────────┐ │
│ │ Application│───▶│ Kernel │ │
│ │ tries to │ │ ┌─────────────────────────┐ │ │
│ │ execute │ │ │ Tetragon eBPF program │ │ │
│ │ malware │ │ │ - Detect pattern │ │ │
│ └────────────┘ │ │ - SIGKILL process │ │ │
│ ▲ │ │ - Block syscall │ │ │
│ │ │ └─────────────────────────┘ │ │
│ │ └────────────────────────────────┘ │
│ │ │ │
│ │ ▼ │
│ │ Process killed ┌────────────┐ │
│ └───────────────────│ Alert │ │
│ before execution │ (during) │ │
│ └────────────┘ │
│ │
│ Timeline: [attempt] ───▶ [blocked + alert] (prevented) │
└─────────────────────────────────────────────────────────────────────────┘
AspectFalcoTetragon
Detection LocationUserspaceKernel (eBPF)
Response CapabilityAlert onlyAlert + Block + Kill
LatencyHigher due to userspace event handlingLower because enforcement happens in-kernel
Attack PreventionAfter the factIn real-time
EnforcementExternal (needs separate tools)Built-in
Process ContextAvailableRich (file descriptors, arguments, environment)

┌─────────────────────────────────────────────────────────────────────────┐
│ Kubernetes Cluster │
│ │
│ ┌───────────────────────────────────────────────────────────────────┐ │
│ │ kube-system namespace │ │
│ │ ┌────────────────────┐ ┌────────────────────────────────────┐ │ │
│ │ │ Tetragon Operator │ │ TracingPolicy CRDs │ │ │
│ │ └────────────────────┘ │ - file-monitoring │ │ │
│ │ │ - process-execution │ │ │
│ │ │ - network-connections │ │ │
│ │ └────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │
│ │ Node 1 │ │ Node 2 │ │ Node 3 │ │
│ │ ┌─────────┐ │ │ ┌─────────┐ │ │ ┌─────────┐ │ │
│ │ │Tetragon │ │ │ │Tetragon │ │ │ │Tetragon │ │ │
│ │ │ Agent │ │ │ │ Agent │ │ │ │ Agent │ │ │
│ │ │(DaemonSet│ │ │ │(DaemonSet│ │ │ │(DaemonSet│ │ │
│ │ └────┬────┘ │ │ └────┬────┘ │ │ └────┬────┘ │ │
│ │ │ │ │ │ │ │ │ │ │
│ │ ┌────┴────┐ │ │ ┌────┴────┐ │ │ ┌────┴────┐ │ │
│ │ │ eBPF │ │ │ │ eBPF │ │ │ │ eBPF │ │ │
│ │ │Programs │ │ │ │Programs │ │ │ │Programs │ │ │
│ │ └────┬────┘ │ │ └────┬────┘ │ │ └────┬────┘ │ │
│ │ │ │ │ │ │ │ │ │ │
│ │ ┌────┴────┐ │ │ ┌────┴────┐ │ │ ┌────┴────┐ │ │
│ │ │ Kernel │ │ │ │ Kernel │ │ │ │ Kernel │ │ │
│ │ └─────────┘ │ │ └─────────┘ │ │ └─────────┘ │ │
│ └───────────────┘ └───────────────┘ └───────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
ComponentRoleDescription
Tetragon AgentKernel enforcementDaemonSet that loads eBPF programs
eBPF ProgramsDetection & actionKernel programs that monitor and enforce
TracingPolicy CRDPolicy definitionKubernetes-native security policies
Tetragon CLIDebuggingtetra command for inspecting events

Terminal window
# Add Cilium Helm repo
helm repo add cilium https://helm.cilium.io
helm repo update
# Install Tetragon
helm install tetragon cilium/tetragon -n kube-system \
--set tetragon.enableProcessCred=true \
--set tetragon.enableProcessNs=true
# Verify installation
kubectl -n kube-system get pods -l app.kubernetes.io/name=tetragon
Terminal window
# Direct apply
kubectl apply -f https://raw.githubusercontent.com/cilium/tetragon/main/install/kubernetes/tetragon.yaml
# Wait for pods
kubectl -n kube-system wait --for=condition=ready pod -l app.kubernetes.io/name=tetragon --timeout=120s
Terminal window
# Install tetra CLI
GOOS=$(go env GOOS)
GOARCH=$(go env GOARCH)
curl -L --remote-name-all https://github.com/cilium/tetragon/releases/latest/download/tetra-${GOOS}-${GOARCH}.tar.gz
sudo tar -C /usr/local/bin -xzvf tetra-${GOOS}-${GOARCH}.tar.gz
Terminal window
# Check Tetragon status
kubectl -n kube-system logs -l app.kubernetes.io/name=tetragon -c tetragon
# Use CLI to see events
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents

TracingPolicy is a Kubernetes CRD that defines what Tetragon monitors and how it responds:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: example-policy
spec:
kprobes:
- call: "sys_execve" # Which syscall to monitor
syscall: true
args: # Capture these arguments
- index: 0
type: "string" # Filename being executed
selectors:
- matchArgs:
- index: 0
operator: "Equal"
values:
- "/bin/bash" # Match specific binaries
matchActions:
- action: Sigkill # Kill the process
TracingPolicy
├── kprobes[] # Kernel probe hooks
│ ├── call # Function name (syscall or kernel function)
│ ├── syscall # Is this a syscall?
│ ├── args[] # Arguments to capture
│ └── selectors[] # Conditions and actions
│ ├── matchArgs[] # Match on arguments
│ ├── matchPIDs[] # Match on process IDs
│ ├── matchNamespaces[] # Match on namespaces
│ └── matchActions[] # What to do on match
├── tracepoints[] # Kernel tracepoint hooks
└── uprobes[] # Userspace probe hooks

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: block-crypto-miners
spec:
kprobes:
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
selectors:
# Block known mining binaries
- matchArgs:
- index: 0
operator: "Postfix"
values:
- "xmrig"
- "cpuminer"
- "minerd"
- "cgminer"
- "bfgminer"
matchActions:
- action: Sigkill
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: protect-sensitive-files
spec:
kprobes:
- call: "fd_install"
syscall: false
args:
- index: 0
type: "int"
- index: 1
type: "file"
selectors:
- matchArgs:
- index: 1
operator: "Equal"
values:
- "/etc/shadow"
- "/etc/passwd"
- "/root/.ssh/id_rsa"
- "/var/run/secrets/kubernetes.io"
matchActions:
- action: Sigkill
- action: NotifyEnforcer # Also send alert
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: block-reverse-shells
spec:
kprobes:
# Detect shell with network redirect
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
- index: 1
type: "string"
selectors:
- matchArgs:
- index: 0
operator: "Postfix"
values:
- "bash"
- "sh"
- "zsh"
- index: 1
operator: "Contains"
values:
- "/dev/tcp"
- "/dev/udp"
- "nc -e"
- "bash -i"
matchActions:
- action: Sigkill
# Also block netcat with execute flag
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
selectors:
- matchArgs:
- index: 0
operator: "Postfix"
values:
- "nc"
- "ncat"
- "netcat"
matchActions:
- action: Sigkill
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: detect-container-escape
spec:
kprobes:
# Detect attempts to access host filesystem
- call: "sys_openat"
syscall: true
args:
- index: 0
type: "int"
- index: 1
type: "string"
selectors:
- matchArgs:
- index: 1
operator: "Prefix"
values:
- "/proc/1/root" # Host root via PID 1
- "/host" # Common host mount path
matchNamespaces:
- namespace: Mnt
operator: NotIn
values:
- "4026531840" # Host mount namespace
matchActions:
- action: Sigkill
- action: NotifyEnforcer
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: block-kubectl-exec
spec:
kprobes:
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
selectors:
- matchArgs:
- index: 0
operator: "Postfix"
values:
- "kubectl"
matchNamespaces:
- namespace: Pid
operator: NotIn
values:
- "host" # Allow from host only
matchActions:
- action: Sigkill

ActionEffectUse Case
SigkillImmediately kill the processStop attacks in progress
SignalSend specific signalCustom process control
OverrideOverride syscall return valueFake “permission denied”
NotifyEnforcerSend event to userspaceAlerting without blocking
UnfollowFdDeprecated / version-specific actionCheck the current TracingPolicy API before use
CopyFdDeprecated / version-specific actionCheck the current TracingPolicy API before use
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: fake-permission-denied
spec:
kprobes:
- call: "sys_openat"
syscall: true
args:
- index: 0
type: "int"
- index: 1
type: "string"
selectors:
- matchArgs:
- index: 1
operator: "Equal"
values:
- "/etc/shadow"
matchActions:
- action: Override
argError: -1 # EPERM - Operation not permitted

This returns “Permission denied” without revealing that security monitoring is active.


Terminal window
# Stream all events
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents
# Filter by event type
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents --process-exec
# JSON output for processing
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents -o json
# Filter by namespace
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents --namespace production
Terminal window
# Process execution events
tetra getevents --process-exec
# Process exit events
tetra getevents --process-exit
# File access events (requires policy)
tetra getevents --process-kprobe
# Network events (requires policy)
tetra getevents --process-kprobe | grep -i "connect"
# Enable export to stdout (for log collectors)
helm upgrade tetragon cilium/tetragon -n kube-system \
--set tetragon.export.stdout.enabled=true \
--set tetragon.export.stdout.format=json
# Events can be collected by Fluentd/Fluent Bit and sent to:
# - Elasticsearch
# - Splunk
# - CloudWatch
# - Any SIEM

A team running a multi-tenant Kubernetes platform can discover during testing that alert-only runtime detection leaves a response gap.

The Attack Scenario:

  1. Attacker compromises a web application pod
  2. Downloads and executes a cryptocurrency miner
  3. Falco detects and alerts
  4. Security team responds in 15 minutes
  5. By then, the miner may already have been running long enough to consume resources and generate cost

The Problem:

  • Falco’s alert arrived only after execution had already begun
  • But “after execution” means the damage is done
  • 15 minutes of crypto mining = noticeable cloud bill
  • More critically: 15 minutes to exfiltrate data

The Tetragon Solution:

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: block-post-exploit-tools
spec:
kprobes:
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
selectors:
# Block common post-exploitation binaries
- matchArgs:
- index: 0
operator: "Postfix"
values:
- "wget"
- "curl"
- "nc"
- "ncat"
- "python"
- "perl"
- "ruby"
matchNamespaces:
- namespace: Pid
operator: NotIn
values:
- "host"
matchActions:
- action: Sigkill
- action: NotifyEnforcer

The Results:

  • Red team attempts to download malware: Process is terminated before the policy-violating action completes
  • Miner never executes
  • Alert still fires for investigation
  • Zero impact from the attack

The Key Insight:

“We went from ‘detect and respond’ to ‘prevent and investigate.’ The attack surface didn’t change—our response time became effectively zero.”


MistakeProblemSolution
Too broad policiesKill legitimate processesTest in audit mode first, narrow selectors
Usually include matchNamespaces
Not testing policiesProduction breakageTest in staging with NotifyEnforcer only
Blocking curl/wget globallyBreak init containers, health checksWhitelist specific pods/namespaces
Ignoring policy orderUnexpected behaviorPolicies are independent, design carefully
Usually collect and analyze Tetragon events

Use both tools together:

┌─────────────────────────────────────────────────────────────────┐
│ Security Architecture │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Layer 1: Prevention (Tetragon) │ │
│ │ │ │
│ │ - Block known-bad binaries (miners, exploit tools) │ │
│ │ - Prevent sensitive file access │ │
│ │ - Kill reverse shell attempts │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Layer 2: Detection (Falco) │ │
│ │ │ │
│ │ - Detect anomalous patterns │ │
│ │ - Alert on suspicious behavior │ │
│ │ - Rich contextual rules │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Layer 3: Response (SIEM/SOAR) │ │
│ │ │ │
│ │ - Correlate events │ │
│ │ - Trigger automated responses │ │
│ │ - Incident investigation │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

Hands-On Exercise: Block a Simulated Attack

Section titled “Hands-On Exercise: Block a Simulated Attack”

Objective: Deploy Tetragon policies to prevent a simulated attack chain.

Terminal window
# Create test namespace
kubectl create namespace tetragon-demo
# Deploy a vulnerable-ish application (just a shell for demo)
kubectl apply -n tetragon-demo -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: target-app
labels:
app: target
spec:
containers:
- name: app
image: ubuntu:22.04
command: ["/bin/bash", "-c", "apt-get update && apt-get install -y curl netcat-openbsd && sleep infinity"]
EOF
# Wait for pod
kubectl wait --for=condition=ready pod -n tetragon-demo target-app --timeout=180s
Terminal window
# Watch Tetragon events in one terminal
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents --namespace tetragon-demo
# In another terminal, simulate normal activity
kubectl exec -n tetragon-demo target-app -- ls /
kubectl exec -n tetragon-demo target-app -- cat /etc/hostname
Terminal window
# Apply a policy to block attack tools
kubectl apply -f - <<EOF
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: block-attack-tools
spec:
kprobes:
- call: "sys_execve"
syscall: true
args:
- index: 0
type: "string"
selectors:
- matchArgs:
- index: 0
operator: "Postfix"
values:
- "curl"
- "wget"
- "nc"
- "ncat"
matchNamespaces:
- namespace: Pod
operator: In
values:
- "tetragon-demo"
matchActions:
- action: Sigkill
- action: NotifyEnforcer
EOF
Terminal window
# Try to use curl (should be killed)
kubectl exec -n tetragon-demo target-app -- curl http://example.com
# You should see:
# command terminated with exit code 137 (SIGKILL)
# Try netcat
kubectl exec -n tetragon-demo target-app -- nc -h
# Also killed immediately
Terminal window
# See the enforcement events
kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents -o json | grep -A 20 "SIGKILL"
Terminal window
kubectl apply -f - <<EOF
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
name: protect-secrets
spec:
kprobes:
- call: "fd_install"
syscall: false
args:
- index: 0
type: "int"
- index: 1
type: "file"
selectors:
- matchArgs:
- index: 1
operator: "Prefix"
values:
- "/etc/shadow"
- "/var/run/secrets"
matchNamespaces:
- namespace: Pod
operator: In
values:
- "tetragon-demo"
matchActions:
- action: Sigkill
EOF
# Try to read /etc/shadow
kubectl exec -n tetragon-demo target-app -- cat /etc/shadow
# Process killed
  • Observed normal events in Tetragon output
  • Applied the block-attack-tools policy
  • Confirmed curl/wget/nc are killed immediately
  • Applied the protect-secrets policy
  • Confirmed sensitive file access is blocked
  • Viewed enforcement events in Tetragon logs
Terminal window
kubectl delete namespace tetragon-demo
kubectl delete tracingpolicy block-attack-tools protect-secrets

What is the fundamental difference between Falco and Tetragon?

Show Answer

Falco detects threats in userspace and alerts; Tetragon can prevent threats at the kernel level

Tetragon uses eBPF to operate inside the kernel, allowing it to block syscalls and kill processes before malicious actions complete. Falco monitors syscalls from userspace and alerts after the fact.

What action immediately terminates a process in Tetragon?

Show Answer

action: Sigkill

This sends SIGKILL to the process, terminating it immediately without allowing cleanup. The process is killed before the monitored syscall completes.

What is a TracingPolicy in Tetragon?

Show Answer

A Kubernetes CRD that defines what Tetragon monitors and how it responds

TracingPolicy allows you to specify kprobes (kernel probes), tracepoints, or uprobes with selectors for matching conditions and actions to take when matches occur.

How can you make Tetragon return “Permission denied” without killing the process?

Show Answer

Use action: Override with argError: -1 (EPERM)

This overrides the syscall return value to make it appear that permission was denied, without revealing that security monitoring blocked the attempt.

Why should you include matchNamespaces in Tetragon policies?

Show Answer

To avoid blocking legitimate system processes and to scope policies to specific namespaces

Without namespace filters, policies might kill essential system processes. The filter ensures policies only apply to intended workloads.

What command streams Tetragon events in real-time?

Show Answer

kubectl exec -n kube-system -ti ds/tetragon -c tetragon -- tetra getevents

The tetra getevents command connects to the Tetragon agent and streams events. Add --namespace to filter by Kubernetes namespace.

When should you use Tetragon vs Falco?

Show Answer

Use Tetragon for prevention of known-bad actions; use Falco for detection of suspicious patterns

Tetragon excels at blocking specific, known-malicious behaviors. Falco excels at detecting anomalous patterns that might indicate novel attacks. Use both for defense in depth.

What is a kprobe in Tetragon?

Show Answer

A hook into a kernel function that allows monitoring and taking action when that function is called

Kprobes are eBPF programs attached to kernel functions. When the function is called (like sys_execve for process execution), the kprobe runs and can inspect arguments, apply selectors, and take actions.


  1. Tetragon operates in the kernel - blocks attacks before they complete
  2. TracingPolicy is the core abstraction - Kubernetes-native security rules
  3. Sigkill stops processes almost immediately - near-instant response time
  4. Override fakes errors - stealth defense without revealing monitoring
  5. Namespace filters are essential - scope policies to specific workloads
  6. Use with Falco, not instead of - prevention + detection = defense in depth
  7. Test policies carefully - too broad = production breakage
  8. Export events for analysis - blocking is half the story
  9. Operational overhead must be validated - eBPF-based enforcement can be efficient, but real impact depends on kernel, policy design, and workload mix
  10. Kernel-level visibility - sees what applications can’t hide


Continue to Module 4.6: KubeArmor to learn about runtime security policies with least-privilege enforcement.

  • github.com: tetragon — The upstream repository README directly describes Tetragon as eBPF-based runtime enforcement and lists process, syscall, and I/O activity with Kubernetes awareness.
  • github.com: falco — The upstream Falco README directly states that Falco detects and alerts on abnormal behavior and observes syscall-related events.