Kubernetes Pod 命名空间内抓包

Jun 10, 2020 22:20 · 686 words · 2 minute read Kubernetes Container Docker Network

有时我们需要抓包来观察 Kubernetes 上的应用程序流量，当然可以使用 kubectl exec -it 先进入 Pod 后直接抓包，但是容器镜像中不一定如我们所愿预先安装好了 tcpdump。既然容器的本质是“用各种类型的命名空间掩饰”的进程，那我们只要进入它的网络命名空间，就可以使用宿主机上的 tcpdump 工具对命名空间的网络栈进行抓包。

以抓取到达 Kube DNS （也就是 CoreDNS）的流量为例：

kubectl get po -l k8s-app=kube-dns -n kube-system
NAME                       READY   STATUS    RESTARTS   AGE
coredns-66bff467f8-b92vk   1/1     Running   22         26d
coredns-66bff467f8-r6ps5   1/1     Running   22         26d

我们需要查出 Pod 中容器的 PID，首先根据 Pod 名称中的关键字过滤出相关 Docker 容器：

$ docker ps | grep coredns
7af1244c7a30        67da37a9a360                                      "/coredns -conf /etc…"   About a minute ago   Up 59 seconds                           k8s_coredns_coredns-66bff467f8-r6ps5_kube-system_a7629e1d-59c8-44cf-8898-b80ea80c5a67_22
bd610a71fd45        67da37a9a360                                      "/coredns -conf /etc…"   About a minute ago   Up 59 seconds                           k8s_coredns_coredns-66bff467f8-b92vk_kube-system_cbc90861-dce9-471c-88ad-270b7e41ccf3_22
cc504bfc4899        k8s.gcr.io/pause:3.2                              "/pause"                 About a minute ago   Up 59 seconds                           k8s_POD_coredns-66bff467f8-r6ps5_kube-system_a7629e1d-59c8-44cf-8898-b80ea80c5a67_177
41d024c741b9        k8s.gcr.io/pause:3.2                              "/pause"                 About a minute ago   Up 59 seconds                           k8s_POD_coredns-66bff467f8-b92vk_kube-system_cbc90861-dce9-471c-88ad-270b7e41ccf3_197

使用 docker inspect 查出 ID 为 7af1244c7a30 的容器的 PID：

docker inspect -f {{.State.Pid}} 7af1244c7a30
4021

拿到 PID 4021 后使用 nsenter 命令进入容器的网络命名空间（Namespace）

$ nsenter --help

Usage:
 nsenter [options] <program> [<argument>...]

Run a program with namespaces of other processes.

Options:
 -t, --target <pid>     target process to get namespaces from
 -m, --mount[=<file>]   enter mount namespace
 -u, --uts[=<file>]     enter UTS namespace (hostname etc)
 -i, --ipc[=<file>]     enter System V IPC namespace
 -n, --net[=<file>]     enter network namespace
 -p, --pid[=<file>]     enter pid namespace
 -U, --user[=<file>]    enter user namespace
 -S, --setuid <uid>     set uid in entered namespace
 -G, --setgid <gid>     set gid in entered namespace
     --preserve-credentials do not touch uids or gids
 -r, --root[=<dir>]     set the root directory
 -w, --wd[=<dir>]       set the working directory
 -F, --no-fork          do not fork before execing <program>
 -Z, --follow-context   set SELinux context according to --target PID

 -h, --help     display this help and exit
 -V, --version  output version information and exit

For more details see nsenter(1).
$ nsenter -n -t 4021

查看 eth0 网卡：

$ ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1450
        inet 10.244.0.249  netmask 255.255.255.0  broadcast 0.0.0.0
        ether 66:5e:9c:02:1b:d1  txqueuelen 0  (Ethernet)
        RX packets 884  bytes 286008 (279.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 862  bytes 88268 (86.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

10.244.0.249 就是 Kubernetes 网络为 Kube DNS Pod 分配的 IP。我们只需在此 eth0 网卡对 53 端口抓包即可：

$ tcpdump -i eth0 port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:12:22.843753 IP k8s-kong.46400 > prl-local-ns-server.shared.domain: 26328+ NS? . (17)
22:12:22.844518 IP k8s-kong.60879 > prl-local-ns-server.shared.domain: 21544+ PTR? 1.55.211.10.in-addr.arpa. (42)
22:12:22.844944 IP prl-local-ns-server.shared.domain > k8s-kong.60879: 21544* 1/0/0 PTR prl-local-ns-server.shared. (82)
22:12:22.845114 IP k8s-kong.52595 > prl-local-ns-server.shared.domain: 35907+ PTR? 249.0.244.10.in-addr.arpa. (43)
22:12:22.856439 IP prl-local-ns-server.shared.domain > k8s-kong.52595: 35907 NXDomain* 0/1/0 (78)
22:12:24.344549 IP k8s-kong.55308 > prl-local-ns-server.shared.domain: 2512+ NS? . (17)
22:12:25.845298 IP k8s-kong.52928 > prl-local-ns-server.shared.domain: 58987+ NS? . (17)
22:12:27.346102 IP k8s-kong.55661 > prl-local-ns-server.shared.domain: 6125+ NS? . (17)
22:12:28.846757 IP k8s-kong.53771 > prl-local-ns-server.shared.domain: 54605+ NS? . (17)

查看更多