2017-08-21 103 views
0

我遇到了Kubernetes问题,我的Pod无法解析主机名(例如google.comkubernetes.default)。Kubernetes:Pod无法解析主机名

我目前在OpenStack的两个CentOS7实例上运行1个主节点和1个节点。我部署使用kubeadm

下面是安装的版本:

kubeadm-1.7.3-1.x86_64 
kubectl-1.7.3-1.x86_64 
kubelet-1.7.3-1.x86_64 
kubernetes-cni-0.5.1-0.x86_64 

以下列出了一些验证步骤,或许给一些洞察我的问题。

我定义了一个busybox的荚:

apiVersion: v1 
kind: Pod 
metadata: 
    name: busybox 
    namespace: default 
spec: 
    containers: 
    - image: busybox 
    command: 
     - sleep 
     - "3600" 
    imagePullPolicy: IfNotPresent 
    name: busybox 
    restartPolicy: Always 

,然后创建荚:

$ kubectl create -f busybox.yaml 

尝试执行名称google.com的DNS查询:

$ kubectl exec -ti busybox -- nslookup google.com 
Server: 10.96.0.10 
Address 1: 10.96.0.10 
nslookup: can't resolve 'google.com' 

尝试执行名称的DNS查询kubernetes.default

$ kubectl exec -ti busybox -- nslookup kubernetes.default 
Server: 10.96.0.10 
Address 1: 10.96.0.10 
nslookup: can't resolve 'kubernetes.default' 

检查我的DNS吊舱运行:

$ kubectl get pods --namespace=kube-system -l k8s-app=kube-dns 
NAME      READY  STATUS RESTARTS AGE 
kube-dns-2425271678-k1nft 3/3  Running 9   5d 

检查我的DNS服务了:

$ kubectl get svc --namespace=kube-system 
NAME  CLUSTER-IP EXTERNAL-IP PORT(S)   AGE 
kube-dns 10.96.0.10 <none>  53/UDP,53/TCP 5d 

检查DNS端点暴露:

$ kubectl get ep kube-dns --namespace=kube-system 
NAME  ENDPOINTS      AGE 
kube-dns 10.244.0.5:53,10.244.0.5:53 5d 

检查我的容器中的/etc/resolv.conf的内容:

$ kubectl exec -ti busybox -- cat /etc/resolv.conf 
nameserver 10.96.0.10 
search default.svc.cluster.local svc.cluster.local cluster.local 
options ndots:5 

如果我理解正确,Kubernetes文档指出我的豆荚应该继承节点(或主?)的DNS配置。然而,即使在它(nameserver 10.92.128.40)只是一条线,我收到以下警告旋转起来吊舱时:

Search Line limits were exceeded, some dns names have been omitted, the applied search line is: default.svc.cluster.local svc.cluster.local cluster.local mydomain.net anotherdomain.net yetanotherdomain.net 

我理解存在一个已知问题,只有这么多的项目可以在/etc/resolv.conf上市。但是,我的容器中的上述搜索行和名称服务器将从何处生成?

最后这里距离kube-dns容器日志:

$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns 
I0817 20:54:58.445280  1 dns.go:48] version: 1.14.3-4-gee838f6 
I0817 20:54:58.452551  1 server.go:70] Using configuration read from directory: /kube-dns-config with period 10s 
I0817 20:54:58.452616  1 server.go:113] FLAG: --alsologtostderr="false" 
I0817 20:54:58.452628  1 server.go:113] FLAG: --config-dir="/kube-dns-config" 
I0817 20:54:58.452638  1 server.go:113] FLAG: --config-map="" 
I0817 20:54:58.452643  1 server.go:113] FLAG: --config-map-namespace="kube-system" 
I0817 20:54:58.452650  1 server.go:113] FLAG: --config-period="10s" 
I0817 20:54:58.452659  1 server.go:113] FLAG: --dns-bind-address="0.0.0.0" 
I0817 20:54:58.452665  1 server.go:113] FLAG: --dns-port="10053" 
I0817 20:54:58.452674  1 server.go:113] FLAG: --domain="cluster.local." 
I0817 20:54:58.452683  1 server.go:113] FLAG: --federations="" 
I0817 20:54:58.452692  1 server.go:113] FLAG: --healthz-port="8081" 
I0817 20:54:58.452698  1 server.go:113] FLAG: --initial-sync-timeout="1m0s" 
I0817 20:54:58.452704  1 server.go:113] FLAG: --kube-master-url="" 
I0817 20:54:58.452713  1 server.go:113] FLAG: --kubecfg-file="" 
I0817 20:54:58.452718  1 server.go:113] FLAG: --log-backtrace-at=":0" 
I0817 20:54:58.452727  1 server.go:113] FLAG: --log-dir="" 
I0817 20:54:58.452734  1 server.go:113] FLAG: --log-flush-frequency="5s" 
I0817 20:54:58.452741  1 server.go:113] FLAG: --logtostderr="true" 
I0817 20:54:58.452746  1 server.go:113] FLAG: --nameservers="" 
I0817 20:54:58.452752  1 server.go:113] FLAG: --stderrthreshold="2" 
I0817 20:54:58.452759  1 server.go:113] FLAG: --v="2" 
I0817 20:54:58.452765  1 server.go:113] FLAG: --version="false" 
I0817 20:54:58.452775  1 server.go:113] FLAG: --vmodule="" 
I0817 20:54:58.452856  1 server.go:176] Starting SkyDNS server (0.0.0.0:10053) 
I0817 20:54:58.453680  1 server.go:198] Skydns metrics enabled (/metrics:10055) 
I0817 20:54:58.453692  1 dns.go:147] Starting endpointsController 
I0817 20:54:58.453699  1 dns.go:150] Starting serviceController 
I0817 20:54:58.453841  1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0] 
I0817 20:54:58.453852  1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0] 
I0817 20:54:58.964468  1 dns.go:171] Initialized services and endpoints from apiserver 
I0817 20:54:58.964523  1 server.go:129] Setting up Healthz Handler (/readiness) 
I0817 20:54:58.964536  1 server.go:134] Setting up cache handler (/cache) 
I0817 20:54:58.964545  1 server.go:120] Status HTTP port 8081 

dnsmasq容器。无视它发现了几个更多的名字服务器,而不仅仅是我在resolv.conf中所说的名字服务器,因为我原来有更多的名字服务器。我试图简单地将其通过移除演员:

$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq 
I0817 20:55:03.295826  1 main.go:76] opts: {{/usr/sbin/dnsmasq [-k --cache-size=1000 --log-facility=- --server=/cluster.local/127.0.0.1#10053 --server=/in-addr.arpa/127.0.0.1#10053 --server=/ip6.arpa/127.0.0.1#10053] true} /etc/k8s/dns/dnsmasq-nanny 10000000000} 
I0817 20:55:03.298134  1 nanny.go:86] Starting dnsmasq [-k --cache-size=1000 --log-facility=- --server=/cluster.local/127.0.0.1#10053 --server=/in-addr.arpa/127.0.0.1#10053 --server=/ip6.arpa/127.0.0.1#10053] 
I0817 20:55:03.731577  1 nanny.go:111] 
W0817 20:55:03.731609  1 nanny.go:112] Got EOF from stdout 
I0817 20:55:03.731642  1 nanny.go:108] dnsmasq[9]: started, version 2.76 cachesize 1000 
I0817 20:55:03.731656  1 nanny.go:108] dnsmasq[9]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth no-DNSSEC loop-detect inotify 
I0817 20:55:03.731681  1 nanny.go:108] dnsmasq[9]: using nameserver 127.0.0.1#10053 for domain ip6.arpa 
I0817 20:55:03.731689  1 nanny.go:108] dnsmasq[9]: using nameserver 127.0.0.1#10053 for domain in-addr.arpa 
I0817 20:55:03.731695  1 nanny.go:108] dnsmasq[9]: using nameserver 127.0.0.1#10053 for domain cluster.local 
I0817 20:55:03.731704  1 nanny.go:108] dnsmasq[9]: reading /etc/resolv.conf 
I0817 20:55:03.731710  1 nanny.go:108] dnsmasq[9]: using nameserver 127.0.0.1#10053 for domain ip6.arpa 
I0817 20:55:03.731717  1 nanny.go:108] dnsmasq[9]: using nameserver 127.0.0.1#10053 for domain in-addr.arpa 
I0817 20:55:03.731723  1 nanny.go:108] dnsmasq[9]: using nameserver 127.0.0.1#10053 for domain cluster.local 
I0817 20:55:03.731729  1 nanny.go:108] dnsmasq[9]: using nameserver 10.92.128.40#53 
I0817 20:55:03.731735  1 nanny.go:108] dnsmasq[9]: using nameserver 10.92.128.41#53 
I0817 20:55:03.731741  1 nanny.go:108] dnsmasq[9]: using nameserver 10.95.207.66#53 
I0817 20:55:03.731747  1 nanny.go:108] dnsmasq[9]: read /etc/hosts - 7 addresses 

而且sidecar容器:

$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c sidecar 
ERROR: logging before flag.Parse: I0817 20:55:04.488391  1 main.go:48] Version v1.14.3-4-gee838f6 
ERROR: logging before flag.Parse: I0817 20:55:04.488612  1 server.go:45] Starting server (options {DnsMasqPort:53 DnsMasqAddr:127.0.0.1 DnsMasqPollIntervalMs:5000 Probes:[{Label:kubedns Server:127.0.0.1:10053 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1} {Label:dnsmasq Server:127.0.0.1:53 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1}] PrometheusAddr:0.0.0.0 PrometheusPort:10054 PrometheusPath:/metrics PrometheusNamespace:kubedns}) 
ERROR: logging before flag.Parse: I0817 20:55:04.488667  1 dnsprobe.go:75] Starting dnsProbe {Label:kubedns Server:127.0.0.1:10053 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1} 
ERROR: logging before flag.Parse: I0817 20:55:04.488766  1 dnsprobe.go:75] Starting dnsProbe {Label:dnsmasq Server:127.0.0.1:53 Name:kubernetes.default.svc.cluster.local. Interval:5s Type:1} 

我大多已被阅读文档提供here。任何方向,洞察力或尝试的东西将不胜感激。

+0

是否从运行k8s的节点执行名称解析? –

回答

2

一些想法浮现在脑海中:

+0

嗨,kube-proxy正在运行:'/ usr/local/bin/kube-proxy --kubeconfig =/var/lib/kube-proxy/kubeconfig.conf --cluster-cidr = 10.244.0.0/16'。根据[this](https://kubernetes.io/docs/tasks/debug-application-cluster/debug-service/#is-kube-proxy-writing-iptables-rules),代理正在创建所需的iptables规则节点。有一件奇怪的事情,对服务运行'curl 10.111.133.184:80'有时会返回一个pod名称,其他时候我会得到“无路由到主机”。在通过'kubeadm'设置时,我确实选择了Flannel。你能指出如何测试它的方向吗? – azurepancake

+0

好吧,如果我正确理解这一点,Flannel会提供覆盖网络,以便Pod可以在不同节点间相互通信。我的'kube-dns-2425271678-k1nft'吊舱的IP地址是'10.244.0.5'。这是一个由Flannel处理的IP(在'/ run/flannel/subnet.env'中定义)。这个DNS pod在节点'kubemaster'上。现在我创建的pod在节点'kubenode01'上。我可以从'kubemaster' ping'10.244.0.5',但不能从'kubenode01'。我可能会认为这可能是法兰绒的问题吗?我的耳朵很湿,所以对于愚蠢的问题表示歉意。 – azurepancake

+0

为了增加一些关于测试代理服务器的信息..我可以创建两个运行nginx的pod,将它们添加到具有80端口和31746端口的服务中。然后,我可以通过使用该节点的外部IP地址以及上述端口,从外部访问该服务。这是否证明kube-proxy至少能够正常工作,因为它似乎成功地将该流量转发给了pod? – azurepancake