已知构建Docker主机的配置管理代码可以处理来自市场的标准RHEL 7映像,因此已知问题是SOE RHEL 7映像中的内容.
RHEL 7.2 / Docker版本1.12.6,内部版本88a4867 / 1.12.6.容器是RHEL 7.3. SELinux处于启用/允许模式. Docker主机是Amazon EC2实例.
一些配置：
# /etc/sysconfig/dockerOPTIONS='--dns=10.0.0.10 --dns=10.0.0.11 --dns-search=example.com'DOCKER_CERT_PATH=/etc/dockerADD_REGISTRY='--add-registry registry.example.com'no_proxy=169.254.169.254,localhost,127.0.0.1,registory.example.comhttp_proxy=http://proxy.example.com:8080https_proxy=http://proxy.example.com:8080ftp_proxy=http://proxy.example.com:8080
容器和主机中的解析器配置是相同的：
# /etc/resolv.confsearch example.comnameserver 10.0.0.10nameserver 10.0.0.11
如果我用–deBUG重新启动docker守护程序,我在journalctl -u docker.service中看到以下内容：
Aug 08 11:44:23 myhost.example.com dockerd-current[17341]: time="2017-08-08T11:44:23.430769581+10:00" level=deBUG msg="name To resolve: http://proxy.example.com."Aug 08 11:44:23 myhost.example.com dockerd-current[17341]: time="2017-08-08T11:44:23.431488213+10:00" level=deBUG msg="query http://proxy.example.com.[1] from 172.18.0.6:38189,forwarding to udp:10.162.182.101"Aug 08 11:44:27 myhost.example.com dockerd-current[17341]: time="2017-08-08T11:44:27.431772666+10:00" level=deBUG msg="Read from DNS server Failed,read udp 172.18.0.6:38189->10.162.182.101:53: I/O timeout"
进一步观察后,如果我指定一个IP地址而不是代理的DNS名称,我可以得到一些网络.虽然这只是一种避免使用DNS而不是真正修复的方法.
实际上,(更新#3)事实证明,我可以通过简单地配置DNS以使用TCP而不是UDP来完全避免这个问题,即
# head -1 /etc/sysconfig/dockerOPTIONS="--dns=10.0.0.10 --dns=10.0.0.11 --dns-search=example.com --dns-opt=use-vc"
(添加一行use-vc告诉解析器使用TCP而不是UDP.)
我确实在iptables中注意到一些可疑的规则,但结果证明这是正常的：
# iptables -n -L DOCKER-ISolATION -v --line-numbersChain DOCKER-ISolATION (1 references)num pkts bytes target prot opt in out source destination 1 0 0 DROP all -- br-1d6a05c10468 docker0 0.0.0.0/0 0.0.0.0/0 2 0 0 DROP all -- docker0 br-1d6a05c10468 0.0.0.0/0 0.0.0.0/0 3 34903 11M RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
删除这两个DROP规则后,我继续看到这个问题.
完整iptables：
# iptables -nL -vChain input (policy ACCEPT 2518 packets,1158K bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 0 packets,0 bytes) pkts bytes target prot opt in out source destination 23348 9674K DOCKER-ISolATION all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0 0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABliSHED 0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0 0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0 23244 9667K DOCKER all -- * br-1d6a05c10468 0.0.0.0/0 0.0.0.0/0 23232 9667K ACCEPT all -- * br-1d6a05c10468 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABliSHED 104 6230 ACCEPT all -- br-1d6a05c10468 !br-1d6a05c10468 0.0.0.0/0 0.0.0.0/0 12 700 ACCEPT all -- br-1d6a05c10468 br-1d6a05c10468 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT 2531 packets,414K bytes) pkts bytes target prot opt in out source destination Chain DOCKER (2 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT tcp -- !br-1d6a05c10468 br-1d6a05c10468 0.0.0.0/0 172.18.0.2 tcp dpt:443 0 0 ACCEPT tcp -- !br-1d6a05c10468 br-1d6a05c10468 0.0.0.0/0 172.18.0.2 tcp dpt:80 0 0 ACCEPT tcp -- !br-1d6a05c10468 br-1d6a05c10468 0.0.0.0/0 172.18.0.3 tcp dpt:389Chain DOCKER-ISolATION (1 references) pkts bytes target prot opt in out source destination 0 0 DROP all -- br-1d6a05c10468 docker0 0.0.0.0/0 0.0.0.0/0 0 0 DROP all -- docker0 br-1d6a05c10468 0.0.0.0/0 0.0.0.0/0 23348 9674K RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
桥配置
# ip addr show docker04: docker0: <NO-CARRIER,broADCAST,MulTICAST,UP> mtu 1500 qdisc noqueue state DOWN link/ether 02:42:a8:73:db:bb brd ff:ff:ff:ff:ff:ff inet 172.17.0.1/16 scope global docker0 valID_lft forever preferred_lft forever# ip addr show br-1d6a05c104683: br-1d6a05c10468: <broADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 02:42:d5:b6:2d:f5 brd ff:ff:ff:ff:ff:ff inet 172.18.0.1/16 scope global br-1d6a05c10468 valID_lft forever preferred_lft forever
和
# docker network inspect brIDge [ { "name": "brIDge","ID": "e159ddd37386cac91e0d011ade99a51f9fe887b8d32d212884beace67483af44","Scope": "local","Driver": "brIDge","EnableIPv6": false,"IPAM": { "Driver": "default","Options": null,"Config": [ { "subnet": "172.17.0.0/16","Gateway": "172.17.0.1" } ] },"Internal": false,"Containers": {},"Options": { "com.docker.network.brIDge.default_brIDge": "true","com.docker.network.brIDge.enable_icc": "true","com.docker.network.brIDge.enable_ip_masquerade": "true","com.docker.network.brIDge.host_binding_ipv4": "0.0.0.0","com.docker.network.brIDge.name": "docker0","com.docker.network.driver.mtu": "1500" },"Labels": {} }] 在日志中：
Aug 04 17:33:32 myhost.example.com systemd[1]: Starting Docker Application Container Engine...Aug 04 17:33:33 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:33.056770003+10:00" level=info msg="libcontainerd: new containerd process,pID: 2140"Aug 04 17:33:34 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:34.740346421+10:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"Aug 04 17:33:34 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:34.741164354+10:00" level=info msg="Loading containers: start."Aug 04 17:33:34 myhost.example.com dockerd-current[2131]: .........................time="2017-08-04T17:33:34.903371015+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:35 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:35.325581993+10:00" level=info msg="Default brIDge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address" Aug 04 17:33:36 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:36+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:37 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:37+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:37 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:37+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:38 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:38+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:39 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:39+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:40 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:40+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:40 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:40+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:42 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:42+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:42 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:42+10:00" level=info msg="Firewalld running: true"Aug 04 17:33:43 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:43.541905145+10:00" level=info msg="Loading containers: done."Aug 04 17:33:43 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:43.541975618+10:00" level=info msg="Daemon has completed initialization"Aug 04 17:33:43 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:43.541998095+10:00" level=info msg="Docker daemon" commit="88a4867/1.12.6" graphdriver=devicemapper version=1.12.6Aug 04 17:33:43 myhost.example.com dockerd-current[2131]: time="2017-08-04T17:33:43.548508756+10:00" level=info msg="API Listen on /var/run/docker.sock"Aug 04 17:33:43 myhost.example.com systemd[1]: Started Docker Application Container Engine.
从容器,我可以@R_419_6817@默认网关,但所有名称解析都失败.
我注意到日志中有一个奇怪的东西(更新#2我现在知道这是一个红色的鲱鱼 – 见下面的讨论)：
# journalctl -u docker.service |grep insmod > /tmp/log # \n's replaced belowJul 26 23:59:02 myhost.example.com dockerd-current[3185]: time="2017-07-26T23:59:02.056295890+10:00" level=warning msg="Running modprobe brIDge br_netfilter Failed with message: insmod /lib/modules/3.10.0-514.26.2.el7.x86_64/kernel/net/brIDge/brIDge.ko sysctl: cannot stat /proc/sys/net/brIDge/brIDge-nf-call-arptables: No such file or directorysysctl: cannot stat /proc/sys/net/brIDge/brIDge-nf-call-iptables: No such file or directorysysctl: cannot stat /proc/sys/net/brIDge/brIDge-nf-call-ip6tables: No such file or directorymodprobe: ERROR: Error running install command for brIDgemodprobe: ERROR: Could not insert 'brIDge': UnkNown error 253insmod /lib/modules/3.10.0-514.26.2.el7.x86_64/kernel/net/llc/llc.ko insmod /lib/modules/3.10.0-514.26.2.el7.x86_64/kernel/net/802/stp.ko install /sbin/modprobe --ignore-install brIDge && /sbin/sysctl -q -w net.brIDge.brIDge-nf-call-arptables=0 net.brIDge.brIDge-nf-call-iptables=0 net.brIDge.brIDge-nf-call-ip6tables=0 insmod /lib/modules/3.10.0-514.26.2.el7.x86_64/kernel/net/brIDge/br_netfilter.ko,error: exit status 1"
更新#1：这来自：
# tail -2 /etc/modprobe.d/dist.conf# disable netfilter on brIDges when the brIDge module is loadedinstall brIDge /sbin/modprobe --ignore-install brIDge && /sbin/sysctl -q -w net.brIDge.brIDge-nf-call-arptables=0 net.brIDge.brIDge-nf-call-iptables=0 net.brIDge.brIDge-nf-call-ip6tables=0
也：
# cat /proc/sys/net/brIDge/brIDge-nf-call-{arp,ip,ip6}tables111 但是,即使我这样做了：
# for i in /proc/sys/net/brIDge/brIDge-nf-call-{arp,ip6}tables ; do echo 0 > $i ; done 仍然没有运气.
我花了整整一天时间,所以现在把头发拉出来.任何关于我还能尝试什么的想法或其他问题可能会非常感激.
更新#4
我使用Netcat进行了一些实验,我已经证明,如果从任何容器发送,所有UDP数据包都不会被转发 – >主办.我尝试使用多个端口,包括53,2115和50000.然而,TCP数据包很好.如果我用iptables -F刷新iptables规则,这仍然是正确的.
此外,我可以将UDP数据包从一个容器发送到另一个容器 – 只有来自容器的UDP流量 – >主机未转发.
要设置测试：
在主机上,其IP为10.1.1.10：
# nc -u -l 50000
在容器上：
# echo "foo" | nc -w1 -u 10.1.1.10 50000
在TCP转储捕获期间,我看到：
17:20:36.761214 IP (tos 0x0,ttl 64,ID 48146,offset 0,flags [DF],proto UDP (17),length 32) 172.17.0.2.41727 > 10.1.1.10.50000: [bad udp cksum 0x2afa -> 0x992f!] UDP,length 4 0x0000: 4500 0020 bc12 4000 4011 53de ac11 0002 E.....@[email protected]..... 0x0010: 0aa5 7424 a2ff c350 000c 2afa 666f 6f0a ..t$...P..*.foo. 0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................17:20:36.761214 IP (tos 0x0,length 4 0x0000: 4500 0020 bc12 4000 4011 53de ac11 0002 E.....@[email protected]..... 0x0010: 0aa5 7424 a2ff c350 000c 2afa 666f 6f0a ..t$...P..*.foo. 0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
我试图通过this修复坏的UDP校验和失败.
然而,我注意到,即使在成功传输UDP数据包(主机 – >主机)和容器 – >期间,也会看到错误的UDP校验和.容器.
总之,我现在知道：
>路由很好
> iptables被刷新了
> SElinux很宽松
>所有TCP都可以在所有方向上运行
>所有来自容器的UDP – >容器很好
>来自主机的所有UDP – >主人很好
>来自主机的所有UDP – >容器很好
>但是没有来自容器的UDP数据包 – >主机被转发
尝试使用grep -r brIDge /etc/modprobe.d /lib/modprobe.d作为初学者,或以其他方式挖掘/etc/modprobe.d或/lib/modprobe.d并尝试查找它定义调用的安装规则的位置sysctl -q -w net.brIDge.brIDge-nf-call-arptables = 0 net.brIDge.brIDge-nf-call-iptables = 0 net.brIDge.brIDge-nf-call-ip6tables = 0
这个sysctl显然是错误的.它要么是多余的要么应该出现在br_netfilter之后,而不是之前.为什么？最近,/ proc / sys / net / brIDge处理已从桥接模块移动到br_netfilter模块.某些版本的内核* .rpm会发生这种情况,而modprobe.d目录的内容则与其他单个软件包一起分发.我在RHEL 7.2上验证了：
# modprobe brIDge# sysctl -q -w net.brIDge.brIDge-nf-call-iptables=0sysctl: cannot stat /proc/sys/net/brIDge/brIDge-nf-call-iptables: No such file or directory# modprobe br_netfilter# sysctl -q -w net.brIDge.brIDge-nf-call-iptables=0 # ok Now
我没有在我的香草RHEL 7.1上看到这些“破碎”的规则,它们的起源对我来说是神秘的.我试过了：
# modprobe -n -vvv brIDgemodprobe: INFO: custom logging function 0x40a130 registeredinsmod /lib/modules/3.10.0-229.11.1.el7.x86_64/kernel/net/llc/llc.koinsmod /lib/modules/3.10.0-229.11.1.el7.x86_64/kernel/net/802/stp.koinsmod /lib/modules/3.10.0-229.11.1.el7.x86_64/kernel/net/brIDge/brIDge.komodprobe: INFO: context 0xf1c270 released# echo "install brIDge echo example_of_a_modprobe_rule" > /etc/modprobe.d/zzz5.conf# modprobe -n -vvv brIDgemodprobe: INFO: custom logging function 0x40a130 registeredinsmod /lib/modules/3.10.0-229.11.1.el7.x86_64/kernel/net/llc/llc.koinsmod /lib/modules/3.10.0-229.11.1.el7.x86_64/kernel/net/802/stp.koinstall echo example_of_a_modprobe_rulemodprobe: INFO: context 0xeaa270 released# rm /etc/modprobe.d/zzz5.conf
更新：看起来像xenserver uses a similar modprobe hack.无论您是否实际运行xenserver,全局更改每个人的内核模块行为都是一个令人讨厌的错误.而这个错误已经向我们反击了.
更新2：现在,您发现/etc/modprobe.d/dist.conf导致此问题而不是docker.无论您是否有docker,modprobe brIDge将始终返回1并打印错误.通常,dist.conf是RHEL6上module-init-tools包的一部分.此文件不应在RHEL7上使用.它不在我的任何RHEL7系统上,它们运行得很好.在RHEL7中,包是kmod,它不包含dist.conf.我会：
rpm -qf /etc/modprobe.d/dist.conf # what package owns this file?
如果dist.conf不归包所有,请对其进行备份并删除任何不会给您带来任何明显好处的行(甚至可能完全删除该文件).
如果dist.conf由一个软件包拥有,请考虑删除/更新该软件包,因为它在RHEL 7.2兼容性方面明显有问题.
总结以上是内存溢出为你收集整理的linux-networking – 不从Docker容器转发的UDP流量 – > Docker主机全部内容，希望文章能够帮你解决linux-networking – 不从Docker容器转发的UDP流量 – > Docker主机所遇到的程序开发问题。
如果觉得内存溢出网站内容还不错，欢迎将内存溢出网站推荐给程序员好友。
欢迎分享，转载请注明来源：内存溢出
微信扫一扫
支付宝扫一扫
评论列表（0条）