[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [aos-devel] OVS / SDN issues with latest origin code



I updated only the openshift binary. For a long time this has worked, needing only documented upgrade steps. Looks like this time I need to update from openshift-sdn HEAD too.

On Wed, Oct 7, 2015 at 11:53 AM, Dan Winship <danw redhat com> wrote:
On 10/07/2015 10:53 AM, Luke Meyer wrote:
I updated my local cluster's (CentOS ansible install) openshift binary
to latest code and it broke horribly; looked like OVS setup was failing.

Based on the logs, you have a new /usr/bin/openshift, but still have the old /usr/bin/openshift-sdn-kube-subnet-setup.sh. Some sort of packaging bug?

-- Dan

I did a yum update -y (overwriting my binary with the Origin 1.0.6 RPMs
and getting my systemd units updated from openshift-* to origin-*) and
it magically worked again. Then I updated the binary again and this is
the error log out of origin-node:

Oct 07 10:47:17 node1.dev.example.com <http://node1.dev.example.com>
ovs-vsctl[16351]: ovs|00001|vsctl|INFO|Called as ovs-vsctl del-br br0
Oct 07 10:47:17 node1.dev.example.com <http://node1.dev.example.com>
ovs-vsctl[16361]: ovs|00001|vsctl|INFO|Called as ovs-vsctl add-br br0 --
set Bridge br0 fail-mode=secure
Oct 07 10:47:17 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: I1007 10:47:17.981524   16327 manager.go:158]
Machine: {NumCores:1 CpuFrequency:2793524 MemoryCapacity:3975933952
MachineID:db20b5e220e04ea8b47ee83bb1304
Oct 07 10:47:17 node1.dev.example.com <http://node1.dev.example.com>
ovs-vsctl[16373]: ovs|00001|vsctl|INFO|Called as ovs-vsctl set bridge
br0 protocols=OpenFlow13
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: I1007 10:47:18.015566   16327 manager.go:164]
Version: {KernelVersion:3.10.0-229.11.1.el7.x86_64
ContainerOsVersion:Employee SKU DockerVersion:1.7.1 Cadv
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: I1007 10:47:18.016310   16327 server.go:730] Adding
manifest file: /etc/origin/node/static.pods
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: I1007 10:47:18.016341   16327 server.go:740]
Watching apiserver
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
ovs-vsctl[16378]: ovs|00001|vsctl|INFO|Called as ovs-vsctl add-port br0
vxlan0 -- set Interface vxlan0 type=vxlan options:remote_ip=flow
options:key=flow ofport_request=1
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
ovs-vsctl[16381]: ovs|00001|vsctl|INFO|Called as ovs-vsctl add-port br0
tun0 -- set Interface tun0 type=internal ofport_request=2
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: I1007 10:47:18.033435   16327 node.go:197] Started
Kubernetes Proxy on 0.0.0.0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
ovs-vsctl[16402]: ovs|00001|vsctl|INFO|Called as ovs-vsctl add-port br0
vovsbr -- set Interface vovsbr ofport_request=9
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: I1007 10:47:18.125696 16327 kube.go:27] Output of
setup script:
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + lock_file=/var/lock/openshift-sdn.lock
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + subnet_gateway=10.1.1.1
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + subnet=10.1.1.0/24 <http://10.1.1.0/24>
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + cluster_subnet=24
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + subnet_mask_len=10.1.0.0/16 <http://10.1.0.0/16>
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + tun_gateway=172.30.0.0/16 <http://172.30.0.0/16>
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + mtu=1450
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + printf 'Container network is "%s"; local host has
subnet "%s", mtu "%d" and gateway "%s".\n' 24 10.1.1.0/24
<http://10.1.1.0/24> 1450 10.1.1.1
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: Container network is "24"; local host has subnet
"10.1.1.0/24 <http://10.1.1.0/24>", mtu "1450" and gateway "10.1.1.1".
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + TUN=tun0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + set +e
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + setup_required
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: +++ awk '/inet / {print $2}'
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: +++ ip a s lbr0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: ++ echo
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip=
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + '[' '' '!=' 10.1.1.1/10.1.0.0/16
<http://10.1.1.1/10.1.0.0/16> ']'
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + return 0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + set -e
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + lockwrap setup
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + flock 200
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + setup
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + rm -f /etc/openshift-sdn/config.env
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl del-br br0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl add-br br0 -- set Bridge br0
fail-mode=secure
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl set bridge br0 protocols=OpenFlow13
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl del-port br0 vxlan0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: ovs-vsctl: no port named vxlan0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + true
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl add-port br0 vxlan0 -- set Interface
vxlan0 type=vxlan options:remote_ip=flow options:key=flow ofport_request=1
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl add-port br0 tun0 -- set Interface tun0
type=internal ofport_request=2
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link del vlinuxbr
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link add vlinuxbr type veth peer name vovsbr
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link set vlinuxbr up
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link set vovsbr up
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link set vlinuxbr txqueuelen 0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link set vovsbr txqueuelen 0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl del-port br0 vovsbr
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: ovs-vsctl: no port named vovsbr
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + true
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ovs-vsctl add-port br0 vovsbr -- set Interface
vovsbr ofport_request=9
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip link set lbr0 down
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + brctl delbr lbr0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + brctl addbr lbr0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + ip addr add 10.1.1.1/10.1.0.0/16
<http://10.1.1.1/10.1.0.0/16> dev lbr0
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: Error: an inet prefix is expected rather than
"10.1.1.1/10.1.0.0/16 <http://10.1.1.1/10.1.0.0/16>".
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: E1007 10:47:18.125726   16327 kube.go:37] Error
executing setup script.
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: Output: + lock_file=/var/lock/openshift-sdn.lock
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + subnet_gateway=10.1.1.1
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + subnet=10.1.1.0/24 <http://10.1.1.0/24>
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + cluster_subnet=24
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + subnet_mask_len=10.1.0.0/16 <http://10.1.0.0/16>
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + tun_gateway=172.30.0.0/16 <http://172.30.0.0/16>
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + mtu=1450
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
origin-node[16327]: + printf 'Container network is "%s"; local host has
subnet "%s", mtu "%d" and gateway "%s".\n' 24 10.1.1.0/24
<http://10.1.1.0/24> 1450 10.1.1.1
Oct 07 10:47:18 node1.dev.example.com <http://node1.dev.example.com>
systemd[1]: origin-node.service: main process exited, code=exited,
status=255/n/a

Has ovs config changed since 1.0.6? Any docs to point at?





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]