| *** hongbin has joined #openstack-kuryr | 01:10 | |
| *** zufar_ has joined #openstack-kuryr | 03:57 | |
| *** pcaruana has joined #openstack-kuryr | 04:22 | |
| *** ndesh has joined #openstack-kuryr | 04:27 | |
| *** ndesh has quit IRC | 04:44 | |
| *** hongbin has quit IRC | 04:49 | |
| *** janki has joined #openstack-kuryr | 04:51 | |
| *** gcheresh has joined #openstack-kuryr | 04:58 | |
| *** jchhatbar has joined #openstack-kuryr | 05:27 | |
| *** janki has quit IRC | 05:30 | |
| *** yboaron_ has joined #openstack-kuryr | 05:36 | |
| *** aperevalov has joined #openstack-kuryr | 06:16 | |
| *** ccamposr has joined #openstack-kuryr | 06:59 | |
| *** takamatsu has joined #openstack-kuryr | 07:07 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Fix incombatilibity with openstacksdk <= 0.17.2 https://review.opendev.org/661335 | 07:09 |
|---|---|---|
| *** takamatsu has quit IRC | 07:13 | |
| dulek | zufar_: That would seem like you're missing kuryr-daemon for some reason? | 07:20 |
| dulek | zufar_: That article is quite old, maybe it doesn't account for the fact that kuryr-daemon is now required to run Kuryr-Kubernetes? | 07:20 |
| *** yboaron_ has quit IRC | 07:22 | |
| dulek | zufar_: If you run kuryr-daemon check out its logs. It should run where kubelet is running. | 07:22 |
| zufar_ | dulek: I try to install kuryr-kubernetes with devstack, the kubernetes deployment not install any dns in kubernetes | 07:28 |
| zufar_ | dulek: but my kubernetes instalation (using kubeadm) is installing coredns, and the coredns always error because liveness probe | 07:28 |
| zufar_ | dulek: all function is working now, except coredns feature, I can create pod in kubernetes, and my Instance can connect into it normally, I can create clusterip and loadbalancer service in kubernetes normally | 07:29 |
| dulek | zufar_: That log you put there shows that coredns pod haven't got wired by kuryr-daemon. | 07:29 |
| dulek | zufar_: Ah, okay, that was transient. | 07:30 |
| dulek | zufar_: And after those initial errors it started to run. | 07:30 |
| zufar_ | yes its starting to run | 07:30 |
| dulek | zufar_: Do you have logs of coredns pod? | 07:30 |
| zufar_ | dulek: this is make coredns always restarting, Liveness probe failed: Get http://10.1.2.37:8080/health: net/http: request canceled while waiting for connection | 07:31 |
| zufar_ | coredns try to probe him self, not kubernetes API | 07:31 |
| dulek | zufar_: Ah, so it seems kubelet is unable to reach coredns liveness probe? | 07:32 |
| zufar_ | I dont know why, but when I install flannel, coreDNS get work again, and I lost all connection between openstack instance and kubernetes pod | 07:32 |
| zufar_ | dulek: I think coreDNS pod get wrong ip address (10.1.2.37) for probe. | 07:33 |
| zufar_ | because 10.1.2.37 its coreDNS pod IP. | 07:33 |
| dulek | zufar_: Nah, that would be correct, the probe is probably just /health endpoint on the coredns server. | 07:34 |
| zufar_ | are coredns have /health endpoint? | 07:34 |
| dulek | zufar_: I'd start debugging this by looking at the SG's. Manually disable the liveness probe in coredns Deployment/DaemonSet definition and try to reach it manually. | 07:34 |
| zufar_ | I think the endpoint is in kubernetes-api | 07:34 |
| dulek | zufar_: Liveness endpoint? That wouldn't make any sense. Check out liveness probe definition in the coredns controller. | 07:35 |
| zufar_ | dulek: Sorry, so coreDNS have /health endpoint? | 07:38 |
| zufar_ | dulek: I think coreDNS try to check kubernetes API /health endpoint | 07:39 |
| zufar_ | dulek: If the coreDNS have /health endpoint, so IP address used is right | 07:42 |
| dulek | zufar_: Hey, that's how liveness probes work, right? | 07:43 |
| dulek | zufar_: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ | 07:43 |
| *** maysams has joined #openstack-kuryr | 07:43 | |
| dulek | zufar_: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#define-a-liveness-http-request | 07:43 |
| zufar_ | dulek: yes, so why the pod cannot contact himself? hm | 07:43 |
| dulek | zufar_: It's not the pod. It's kubelet that runs liveness probe. | 07:44 |
| zufar_ | dulek: wait a minute, I try to deploy the whole cluster | 07:44 |
| zufar_ | dulek: so we need another CNI like loopback? | 07:44 |
| dulek | zufar_: Like I said - disable the liveness probe checking, so the pod will stay alive. Check out its logs - maybe something's there. If not, try reaching the liveness probe from the host. If you can't, check out the SG's. | 07:44 |
| zufar_ | dulek: okey thank you | 07:45 |
| dulek | zufar_: Yeah, normally loopback was required for that stuff to work. | 07:45 |
| dulek | zufar_: Remember that this 10.1.2.37 will be a port on Neutron - that's how you track the SG's that are attached to the pod. | 07:45 |
| zufar_ | okey, so kubelet missing route into that pod right? | 07:47 |
| zufar_ | I use different architecture to deploy kuryr-kubernetes, I split the kubernetes from openstack (independent cluster) | 07:47 |
| zufar_ | and I install neutron-openvswitch-agent in all of kubernetes nodes. | 07:48 |
| dulek | zufar_: Do you have what we call "kubelet port" in DevStack? | 07:48 |
| dulek | zufar_: Because DevStack can get you a K8s as well. | 07:48 |
| dulek | zufar_: https://github.com/openstack/kuryr-kubernetes/blob/3e3ed9dbb31133b5175ff792a9636603b2df95e1/devstack/lib/kuryr_kubernetes#L15-L97 | 07:50 |
| dulek | zufar_: This is to enable kubelet on the host to connect to OpenStack Neutron networks. | 07:50 |
| dulek | zufar_: We simply put it on the pod network, IIRC. | 07:50 |
| zufar_ | dulek: okey I get the point, let me try first | 07:51 |
| zufar_ | dulek: thank you very much | 07:51 |
| *** yboaron_ has joined #openstack-kuryr | 08:34 | |
| *** yboaron_ has quit IRC | 08:40 | |
| *** yboaron_ has joined #openstack-kuryr | 08:41 | |
| *** jchhatba_ has joined #openstack-kuryr | 09:18 | |
| *** jchhatbar has quit IRC | 09:21 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-tempest-plugin master: detect failed curl when streamed from pod https://review.opendev.org/626892 | 09:55 |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Populate pools upon namespace creation https://review.opendev.org/656975 | 09:59 |
| *** ccamposr has quit IRC | 10:07 | |
| *** ccamposr has joined #openstack-kuryr | 10:25 | |
| openstackgerrit | Ilya Maximets proposed openstack/kuryr-kubernetes master: Add note about 'protoc' compiler version. https://review.opendev.org/661717 | 10:50 |
| openstackgerrit | Ilya Maximets proposed openstack/kuryr-kubernetes master: Add note about 'protoc' compiler version https://review.opendev.org/661717 | 10:50 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Fix incombatilibity with openstacksdk <= 0.17.2 https://review.opendev.org/661335 | 11:11 |
| *** ccamposr__ has joined #openstack-kuryr | 11:21 | |
| *** ccamposr has quit IRC | 11:22 | |
| *** rh-jelabarre has joined #openstack-kuryr | 11:51 | |
| openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Fix wrong behiviour of VIFHandler https://review.opendev.org/661742 | 12:12 |
| openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Fix wrong behiviour of VIFHandler https://review.opendev.org/661742 | 12:15 |
| openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Fix wrong behiviour of VIFHandler https://review.opendev.org/661742 | 12:56 |
| openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Fix wrong behaviour of VIFHandler https://review.opendev.org/661742 | 12:57 |
| *** jchhatba_ has quit IRC | 13:08 | |
| *** jchhatba_ has joined #openstack-kuryr | 13:08 | |
| openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Fix wrong behaviour of VIFHandler https://review.opendev.org/661742 | 13:17 |
| openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Fix wrong behaviour of VIFHandler https://review.opendev.org/661742 | 13:26 |
| openstackgerrit | Maysa de Macedo Souza proposed openstack/kuryr-kubernetes master: Fix CRD update on SG rule creation https://review.opendev.org/659634 | 13:38 |
| openstackgerrit | Merged openstack/kuryr-tempest-plugin master: detect failed curl when streamed from pod https://review.opendev.org/626892 | 13:48 |
| openstackgerrit | Maysa de Macedo Souza proposed openstack/kuryr-kubernetes master: Fix CRD update on SG rule creation https://review.opendev.org/659634 | 13:52 |
| *** jchhatba_ has quit IRC | 13:55 | |
| openstackgerrit | Maysa de Macedo Souza proposed openstack/kuryr-kubernetes master: Fix Pod creation do not triggers affected SVC SG update https://review.opendev.org/661569 | 14:24 |
| *** yboaron_ has quit IRC | 15:07 | |
| openstackgerrit | Merged openstack/kuryr-kubernetes stable/stein: NP: Create allow-all SG and add it to pod SG's https://review.opendev.org/652581 | 15:09 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Use proper log type when logging errors in cni https://review.opendev.org/656735 | 15:09 |
| *** gcheresh has quit IRC | 15:22 | |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-tempest-plugin master: Fixing service connectivity testing https://review.opendev.org/636329 | 15:28 |
| *** maysams has quit IRC | 16:20 | |
| *** irclogbot_1 has quit IRC | 17:17 | |
| *** irclogbot_0 has joined #openstack-kuryr | 17:18 | |
| *** gcheresh has joined #openstack-kuryr | 17:44 | |
| *** gcheresh has quit IRC | 18:06 | |
| *** gdwornicki has joined #openstack-kuryr | 18:15 | |
| *** gdwornicki has quit IRC | 18:22 | |
| *** gdwornicki has joined #openstack-kuryr | 18:36 | |
| *** gdwornicki has quit IRC | 18:49 | |
| *** gdwornicki has joined #openstack-kuryr | 18:54 | |
| *** ccamposr__ has quit IRC | 18:56 | |
| *** ccamposr has joined #openstack-kuryr | 18:58 | |
| zufar_ | Hi dulek | 19:17 |
| *** gcheresh has joined #openstack-kuryr | 19:17 | |
| zufar_ | I now create kubelet port and can ping pod network | 19:17 |
| zufar_ | but my kuryr-kubernetes controller always get error like | 19:18 |
| *** ccamposr has quit IRC | 19:18 | |
| zufar_ | ERROR kuryr_kubernetes.controller.drivers.lbaasv2 [-] Error when creating listener: {"debuginfo": null, "faultcode": "Client", "faultstring": "Another Listener on this Load Balancer is already using protocol_port 53"} | 19:18 |
| zufar_ | and my coreDNS error is change to 503 erroe | 19:18 |
| zufar_ | Liveness probe failed: HTTP probe failed with statuscode: 503 | 19:18 |
| zufar_ | Do you or anyone know what is happen? full log: https://gist.github.com/zufardhiyaulhaq/1cb4972d919f7c09bda7e640672e84d7 or http://paste.openstack.org/show/752183/ | 19:19 |
| *** takamatsu has joined #openstack-kuryr | 19:28 | |
| *** pcaruana has quit IRC | 19:30 | |
| *** takamatsu has quit IRC | 20:02 | |
| *** gcheresh has quit IRC | 20:14 | |
| *** gdwornicki has quit IRC | 20:30 | |
| *** takamatsu has joined #openstack-kuryr | 20:42 | |
| *** ccamposr has joined #openstack-kuryr | 20:46 | |
| *** ccamposr__ has joined #openstack-kuryr | 20:48 | |
| *** ccamposr has quit IRC | 20:51 | |
| *** ccamposr__ has quit IRC | 21:44 | |
| *** gdwornicki has joined #openstack-kuryr | 22:38 | |
| *** takamatsu has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!