| *** yamamoto has joined #openstack-kuryr | 00:34 | |
| *** yamamoto has quit IRC | 00:39 | |
| *** hongbin has joined #openstack-kuryr | 01:08 | |
| *** kiennt26 has joined #openstack-kuryr | 01:09 | |
| *** ltomasbo has quit IRC | 02:19 | |
| *** ltomasbo` has joined #openstack-kuryr | 02:19 | |
| *** kiennt26 has quit IRC | 02:19 | |
| *** ltomasbo` has quit IRC | 02:19 | |
| *** ltomasbo` has joined #openstack-kuryr | 02:19 | |
| *** yboaron has joined #openstack-kuryr | 02:41 | |
| *** yboaron_ has quit IRC | 02:43 | |
| *** hongbin has quit IRC | 03:39 | |
| *** yamamoto has joined #openstack-kuryr | 03:52 | |
| *** hongbin has joined #openstack-kuryr | 04:12 | |
| *** kiennt26 has joined #openstack-kuryr | 04:12 | |
| *** hongbin has quit IRC | 04:20 | |
| *** jappleii__ has quit IRC | 04:25 | |
| *** threestrands has joined #openstack-kuryr | 04:25 | |
| *** janki has joined #openstack-kuryr | 04:35 | |
| *** gcheresh has joined #openstack-kuryr | 04:40 | |
| *** yamamoto_ has joined #openstack-kuryr | 04:42 | |
| *** kiennt26 has quit IRC | 04:45 | |
| *** yamamoto has quit IRC | 04:45 | |
| *** yamamoto_ has quit IRC | 05:16 | |
| *** yamamoto has joined #openstack-kuryr | 05:35 | |
| *** itzikb has joined #openstack-kuryr | 06:03 | |
| *** ltomasbo` is now known as ltomasbo | 06:08 | |
| *** yboaron has quit IRC | 06:16 | |
| *** threestrands has quit IRC | 06:30 | |
| *** yboaron has joined #openstack-kuryr | 07:17 | |
| *** yboaron has quit IRC | 07:18 | |
| *** yboaron has joined #openstack-kuryr | 07:18 | |
| *** rraja has joined #openstack-kuryr | 07:27 | |
| *** pmannidi has quit IRC | 07:45 | |
| openstackgerrit | Itzik Brown proposed openstack/kuryr-kubernetes master: Running tests in serial https://review.openstack.org/579808 | 07:59 |
|---|---|---|
| *** gcheresh has quit IRC | 08:26 | |
| *** gcheresh has joined #openstack-kuryr | 08:31 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-tempest-plugin master: Test restart of CNI and Controller kuryr pods https://review.openstack.org/552498 | 08:37 |
| *** itzikb is now known as itzikb|afk | 08:43 | |
| *** garyloug has joined #openstack-kuryr | 08:47 | |
| *** gcheresh has quit IRC | 09:04 | |
| *** gcheresh has joined #openstack-kuryr | 09:05 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Switch to using stestr https://review.openstack.org/579253 | 09:19 |
| *** itzikb|afk is now known as itzikb | 09:33 | |
| *** itzikb is now known as itzikb|afk | 09:56 | |
| openstackgerrit | Merged openstack/kuryr-tempest-plugin master: Upgrade musl(from alpine) and busybox versions https://review.openstack.org/580699 | 10:04 |
| *** yamamoto has quit IRC | 10:15 | |
| *** yamamoto has joined #openstack-kuryr | 10:19 | |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Ensure isolation between namespaces https://review.openstack.org/579181 | 10:26 |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Ensure OpenShift gate uses the namespace subnet/sg drivers https://review.openstack.org/580680 | 10:27 |
| *** itzikb|afk is now known as itzikb | 10:35 | |
| *** yamamoto has quit IRC | 10:45 | |
| *** livelace has joined #openstack-kuryr | 10:49 | |
| livelace | Hello everybody, what does this error mean ? | 10:56 |
| livelace | 2018-07-09 12:04:21.868 130 ERROR kuryr_kubernetes.cni.daemon.service [-] Timed out waiting for requested pod to appear in registry: Resource not ready: | 10:56 |
| openstackgerrit | Shachar Snapiri proposed openstack/kuryr-kubernetes master: Move tempest-octavia-centos-7 back to check https://review.openstack.org/579599 | 10:57 |
| irenab | livelace, resource not ready exception allows to retry internally | 11:02 |
| livelace | irenab, I see a lot of such errors in daemonsets logs | 11:05 |
| livelace | Pods stuck and cannot work | 11:06 |
| irenab | livelace, do you run containarized? | 11:06 |
| livelace | irenab, What "registry" is it ? | 11:07 |
| *** livelace1 has joined #openstack-kuryr | 11:08 | |
| livelace1 | irenab: Yes, daemonset - containers inside Openshift Origin | 11:08 |
| livelace1 | "registry" - it's a registry inside k8s/origin ? | 11:10 |
| irenab | livelace, I think dulek is the best person to guide you at this issue | 11:10 |
| livelace1 | dulek: Ping | 11:10 |
| livelace1 | irenab: Thanks | 11:10 |
| *** livelace has quit IRC | 11:13 | |
| dulek | livelace1: Yeah, so basically kuryr-daemon watches K8s API for new pods appearing. | 11:14 |
| dulek | livelace1: Fact that the pod was not found may mean that connection to K8s API is broken or watcher died (see bug https://bugs.launchpad.net/kuryr-kubernetes/+bug/1776676). | 11:14 |
| openstack | Launchpad bug 1776676 in kuryr-kubernetes "CNI goes useless if the watcher loses connectivity with the API for longer than the retry timeout" [Critical,In progress] - Assigned to Antoni Segura Puimedon (celebdor) | 11:14 |
| livelace1 | dulek: ok, but pods stuck with error like "CNI daemon return error" | 11:15 |
| livelace1 | dulek: I catch this error from time to time, but I cannot understand why | 11:15 |
| dulek | livelace1: Sure, because after some timeout kuryr-daemon returns that it couldn't found the pod, so kubelet may retry the request. | 11:15 |
| livelace1 | dulek: one moment please, i'm reading the bug | 11:16 |
| livelace1 | dulek: The fix is not included in lates Docker images of CNI daemon. I understand it correctly ? | 11:19 |
| dulek | livelace1: It's not, not even merged. :( | 11:19 |
| dulek | livelace1: Is it happening always on a single node? | 11:19 |
| dulek | livelace1: Can you check out if watcher haven't stopped there? It's a log message similar to "Watcher stopped". | 11:20 |
| livelace1 | dulek: yes, cni and controller probes are in disabled state. the problem exists on all nodes. | 11:20 |
| livelace1 | dulek: No, no records that contain 'grep -i stopped' | 11:22 |
| dulek | livelace1: You mean unhealthy state? So maybe they are disconnected from K8s API? | 11:22 |
| livelace1 | dulek: I restart daemonsets by hand, that, I guess helps to avoid the problem | 11:23 |
| dulek | livelace1: You can also try one workaround - set this option to an hour, or something: https://github.com/openstack/kuryr-kubernetes/blob/e9fd1004ab9ef843b3d168b278c2775d1f01ab6c/kuryr_kubernetes/config.py#L153-L155 | 11:24 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: (WIP) Implement Network Policies Driver https://review.openstack.org/580982 | 11:26 |
| livelace1 | dulek: Once again, for sure. If readiness probes are disabled for CNI daemon - the bug above happens, yes ? | 11:27 |
| livelace1 | dulek: I mean those probes https://paste.fedoraproject.org/paste/~Ui30hWdk0DJY7QM-9r55A | 11:28 |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-tempest-plugin master: Namespace isolation tempest coverage https://review.openstack.org/580678 | 11:28 |
| dulek | livelace1: Yes, with probes kuryr-daemon will get restarted eventually and will reconnect. | 11:29 |
| dulek | livelace1: The workaround I've provided extends the timeout of reconnecting to K8s API, so even without the probes kuryr-daemon will retry connection. | 11:30 |
| livelace1 | dulek: I will try your WA and I will try to enable probes for controller and cni daemons | 11:30 |
| livelace1 | dulek: Thanks for your help! | 11:30 |
| dulek | livelace1: No problem! | 11:30 |
| *** yamamoto has joined #openstack-kuryr | 11:33 | |
| *** atoth has joined #openstack-kuryr | 11:34 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Switch to using stestr https://review.openstack.org/579253 | 11:35 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Add octavia precreated image info https://review.openstack.org/581336 | 11:57 |
| openstackgerrit | Genadi Chereshnya proposed openstack/kuryr-tempest-plugin master: Testing VM to loadBalancer service connectivity https://review.openstack.org/571968 | 12:00 |
| openstackgerrit | Genadi Chereshnya proposed openstack/kuryr-tempest-plugin master: Create service with unsupported type https://review.openstack.org/581337 | 12:00 |
| *** tzumainn has joined #openstack-kuryr | 12:07 | |
| *** janki has quit IRC | 12:24 | |
| *** yamamoto has quit IRC | 12:26 | |
| *** yamamoto has joined #openstack-kuryr | 12:26 | |
| openstackgerrit | Gary Loughnane proposed openstack/kuryr-kubernetes master: Add DPDK support for nested pods https://review.openstack.org/559363 | 12:29 |
| *** itzikb has quit IRC | 12:51 | |
| dulek | dmellado: Hey, I think I've figured out requirements issue. | 13:14 |
| dulek | dmellado: Looks like requirements-check job is running only on patches that change requirements. | 13:15 |
| dulek | dmellado: Both of patches failing on Flask version are doing that. | 13:15 |
| dulek | dmellado: I'll take a look after I reconnect my router. Did I ever said how much I hate cables? | 13:15 |
| *** janki has joined #openstack-kuryr | 13:18 | |
| *** yboaron_ has joined #openstack-kuryr | 13:34 | |
| *** yboaron has quit IRC | 13:36 | |
| *** hongbin has joined #openstack-kuryr | 13:40 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Sync up Flask version with global-requirements https://review.openstack.org/581373 | 14:06 |
| dulek | dmellado, irenab: ^ This should fix the patches that have issues with check-requirements job failing. | 14:07 |
| dmellado | hey dulek | 14:07 |
| dmellado | let me check, I didn't see your pings ;) | 14:07 |
| dmellado | lol, dulek what happened to your router? | 14:07 |
| dulek | dmellado: It gets restarted every 30-60 minutes. | 14:08 |
| dulek | And of course the only thing that the guy on the line could help me was how to reset the router. | 14:08 |
| dulek | Even though I pointed out connection timeouts in router's logs. | 14:08 |
| dulek | So it's now my turn to get some fun with dealing with ISP. :D | 14:09 |
| dmellado | LOL | 14:10 |
| dmellado | dulek: what would you do that? They'd just say | 14:10 |
| dmellado | did you try to turn it off and on again? xD | 14:10 |
| dmellado | at least the spanish ones xD | 14:10 |
| dulek | dmellado: Nah, full settings reset. The guy might suspected I've played with channel ranges or other stuff. Still he could just asked. | 14:11 |
| dulek | dmellado: I guess it's because most of the people calling are: #1 Rude, #2 Non-technical, #3 They've actually broken that themselves. | 14:12 |
| ltomasbo | yboaron_, sorry, I got connectivity issues... | 14:12 |
| dmellado | dulek: I've got a friend who once worked in some kind of 'internet access center for elderly people' | 14:13 |
| * dulek guesses Internet is just broken worldwide today. | 14:13 | |
| dmellado | who quit after a couple of months | 14:13 |
| dmellado | but shared some totally juicy stories such as | 14:13 |
| dmellado | do 'double click' there | 14:13 |
| dmellado | and they did that, they started to say 'click click' | 14:13 |
| dmellado | all over | 14:13 |
| dmellado | xD | 14:13 |
| dmellado | so I guess that kind of work (1-2-3) could get to be stressful | 14:13 |
| dmellado | ltomasbo: you too? xD | 14:14 |
| dmellado | did you try turning it off and on again? | 14:14 |
| ltomasbo | almost... | 14:14 |
| dulek | dmellado: I've also worked few months as an admin at the… social welfare home? Nursing home? | 14:14 |
| dmellado | oh, that ought to be tough as well xD | 14:14 |
| dulek | dmellado: So of course I was also asked to "take a look" at laptops of residents there. | 14:15 |
| dmellado | OF COURSE xD | 14:15 |
| dmellado | which would of course too be full of malware and virus crap | 14:15 |
| dmellado | and weird stuff | 14:15 |
| dulek | dmellado: Yeah. And you kinda suspect how they've got it. | 14:16 |
| dmellado | yeah, 'kinda' xD | 14:16 |
| dmellado | dulek: I just hoped that they paid you *really well* | 14:16 |
| dmellado | for that kind of job xD | 14:16 |
| dulek | dmellado: But I've also got to work with a blind guy that tried to learn using computer. He had all the screen reader software and memorized the keyboard on his own. | 14:17 |
| dmellado | wow, that's impressive! | 14:17 |
| dulek | That was really a rewarding experience. I mean I'll never design a website or UI that'll be non-accesible. | 14:17 |
| *** gcheresh has quit IRC | 14:21 | |
| dmellado | that's really evil, I guess those guys should hate flash with passion | 14:29 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Proceed CNI output in format of version 0.3.1 https://review.openstack.org/580162 | 14:35 |
| *** yboaron_ has quit IRC | 14:36 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Sync up Flask version with global-requirements https://review.openstack.org/581373 | 14:36 |
| *** yboaron_ has joined #openstack-kuryr | 14:38 | |
| *** yboaron_ has quit IRC | 14:56 | |
| *** yboaron has joined #openstack-kuryr | 14:57 | |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: (WIP) Implement Network Policies Driver https://review.openstack.org/580982 | 15:02 |
| *** yboaron has quit IRC | 15:08 | |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Add default namespace project driver https://review.openstack.org/581397 | 15:21 |
| *** rh-jelabarre has quit IRC | 15:28 | |
| *** yamamoto has quit IRC | 15:29 | |
| *** yamamoto has joined #openstack-kuryr | 15:31 | |
| dulek | dmellado: Current version of https://review.openstack.org/#/c/581373 passes check-requirements. | 15:31 |
| dmellado | dulek: awesome | 15:31 |
| dmellado | dulek: there you go, man | 15:32 |
| dulek | Thanks! | 15:32 |
| *** yamamoto has quit IRC | 15:34 | |
| *** rh-jelabarre has joined #openstack-kuryr | 15:35 | |
| *** rh-jelabarre has quit IRC | 15:41 | |
| *** rh-jelabarre has joined #openstack-kuryr | 15:44 | |
| *** rraja has quit IRC | 15:57 | |
| *** janki has quit IRC | 16:11 | |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: (WIP) Implement Network Policies Driver https://review.openstack.org/580982 | 16:14 |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Ensure isolation between namespaces https://review.openstack.org/579181 | 16:32 |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Add namespace isolation for services https://review.openstack.org/581421 | 16:32 |
| *** yamamoto has joined #openstack-kuryr | 16:34 | |
| *** yamamoto has quit IRC | 16:40 | |
| *** rh-jelabarre has quit IRC | 16:52 | |
| *** maysamacedos has joined #openstack-kuryr | 16:55 | |
| *** atoth has quit IRC | 16:57 | |
| *** atoth has joined #openstack-kuryr | 17:00 | |
| *** rh-jelabarre has joined #openstack-kuryr | 17:05 | |
| *** garyloug has quit IRC | 17:26 | |
| *** livelace has joined #openstack-kuryr | 17:32 | |
| *** yamamoto has joined #openstack-kuryr | 17:37 | |
| *** atoth_ has joined #openstack-kuryr | 17:41 | |
| *** yamamoto has quit IRC | 17:41 | |
| *** atoth has quit IRC | 17:44 | |
| *** atoth_ has quit IRC | 17:45 | |
| *** atoth has joined #openstack-kuryr | 17:46 | |
| *** atoth has quit IRC | 17:48 | |
| *** yamamoto has joined #openstack-kuryr | 18:39 | |
| *** yamamoto has quit IRC | 18:44 | |
| *** itzikb has joined #openstack-kuryr | 18:59 | |
| *** gcheresh has joined #openstack-kuryr | 19:01 | |
| *** atoth has joined #openstack-kuryr | 19:06 | |
| *** livelace has quit IRC | 19:07 | |
| *** maysamacedos has quit IRC | 19:12 | |
| *** irenab has quit IRC | 19:18 | |
| *** irenab has joined #openstack-kuryr | 19:18 | |
| *** irenab has quit IRC | 19:28 | |
| *** irenab has joined #openstack-kuryr | 19:29 | |
| *** yamamoto has joined #openstack-kuryr | 19:32 | |
| *** maysamacedos has joined #openstack-kuryr | 19:35 | |
| *** yamamoto has quit IRC | 19:37 | |
| *** itzikb has quit IRC | 19:42 | |
| *** gcheresh has quit IRC | 19:44 | |
| *** yboaron has joined #openstack-kuryr | 20:00 | |
| *** maysamacedos has quit IRC | 20:14 | |
| *** atoth has quit IRC | 20:27 | |
| *** atoth has joined #openstack-kuryr | 20:27 | |
| *** yamamoto has joined #openstack-kuryr | 20:33 | |
| *** yamamoto has quit IRC | 20:39 | |
| *** irenab has quit IRC | 21:05 | |
| *** irenab has joined #openstack-kuryr | 21:05 | |
| *** rh-jelabarre has quit IRC | 21:08 | |
| *** lihi has quit IRC | 21:10 | |
| *** lihi has joined #openstack-kuryr | 21:11 | |
| *** lihi has quit IRC | 21:15 | |
| *** lihi has joined #openstack-kuryr | 21:18 | |
| openstackgerrit | Yossi Boaron proposed openstack/kuryr-tempest-plugin master: OCP-Router - add support OCP routes https://review.openstack.org/578310 | 21:48 |
| *** yboaron has quit IRC | 22:13 | |
| *** yamamoto has joined #openstack-kuryr | 22:36 | |
| *** yamamoto has quit IRC | 22:42 | |
| *** hongbin has quit IRC | 23:06 | |
| *** yamamoto has joined #openstack-kuryr | 23:38 | |
| *** yamamoto has quit IRC | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!