| *** yamamoto has joined #openstack-kuryr | 00:02 | |
| *** yamamoto has quit IRC | 00:07 | |
| *** hongbin has quit IRC | 00:16 | |
| *** maysamacedos has quit IRC | 00:42 | |
| *** dougbtv_ has joined #openstack-kuryr | 00:57 | |
| *** dougbtv has quit IRC | 00:58 | |
| *** yamamoto has joined #openstack-kuryr | 01:03 | |
| *** yamamoto has quit IRC | 01:08 | |
| *** slaweq has joined #openstack-kuryr | 01:17 | |
| *** slaweq has quit IRC | 01:21 | |
| *** slaweq has joined #openstack-kuryr | 01:54 | |
| *** maysamacedos has joined #openstack-kuryr | 01:56 | |
| *** slaweq has quit IRC | 01:59 | |
| *** salv-orl_ has joined #openstack-kuryr | 02:03 | |
| *** yamamoto has joined #openstack-kuryr | 02:04 | |
| *** salv-orlando has quit IRC | 02:07 | |
| *** hongbin has joined #openstack-kuryr | 02:08 | |
| *** yamamoto has quit IRC | 02:09 | |
| *** slaweq has joined #openstack-kuryr | 02:58 | |
| *** slaweq has quit IRC | 03:03 | |
| *** yamamoto has joined #openstack-kuryr | 03:06 | |
| *** kiennt26 has joined #openstack-kuryr | 03:09 | |
| *** yamamoto has quit IRC | 03:11 | |
| *** slaweq has joined #openstack-kuryr | 03:54 | |
| *** slaweq has quit IRC | 03:59 | |
| *** yamamoto has joined #openstack-kuryr | 04:07 | |
| *** slaweq has joined #openstack-kuryr | 04:13 | |
| *** yamamoto has quit IRC | 04:13 | |
| *** slaweq has quit IRC | 04:17 | |
| *** janonymous has joined #openstack-kuryr | 04:35 | |
| *** maysamacedos has quit IRC | 04:53 | |
| *** hongbin has quit IRC | 04:56 | |
| *** yboaron has joined #openstack-kuryr | 04:59 | |
| *** yamamoto has joined #openstack-kuryr | 05:04 | |
| *** slaweq has joined #openstack-kuryr | 05:30 | |
| *** slaweq has quit IRC | 05:34 | |
| *** janki has joined #openstack-kuryr | 05:42 | |
| *** yamamoto has quit IRC | 05:51 | |
| *** yamamoto has joined #openstack-kuryr | 06:02 | |
| *** pmannidi has quit IRC | 06:04 | |
| *** gcheresh has joined #openstack-kuryr | 06:06 | |
| *** kaliya has quit IRC | 06:10 | |
| *** Natanbro has joined #openstack-kuryr | 06:14 | |
| *** juriarte has joined #openstack-kuryr | 06:30 | |
| *** kaliya has joined #openstack-kuryr | 07:14 | |
| *** slaweq has joined #openstack-kuryr | 07:38 | |
| *** pcaruana has joined #openstack-kuryr | 07:47 | |
| *** openstackgerrit has joined #openstack-kuryr | 08:35 | |
| openstackgerrit | Nguyen Hung Phuong proposed openstack/kuryr-libnetwork master: Clean imports in code https://review.openstack.org/546533 | 08:35 |
|---|---|---|
| *** garyloug has joined #openstack-kuryr | 09:08 | |
| dulek | irenab: Can you take a look on this backport: https://review.openstack.org/#/c/546182/ ? It's fixing 2 critical issues and today's the deadline for another RC. | 09:43 |
| *** vikasc has quit IRC | 09:44 | |
| celebdor | dulek: wasn't it tomorrow? | 09:54 |
| irenab | dulek, sure | 09:55 |
| *** kiennt26 has quit IRC | 09:57 | |
| dulek | celebdor: Deadlines, deadline. Checking again. | 09:58 |
| dulek | celebdor's right: Just a quick reminder that Thursday, 22 March, is the deadline for any final | 09:58 |
| dulek | Queens release candidates. | 09:58 |
| dulek | (from smcginnis email) | 09:58 |
| dulek | Anyway it's better to be wrong in that direction. :D | 09:58 |
| *** vikasc has joined #openstack-kuryr | 09:58 | |
| dulek | BTW - smcginnis made a mistake too - it's not "March", but February. :P | 09:59 |
| irenab | dulek, done | 10:00 |
| dulek | irenab: Thank you! | 10:01 |
| *** pcaruana has quit IRC | 10:05 | |
| celebdor | dulek: fix the typo in https://review.openstack.org/#/c/545363/3 | 10:06 |
| celebdor | then irenab and I can merge this | 10:06 |
| dulek | celebdor: However lame that sounds I've fought with my email client this morning, so I haven't 100% verified it yet. | 10:07 |
| celebdor | dulek: which client? | 10:07 |
| dulek | celebdor: I need maybe 30 minutes to run it again with SG different than default and make sure it reverts LBaaS correctly. | 10:08 |
| dulek | celebdor: Evolution + Gnome's Google integration. It's okay now, dnf upgrade + reboot helped somehow. | 10:08 |
| celebdor | :-) | 10:08 |
| celebdor | dnf upgrade usually breaks more than it fixes when you run it just to solve an issue :P | 10:09 |
| dulek | celebdor: I guess I'm lucky today…? :D | 10:10 |
| celebdor | dulek: don't get too confident | 10:10 |
| celebdor | it will bite you when you least expect it | 10:10 |
| *** pcaruana has joined #openstack-kuryr | 10:20 | |
| openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Add multi pools support https://review.openstack.org/528345 | 10:33 |
| *** aojeagarcia has quit IRC | 10:34 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Services: Set missing SGs for haproxy provider https://review.openstack.org/545363 | 10:50 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Add timeout to the wait_for method in plugin.sh https://review.openstack.org/545619 | 10:50 |
| dulek | celebdor, irenab, yboaron: https://review.openstack.org/545363 - updated. | 10:50 |
| celebdor | dulek: did you verify it? | 10:51 |
| dulek | celebdor: Yup. It deletes the LBaaS on error and sets the correct SG on the port. | 10:52 |
| celebdor | dulek: good | 10:52 |
| dulek | celebdor: Plus I've played with rules a bit and was able to enable/disable traffic. | 10:53 |
| celebdor | let's get this in | 10:53 |
| celebdor | then, for release, we still miss the patch for N-S | 10:53 |
| celebdor | that sets the listener ports to an SG per listener or LB | 10:53 |
| dulek | celebdor: Is that difficult? Sounds like a very similar patch, but done on ensure_listener? | 10:54 |
| celebdor | dulek: it's simple enough | 10:54 |
| celebdor | I can even give whoever volunteers sample code | 10:55 |
| dulek | celebdor: Let's find a volunteer in the meeting (though it'll probably be me anyway). | 10:55 |
| * dulek runs to grab a coffee before meeting. | 10:56 | |
| celebdor | https://github.com/openstack/octavia/blob/master/octavia/network/drivers/neutron/allowed_address_pairs.py#L133 | 10:56 |
| celebdor | dulek: yboaron: Whoever takes it | 10:56 |
| celebdor | here's how octavia does it | 10:56 |
| *** yboaron has quit IRC | 11:05 | |
| *** yboaron has joined #openstack-kuryr | 11:06 | |
| *** vikasc has quit IRC | 11:11 | |
| *** pcaruana has quit IRC | 11:19 | |
| *** vikasc has joined #openstack-kuryr | 11:25 | |
| irenab | dulek, looking into the patch | 11:28 |
| dulek | irenab: Thanks! :) | 11:28 |
| *** pcaruana has joined #openstack-kuryr | 11:33 | |
| *** danil has joined #openstack-kuryr | 11:40 | |
| *** yamamoto has quit IRC | 11:41 | |
| *** yboaron has quit IRC | 11:58 | |
| *** janki has quit IRC | 12:07 | |
| *** yboaron has joined #openstack-kuryr | 12:08 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Run OpenShift 3.7.1 in DevStack plugin https://review.openstack.org/523370 | 12:10 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-tempest-plugin master: Merge create and list pods tests https://review.openstack.org/546579 | 12:12 |
| dmellado | dulek: for your sake and enjoy ^^ | 12:13 |
| dulek | dmellado: Makes sense, thanks! | 12:14 |
| *** celebdor has quit IRC | 12:18 | |
| dmellado | irenab: any reason not to +2 on the gate patches? | 12:24 |
| dmellado | I'd like to get that patch merged this week before PTG without being a BDFL | 12:24 |
| dmellado | xD | 12:25 |
| irenab | dmellado, link? | 12:25 |
| dmellado | https://review.openstack.org/#/c/544959/ | 12:25 |
| irenab | dmellado, the only reason is distruction to other stuff .. | 12:25 |
| dmellado | what do you mean by distruction? | 12:25 |
| irenab | dmellado, where can I see the failures of DF/ovn jobs? | 12:26 |
| dmellado | oh, they're there | 12:26 |
| dmellado | http://logs.openstack.org/59/544959/32/check/kuryr-kubernetes-tempest-dragonflow/54ae5e4/controller/logs/test_results.html.gz | 12:26 |
| dmellado | seems to be an issue with the ovs firewall | 12:27 |
| dmellado | which affects both ovn, dragonflow and on | 12:27 |
| dmellado | also oanson was speaking about some issue on DF which looked really similar to that | 12:27 |
| dulek | Quick thought - aren't we missing python-neutronclient in requirements.txt? | 12:29 |
| dmellado | dulek: did you see any failure? | 12:30 |
| dulek | dmellado: Nope, just noticed that we import it but it's not in requirements.txt. | 12:30 |
| dmellado | oh, do we? | 12:31 |
| dmellado | then yeah, it should be on requirements | 12:31 |
| dmellado | most probably it's just pulled by some another requirement | 12:31 |
| * dmellado checking | 12:31 | |
| *** celebdor has joined #openstack-kuryr | 12:31 | |
| dulek | dmellado: Or it's installed because of Neutron and that's why it works. | 12:32 |
| dmellado | dulek: it actually is installed by neutron | 12:33 |
| dmellado | https://github.com/openstack/neutron/blob/master/requirements.txt#L18 | 12:34 |
| dmellado | so I'd say that only if we do think on a neutron-less kuryr | 12:34 |
| dmellado | it would affect us | 12:35 |
| celebdor | dmellado: what are you talking about? | 12:35 |
| dmellado | that said, if we're importing that within our code I guess we can put that inside requirements | 12:35 |
| dmellado | celebdor: it seems that we do import python-neutronclient within kuryr but don't have it on our requirements | 12:35 |
| dmellado | it comes bundled with neutron, but it'd be safer to add that as reqs from our side | 12:35 |
| celebdor | we need to add it to requirements | 12:36 |
| celebdor | I'd say | 12:36 |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Services: Set missing SGs for haproxy provider https://review.openstack.org/545363 | 12:36 |
| dulek | celebdor, irenab: ^ I've added the constant for 'haproxy' phrase. Thanks for pointing that our irenab. | 12:36 |
| * dulek goes for lunch. | 12:37 | |
| dmellado | dulek: could you let me know where actually do we import it | 12:37 |
| dmellado | I can't find it | 12:37 |
| dulek | dmellado: Oh, whoops. | 12:38 |
| dulek | dmellado: My bad, it's kuryrlib. :P | 12:38 |
| dulek | dmellado: My IDE jumped to packaged file and I haven't noticed. | 12:38 |
| dulek | Sorry for confusing you guys! | 12:38 |
| dmellado | dulek: heh, I guess that settles the topic, no worries! xD | 12:38 |
| celebdor | :-) | 12:39 |
| * dmellado goes away | 12:39 | |
| dmellado | ttyl folks | 12:39 |
| celebdor | dulek: are you trying https://review.openstack.org/#/c/523370/2 ? | 12:40 |
| celebdor | (since I see the W-1) | 12:40 |
| celebdor | irenab: now that it has the const, you can merge https://review.openstack.org/#/c/545363/5 | 12:42 |
| *** maysamacedos has joined #openstack-kuryr | 12:42 | |
| celebdor | :-) | 12:42 |
| *** yamamoto has joined #openstack-kuryr | 12:42 | |
| dulek | celebdor: I'm waiting for Jenkins. I wasn't sure if openshift gates are voting, hence -W. | 12:42 |
| * dulek now really goes to eat. ;) | 12:42 | |
| dmellado | dulek: there's no Jenkins, only zuul | 12:42 |
| dmellado | xD | 12:42 |
| * dmellado now really goes away | 12:42 | |
| celebdor | dulek: Smacznego! | 12:44 |
| irenab | dulek, done | 12:45 |
| *** yamamoto has quit IRC | 12:49 | |
| *** yamamoto has joined #openstack-kuryr | 12:51 | |
| *** kiennt26 has joined #openstack-kuryr | 13:00 | |
| dulek | celebdor: https://review.openstack.org/#/c/523370/ - Zull is all green on this one. :) | 13:10 |
| dulek | (OpenShift 3.7.1) | 13:11 |
| *** garyloug has quit IRC | 13:12 | |
| *** kaliya has quit IRC | 13:15 | |
| *** kaliya has joined #openstack-kuryr | 13:17 | |
| *** janki has joined #openstack-kuryr | 13:21 | |
| *** kaliya has quit IRC | 13:21 | |
| *** kaliya has joined #openstack-kuryr | 13:25 | |
| *** danil has quit IRC | 13:43 | |
| dulek | celebdor: Can you kick https://review.openstack.org/#/c/545363/ in? | 13:56 |
| dulek | celebdor: Ah, you've already did. Thanks! | 13:56 |
| celebdor | dulek: you know I'm always eager to 2+1 | 13:57 |
| celebdor | w+1 | 13:57 |
| *** garyloug has joined #openstack-kuryr | 14:00 | |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Services: Set missing SGs for haproxy provider https://review.openstack.org/545363 | 14:04 |
| dulek | ^ I've cherry-picked this into the stable/queens: https://review.openstack.org/#/c/546640/ | 14:07 |
| dulek | yboaron: Can you explain a bit what needs to be done to fix LBaaS SGs for north-south traffic? | 14:13 |
| dulek | yboaron: I've looked through Octavia's code and played a bit with `neutron lbaas-*` commands, but I'm still not sure what's exactly needs to be done. | 14:14 |
| yboaron | dulek, So , for each service of type LoadBalancer we should | 14:14 |
| yboaron | dulek, 1. create LB and add rule for each listener protocols,port to | 14:15 |
| yboaron | dulek, and of course add this SG to LB VIP | 14:15 |
| celebdor | dulek: did you check the octavia link I sent earlier? | 14:15 |
| dulek | celebdor: Yup, I'm asking because I don't understand everything there. | 14:16 |
| dulek | yboaron: Your second message - you mean SG, not LB, right? | 14:16 |
| yboaron | dulek, I meant updated VIP port to include this SG | 14:17 |
| dulek | yboaron: So the *port* I'm adding the SG to is the same as for east-west? | 14:17 |
| yboaron | dulek, Not sure I understood U , you should 'open' the listeners protocol,port in SG | 14:18 |
| yboaron | dulek, by crating a rule for each LB's listener | 14:18 |
| yboaron | dulek, creating | 14:19 |
| celebdor | from 0.0.0.0/0 | 14:19 |
| celebdor | ideally the remote cidr would be configurable | 14:19 |
| celebdor | but I think 0.0.0.0/0 is fine for now | 14:19 |
| yboaron | dulek/ celebdor , can you send pointer to Octavia code ? | 14:20 |
| dulek | yboaron: Now "for each service of type LoadBalancer" | 14:20 |
| dulek | Aww. :P | 14:20 |
| dulek | yboaron: https://github.com/openstack/octavia/blob/master/octavia/network/drivers/neutron/allowed_address_pairs.py#L133 | 14:20 |
| dulek | yboaron: Now "for each service of type LoadBalancer" you mean K8s service? | 14:20 |
| yboaron | dulek, correct | 14:21 |
| dulek | Okay, thanks, getting back to reading code. | 14:21 |
| maysamacedos | yboaron you've questioned why the nested case does not have an implementation of ishealth | 14:22 |
| celebdor | irenab: pls merge https://review.openstack.org/#/c/523370/ | 14:22 |
| yboaron | maysamacedos, Hi , Yes | 14:23 |
| celebdor | yboaron: Did you already stack with openshift 3.7.1 to reproduce https://review.openstack.org/#/c/543355/ ? | 14:24 |
| maysamacedos | yboaron: as I'm only checking for presence of ovs-br depending on the vif binding | 14:24 |
| yboaron | OK , maysamacedos - 10x | 14:24 |
| maysamacedos | it will only be overwrite in the baremetal case | 14:25 |
| yboaron | celebdor, Not - yet - I'm working on https://bugs.launchpad.net/kuryr-kubernetes/+bug/1748890 | 14:25 |
| openstack | Launchpad bug 1748890 in kuryr-kubernetes "Neutron loadbalancer is not deleted after endpoint has been deleted already." [High,New] - Assigned to Yossi Boaron (yossi-boaron-1234) | 14:25 |
| yboaron | celebdor, this one cause because we are trying to annotate (store neutron resources) on deleted k8s resources and --> Orphans :-) | 14:26 |
| yboaron | celebdor, I thought to implement rollback for LBaaS handler - although it's common problem for all resources , what do u think ? | 14:27 |
| celebdor | yboaron: did you confirm it? | 14:27 |
| yboaron | celebdor, you mean the bug ? | 14:28 |
| yboaron | celebdor, or the rollback ? | 14:28 |
| celebdor | the bug | 14:29 |
| celebdor | yboaron: cause I see status = New | 14:29 |
| celebdor | if you reproduced | 14:30 |
| celebdor | then I'll change to status = Confirmed | 14:30 |
| yboaron | celebdor, not yes - working on that -restacking | 14:30 |
| celebdor | ok | 14:30 |
| celebdor | yboaron: stacking with 3.7.1 from dulek's patch, right? | 14:31 |
| yboaron | celebdor, yep | 14:31 |
| yboaron | celebdor, what do u say about rollback ? | 14:33 |
| *** dmellado has quit IRC | 14:37 | |
| *** dmellado has joined #openstack-kuryr | 14:42 | |
| celebdor | yboaron: you mean reverting the change to selflink? | 14:42 |
| *** Natanbro has quit IRC | 14:43 | |
| yboaron | celebdor, No , I mean add support for rollback of openstack resources in case of annotation failure -due to resource deleted at K8S | 14:43 |
| yboaron | celebdor, 1. resource crated at K8s | 14:44 |
| yboaron | celebdor, 2. kuryr -handler | 14:44 |
| yboaron | 3. before kuryr handler annotates - resources deleted from K8s | 14:44 |
| yboaron | celebdor, 4. annotation fails - but orphan resources will stay in openstack | 14:45 |
| yboaron | celebdor, 5. since handler delete is based on annotation | 14:45 |
| celebdor | we should find out which we orphan | 14:46 |
| celebdor | and file bugs | 14:46 |
| yboaron | celebdor, not sure I understood your last comment, there is an open bug for that case | 14:47 |
| celebdor | yboaron: for all of the resources? | 14:47 |
| celebdor | which is it? | 14:47 |
| yboaron | celebdor, LBaaS | 14:48 |
| yboaron | celebdor, moving to neutron tagging could also help with that .. | 14:49 |
| *** kiennt26 has quit IRC | 15:00 | |
| celebdor | maybe | 15:00 |
| *** yboaron has quit IRC | 15:01 | |
| dulek | celebdor: yboaron was all the time writing about this one: https://bugs.launchpad.net/kuryr-kubernetes/+bug/1748890 | 15:02 |
| openstack | Launchpad bug 1748890 in kuryr-kubernetes "Neutron loadbalancer is not deleted after endpoint has been deleted already." [High,New] - Assigned to Yossi Boaron (yossi-boaron-1234) | 15:02 |
| dulek | celebdor: Not the selfLink one. | 15:02 |
| celebdor | ah, right | 15:05 |
| celebdor | thanks dulek | 15:05 |
| *** maysamacedos has quit IRC | 15:12 | |
| *** slaweq has quit IRC | 15:20 | |
| *** slaweq has joined #openstack-kuryr | 15:21 | |
| *** slaweq has quit IRC | 15:26 | |
| *** gcheresh has quit IRC | 15:40 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Add experimental containerized jobs https://review.openstack.org/534827 | 15:48 |
| *** yamamoto has quit IRC | 15:55 | |
| *** premsankar has joined #openstack-kuryr | 15:58 | |
| *** celebdor has quit IRC | 16:09 | |
| *** janki has quit IRC | 16:09 | |
| *** gcheresh has joined #openstack-kuryr | 16:09 | |
| *** hongbin has joined #openstack-kuryr | 16:14 | |
| *** pcaruana has quit IRC | 16:16 | |
| *** openstackgerrit has quit IRC | 16:19 | |
| *** maysamacedos has joined #openstack-kuryr | 16:19 | |
| *** celebdor has joined #openstack-kuryr | 16:22 | |
| *** gcheresh has quit IRC | 16:25 | |
| *** celebdor has quit IRC | 16:27 | |
| *** slaweq has joined #openstack-kuryr | 16:33 | |
| *** slaweq has quit IRC | 16:38 | |
| *** slaweq has joined #openstack-kuryr | 16:38 | |
| *** slaweq has quit IRC | 16:44 | |
| *** slaweq has joined #openstack-kuryr | 16:44 | |
| *** slaweq has quit IRC | 16:44 | |
| *** slaweq has joined #openstack-kuryr | 16:45 | |
| *** celebdor has joined #openstack-kuryr | 16:45 | |
| *** openstackgerrit has joined #openstack-kuryr | 16:46 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Add experimental containerized jobs https://review.openstack.org/534827 | 16:46 |
| *** yboaron has joined #openstack-kuryr | 16:47 | |
| *** salv-orl_ has quit IRC | 16:49 | |
| *** slaweq has quit IRC | 16:50 | |
| *** salv-orlando has joined #openstack-kuryr | 16:50 | |
| *** salv-orlando has quit IRC | 16:54 | |
| *** yamamoto has joined #openstack-kuryr | 16:56 | |
| *** yamamoto has quit IRC | 17:02 | |
| *** celebdor has quit IRC | 17:29 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Dummy commit to test tempest change https://review.openstack.org/546708 | 17:30 |
| *** celebdor has joined #openstack-kuryr | 17:44 | |
| celebdor | dulek: how's it with the listener port SG | 17:45 |
| celebdor | ? | 17:45 |
| dulek | celebdor: Rather bad. I think I understand the code, but still struggle with deciding where to put different stuff. | 17:46 |
| dulek | celebdor: E.g. DefaultServiceSecurityGroupsDriver sounds like a great place to create the SG, buuut… I don't know what provider I'll have there. | 17:47 |
| celebdor | dulek: I didn't even think about drivers | 17:48 |
| celebdor | xD | 17:48 |
| *** openstackgerrit has quit IRC | 17:48 | |
| dulek | celebdor: Maybe I'm simply screwing this up, but as VIFDriver creates VIF, I thought SGDriver should create SG. | 17:49 |
| celebdor | dulek: makes sense | 17:54 |
| celebdor | I was just thinking that this would be done by | 17:54 |
| celebdor | brb | 17:54 |
| *** celebdor has quit IRC | 17:54 | |
| *** yamamoto has joined #openstack-kuryr | 17:58 | |
| *** celebdor has joined #openstack-kuryr | 18:03 | |
| celebdor | dulek: sorry | 18:03 |
| celebdor | my btrfs partition is killing my laptop | 18:03 |
| *** yamamoto has quit IRC | 18:04 | |
| dulek | celebdor: I'd need to add SG to LBaaSListener. Is that okay? | 18:10 |
| celebdor | dulek: you mean to the handler? | 18:13 |
| dulek | celebdor: o.vo. | 18:13 |
| celebdor | dulek: what do you need it in the ovo for? | 18:13 |
| dulek | celebdor: To know which SG to add the rule to in LBaaSv2Driver._create_listener | 18:14 |
| celebdor | mmm | 18:16 |
| celebdor | let's do a bluejeans and you show me | 18:16 |
| celebdor | better | 18:16 |
| dulek | Okay, moment. | 18:17 |
| celebdor | dulek: https://bluejeans.com/2742429434 | 18:18 |
| *** gcheresh has joined #openstack-kuryr | 18:21 | |
| yboaron | ping celebdor | 18:29 |
| dulek | yboaron: Yay, I'm not the only one working late. Toni just told me to ping him on signal if I need any help with the bug, so I guess you can do it as well. | 18:38 |
| yboaron | dulek, 10x , question about CNI side , I have non containerized deployment - pod not get connected , I can see failure in controller side - that neutron port is still sown after x retries | 18:40 |
| dulek | yboaron: It's with CNI daemon? | 18:41 |
| yboaron | dulek, good point , I used the deault openshift local.conf , checking it | 18:41 |
| dulek | yboaron: Then it's without. | 18:41 |
| dulek | yboaron: Take a look on the kubelet logs (devstack@kubelet). | 18:42 |
| yboaron | dulek, so no logs from our kuryr-cni side , | 18:42 |
| dulek | Most likely something slipped there. | 18:42 |
| yboaron | dulek, OK , i guess it's related to statefulset - since 'regular' pod creation is fine | 18:42 |
| dulek | Uhm… | 18:43 |
| yboaron | dulek, how it's going with LBaaS - SG ? | 18:43 |
| dulek | yboaron: celebdor helped me a bit with deciding where to put stuff. | 18:43 |
| dulek | This all Handlers-Pipelines-Drivers-Watchers system is awesome, but hard to understand and debug. | 18:44 |
| yboaron | dulek, indeed | 18:44 |
| *** maysamacedos has quit IRC | 18:45 | |
| *** garyloug has quit IRC | 18:50 | |
| celebdor | yboaron: I'm back | 18:53 |
| celebdor | what? | 18:53 |
| yboaron | celebdor, welcome back ! | 18:53 |
| yboaron | celebdor, debugging the staefulset on openshift 3.7.1 - can't create statefull set pods (from controller side all looks fine) - but neutron port is down | 18:55 |
| yboaron | celebdor, 'regular' pod creation is fine | 18:55 |
| yboaron | dulek, helped gave me few ideas for debugging the kubelet/cni side | 18:55 |
| *** maysamacedos has joined #openstack-kuryr | 18:56 | |
| yboaron | celebdor, it's non containerized deployment | 18:56 |
| celebdor | :O | 18:56 |
| celebdor | that is interesting | 18:56 |
| celebdor | :-) | 18:56 |
| celebdor | yboaron: did you check if the veth was created in ovs | 18:57 |
| celebdor | and it got the right external ids? | 18:57 |
| yboaron | celebdor, no | 18:57 |
| yboaron | celebdor, what do u mean by external ids ? | 18:57 |
| yboaron | celebdor, I need to practice more the kubelet/cni side ... | 18:58 |
| *** yamamoto has joined #openstack-kuryr | 19:00 | |
| *** janonymous has quit IRC | 19:01 | |
| celebdor | yboaron: this is ovs agent side | 19:02 |
| celebdor | ovs-brctl | 19:02 |
| celebdor | sorry | 19:02 |
| celebdor | ovs-vsctl | 19:02 |
| yboaron | celebdor, yes it's OVS . | 19:02 |
| *** salv-orlando has joined #openstack-kuryr | 19:03 | |
| celebdor | yboaron: ovs-vsctl --columns=ofport,name list Interface | 19:03 |
| *** yamamoto has quit IRC | 19:06 | |
| yboaron | celebdor, https://pastebin.com/WedQmdS8 | 19:06 |
| yboaron | celebdor, two 'regular' pods and one that belongs to statefulset | 19:07 |
| celebdor | yboaron: don't filter columns | 19:13 |
| celebdor | give me the whole thing | 19:13 |
| celebdor | in a pastebin | 19:13 |
| celebdor | sorry | 19:13 |
| yboaron | celebdor, https://pastebin.com/YNZeKmXv | 19:19 |
| celebdor | yboaron: so cross-match the uuids reported in the annotations | 19:21 |
| celebdor | with what's on this pastebin | 19:21 |
| celebdor | and see if the pod ports are missing the external-ids | 19:21 |
| celebdor | that would prevent them from going active | 19:22 |
| yboaron | celebdor, I think the problem is not the kuryr-cni , but volume processing for the pod | 19:23 |
| yboaron | celebdor, I used the statefulset yaml from : https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ ( see no #link ) | 19:24 |
| yboaron | celebdor, found the following : | 19:25 |
| yboaron | celebdor, 12455 desired_state_of_world_populator.go:272] Error processing volume "www" for pod "web-0_default(b257c649-172a-11e8-8a88-fa163efbd5ce)": error pro | 19:25 |
| yboaron | Feb 21 19:05:48 ffg openshift[12455]: W0221 19:05:48.795314 12455 container_manager_linux.go:747] CPUAccounting not enabled for pid: 12455 | 19:25 |
| yboaron | Feb 21 19:05:48 ffg openshift[12455]: W0221 19:05:48.795333 12455 container_manager_linux.go:750] MemoryAccounting not enabled for pid: 12455 | 19:25 |
| yboaron | Feb 21 19:07:40 ffg openshift[12455]: E0221 19:07:40.032638 12455 kubelet.go:1594] Unable to mount volumes for pod "web-0_default(b257c649-172a-11e8-8a88-fa163efbd5ce)": timeout expired waiting for volumes to | 19:25 |
| yboaron | Feb 21 19:07:40 ffg openshift[12455]: E0221 19:07:40.032690 | 19:25 |
| yboaron | celebdor, at devstack@openshift-node.service | 19:25 |
| yboaron | celebdor, the kubelet/cni debug prints are print to : devstack@openshift-node.service , right ? | 19:26 |
| celebdor | yes | 19:26 |
| celebdor | why did you define volumes for the pod? | 19:27 |
| *** salv-orlando has quit IRC | 19:38 | |
| *** salv-orlando has joined #openstack-kuryr | 19:39 | |
| *** maysamacedos has quit IRC | 19:39 | |
| *** salv-orlando has quit IRC | 19:43 | |
| yboaron | celebdor, because it;s statefull , wen I tried to delete the volume from pod - I got the pod in error state | 19:44 |
| yboaron | celebdor, I didn't find any sample for statefulset without volume | 19:45 |
| yboaron | celebdor, will google it .. | 19:45 |
| celebdor | ok | 19:48 |
| *** juriarte has quit IRC | 19:48 | |
| *** maysamacedos has joined #openstack-kuryr | 19:52 | |
| *** yamamoto has joined #openstack-kuryr | 20:02 | |
| *** yamamoto has quit IRC | 20:07 | |
| *** premsankar has quit IRC | 20:28 | |
| *** hongbin has quit IRC | 20:33 | |
| *** salv-orlando has joined #openstack-kuryr | 20:39 | |
| *** salv-orlando has quit IRC | 20:44 | |
| *** openstackgerrit has joined #openstack-kuryr | 21:00 | |
| openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Services: Set SGs for N-S with haproxy provider https://review.openstack.org/546777 | 21:00 |
| dulek | celebdor: For some reason you're online so here it is. ^ | 21:01 |
| * dulek logs off for today. | 21:01 | |
| *** salv-orlando has joined #openstack-kuryr | 21:03 | |
| *** openstackgerrit has quit IRC | 21:03 | |
| *** yamamoto has joined #openstack-kuryr | 21:04 | |
| *** yamamoto has quit IRC | 21:09 | |
| *** maysamacedos has quit IRC | 21:15 | |
| *** maysamacedos has joined #openstack-kuryr | 21:24 | |
| *** olivierbourdon38 has quit IRC | 21:26 | |
| *** maysamacedos has quit IRC | 21:28 | |
| *** gcheresh has quit IRC | 21:39 | |
| *** celebdor has quit IRC | 21:52 | |
| *** openstackgerrit has joined #openstack-kuryr | 21:56 | |
| openstackgerrit | Yossi Boaron proposed openstack/kuryr-kubernetes master: [WIP] Services: Rollback openstack resources in case of annotation failure https://review.openstack.org/546784 | 21:56 |
| *** dmellado has quit IRC | 22:00 | |
| *** yboaron has quit IRC | 22:01 | |
| *** yamamoto has joined #openstack-kuryr | 22:05 | |
| *** yamamoto has quit IRC | 22:10 | |
| *** celebdor has joined #openstack-kuryr | 22:29 | |
| *** maysamacedos has joined #openstack-kuryr | 22:31 | |
| *** salv-orlando has quit IRC | 22:34 | |
| *** salv-orlando has joined #openstack-kuryr | 22:34 | |
| *** yamamoto has joined #openstack-kuryr | 22:35 | |
| *** celebdor has quit IRC | 22:38 | |
| *** salv-orlando has quit IRC | 22:39 | |
| *** celebdor has joined #openstack-kuryr | 22:50 | |
| *** salv-orlando has joined #openstack-kuryr | 22:53 | |
| *** yamamoto has quit IRC | 23:08 | |
| *** salv-orlando has quit IRC | 23:32 | |
| *** salv-orlando has joined #openstack-kuryr | 23:33 | |
| *** salv-orlando has quit IRC | 23:37 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!