| *** livelace has quit IRC | 00:11 | |
| *** livelace has joined #openstack-kuryr | 00:11 | |
| *** hongbin has joined #openstack-kuryr | 01:04 | |
| openstackgerrit | Merged openstack/kuryr-libnetwork master: Fix the release job failure https://review.openstack.org/591816 | 02:19 |
|---|---|---|
| *** hongbin has quit IRC | 03:57 | |
| *** janki has joined #openstack-kuryr | 04:39 | |
| openstackgerrit | melissaml proposed openstack/kuryr-kubernetes master: Remove the duplicated word https://review.openstack.org/592869 | 06:21 |
| openstackgerrit | Merged openstack/kuryr-libnetwork master: Don't trigger fullstack job on doc change https://review.openstack.org/591887 | 06:38 |
| *** pcaruana has joined #openstack-kuryr | 06:54 | |
| openstackgerrit | melissaml proposed openstack/kuryr-kubernetes master: Remove the duplicated word https://review.openstack.org/592869 | 06:54 |
| *** garyloug has joined #openstack-kuryr | 08:11 | |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Add Rocky cycle release notes https://review.openstack.org/592926 | 08:17 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Make daemon-containerized-octavia gate non-voting https://review.openstack.org/592930 | 08:36 |
| dulek | dmellado: I see OVN failures on 2 patches now. I suspect gate's broken. :( | 09:30 |
| dmellado | Ovn now? Damn! | 09:30 |
| dmellado | dulek same issue on both patches? | 09:31 |
| dulek | dmellado: "ovs-ofctl: br-int is not a bridge or a socket" | 09:32 |
| dulek | dmellado: But I see some tests just succeeded. I'll check ovn's repo… | 09:32 |
| dmellado | Hmm maybe some change on their devstack plugin | 09:32 |
| dmellado | It has happened before... | 09:32 |
| dmellado | dulek, ping dalvarez on the issue | 09:33 |
| dmellado | I'll be calling it a day soon | 09:33 |
| dulek | dmellado: I'm merging rest of patches that you've did +2. | 09:33 |
| dulek | dmellado: Then I'll check my patch fixing pools again and fix the HA issue. | 09:34 |
| dmellado | awesome, I've the release patch ready but I'll be getting the head on Mon | 09:34 |
| dmellado | and use it as a sha | 09:34 |
| dmellado | hash | 09:34 |
| dulek | dmellado: I suspect HA tests will get backported into tempest after my return as I won't be able to finish them today. | 09:35 |
| dmellado | dulek: no worries at all on that | 09:35 |
| dmellado | enjoy your time off ;) | 09:35 |
| dulek | dmellado: Yeah, I don't see that as a very serious issue at the moment. :) | 09:36 |
| *** dalvarez has joined #openstack-kuryr | 09:40 | |
| dmellado | dulek: let me invoke dalvarez | 09:40 |
| dmellado | dalvarez: dalvarez | 09:40 |
| dalvarez | o/ | 09:40 |
| dmellado | he's like beetlejuice | 09:40 |
| dmellado | xD | 09:40 |
| dalvarez | Atleti 4 - R. Madrid 2 | 09:41 |
| dmellado | dalvarez, we're having some issue with the ovn gate | 09:41 |
| dalvarez | ok | 09:41 |
| dulek | dalvarez: "ovs-ofctl: br-int is not a bridge or a socket" | 09:41 |
| dmellado | dulek: could you ping him to the patches? | 09:41 |
| dalvarez | dulek: that doesn't sound good indeed | 09:41 |
| dalvarez | can i get some link to logs? | 09:41 |
| dulek | dalvarez: http://logs.openstack.org/94/585694/4/check/kuryr-kubernetes-tempest-daemon-containerized-ovn/dfd8319/ | 09:41 |
| dalvarez | we don't invoke ovs-ofctl directly from networking-ovn | 09:42 |
| dalvarez | you mean it's devstack? | 09:42 |
| dulek | dalvarez: Yup, upstream gate. | 09:42 |
| dmellado | I was suspecting your devstack plugin, tbh | 09:42 |
| dmellado | dmellado: | 09:42 |
| dmellado | dalvarez: | 09:42 |
| dalvarez | dalvarez: | 09:42 |
| dalvarez | so | 09:42 |
| dalvarez | dulek: can you point me to the exact file with the error in those logs? | 09:42 |
| dalvarez | http://logs.openstack.org/94/585694/4/check/kuryr-kubernetes-tempest-daemon-containerized-ovn/dfd8319/controller/logs/devstacklog.txt.gz#_2018-08-16_19_49_51_048 | 09:43 |
| dalvarez | this? | 09:43 |
| dulek | dalvarez: http://logs.openstack.org/94/585694/4/check/kuryr-kubernetes-tempest-daemon-containerized-ovn/dfd8319/job-output.txt.gz#_2018-08-16_19_49_51_048823 | 09:43 |
| dulek | dalvarez: Same. | 09:43 |
| dalvarez | ok | 09:43 |
| dmellado | dalvarez++ | 09:43 |
| dmellado | thanks! | 09:43 |
| dulek | dalvarez: This happens in… devstack@ovs-vswitchd exactly. | 09:43 |
| dulek | dalvarez: And I'm not sure if this is a transient issue or not. Just saw some hits on our gates. | 09:44 |
| dalvarez | dulek: yeah i was looking into vswitchd logs http://logs.openstack.org/94/585694/4/check/kuryr-kubernetes-tempest-daemon-containerized-ovn/dfd8319/controller/logs/screen-ovs-vswitchd.txt.gz | 09:46 |
| dalvarez | dulek: dmellado no idea with those logs... looks like br-int wasn't created for some reason | 09:47 |
| dalvarez | 2018-08-16 19:41:24.068572 | controller | OVS_BRIDGE=br-int | 09:47 |
| dalvarez | but i can see this | 09:47 |
| dalvarez | dulek: so if you do a recheck it might not happen? | 09:48 |
| dulek | dalvarez: I've noticed at least one successful run, already rechecked a few more. | 09:48 |
| dulek | dalvarez: I don't see anything related in openstack/neutron git log as well. Maybe it's only some infra's hiccup. | 09:49 |
| dulek | dalvarez: If it doesn't ring a bell for you, then let's assume it's fixed and we'll never see it again. ;) | 09:49 |
| dalvarez | dulek: and [root@ovn system]# grep ofctl /etc/systemd/system/devstack\@ovs-vswitchd.service | 09:50 |
| dalvarez | [root@ovn system]# | 09:50 |
| dulek | dalvarez: Don't get it? I don't have access to this system - it's upstream gate VM. | 09:50 |
| dalvarez | dulek: yeah that's from one of my systems | 09:51 |
| dalvarez | dulek: im trying to relate the ovs-ofctl failure with ovs-vswitchd starting, i dont think they're related | 09:51 |
| dalvarez | or well maybe it is | 09:51 |
| dalvarez | dulek: trying to find out who's calling ovs-ofctl | 09:53 |
| dalvarez | dmellado: dulek looks like ovs is crashing | 09:56 |
| dalvarez | ovs-vswitchd crashes... why? not sure, no info in logs. If it's unstable you may want to reproduce locally and report a bug with the coredump and in the meantime, not sure if it makes sense to run against a particular release of OVS as it's running against OVS master branch which has this things | 09:57 |
| dalvarez | http://logs.openstack.org/94/585694/4/check/kuryr-kubernetes-tempest-daemon-containerized-ovn/dfd8319/job-output.txt.gz#_2018-08-16_19_47_52_740561 | 09:58 |
| dulek | dalvarez: I will try locally if it'll fail once again. | 10:01 |
| dmellado | hm it also happened in the patch for another gate | 10:01 |
| dmellado | in fact, in the patch for making another one non-voting | 10:01 |
| dmellado | dulek: dalvarez before I go I'll create a bug for the ovn gate and amend | 10:01 |
| dmellado | my patch so I'll make ovn gate non-voting for now as well | 10:02 |
| dulek | dmellado: Good, it's better to do it like that at release time. | 10:02 |
| dulek | dmellado: BTW fail on kuryr-kubernetes-tempest-octavia is unrelated, some infra issue. | 10:04 |
| dalvarez | dmellado: the "another gate" is also ovn? | 10:04 |
| dmellado | nope | 10:04 |
| dmellado | daemon-containerized-octavia | 10:04 |
| dalvarez | ack | 10:04 |
| dmellado | dulek: ack, let me amend the patch and we'll be on with that | 10:04 |
| dalvarez | yeah seems like some openvswitch issue | 10:04 |
| dalvarez | we're not hitting it (or at least i haven't noticed) so not sure if it's some early stuff you guys do | 10:05 |
| dalvarez | let's keep an eye, still is not OVN specific but OVS.. it's making ovs-vswitchd crash | 10:05 |
| openstackgerrit | Daniel Mellado proposed openstack/kuryr-kubernetes master: Make daemon-cont-octavia and ovn gate non-voting https://review.openstack.org/592930 | 10:05 |
| dmellado | dalvarez: dulek | 10:05 |
| dmellado | ^^ | 10:05 |
| dmellado | I'll be a dictator and self approve this to unblock gates for now | 10:06 |
| dmellado | https://d3i6li5p17fo2k.cloudfront.net/en/rimage/pivot_half_1152/image/1021/0882f5a2488558b1428672fe130c788c | 10:06 |
| dmellado | with this off I go for today folks | 10:07 |
| dmellado | I'll be keeping the irc on the app so ping me if you run into trouble | 10:08 |
| dmellado | good weekend y'all! | 10:08 |
| openstackgerrit | Merged openstack/kuryr-tempest-plugin master: Namespace tests: use requests instead of subprocess https://review.openstack.org/590740 | 10:17 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Add NPWG Multi-VIF driver option in devstack https://review.openstack.org/592291 | 10:34 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Ensure delete_network_pools include all the ports https://review.openstack.org/590739 | 11:15 |
| *** rh-jelabarre has joined #openstack-kuryr | 11:15 | |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: OCP-Router: Add Zuul experimental gates for Openshift Route https://review.openstack.org/565060 | 12:16 |
| openstackgerrit | Merged openstack/kuryr-kubernetes master: Make daemon-cont-octavia and ovn gate non-voting https://review.openstack.org/592930 | 12:18 |
| openstackgerrit | Peng Liu proposed openstack/kuryr-tempest-plugin master: NPWG multi-vif driver tempest tests https://review.openstack.org/590643 | 12:21 |
| *** janki has quit IRC | 13:08 | |
| *** janki has joined #openstack-kuryr | 13:12 | |
| openstackgerrit | Alexey Perevalov proposed openstack/kuryr-kubernetes master: Introduce VIFRegistry https://review.openstack.org/591352 | 13:25 |
| openstackgerrit | Alexey Perevalov proposed openstack/kuryr-kubernetes master: Add SR-IOV pod vif driver https://review.openstack.org/512280 | 13:25 |
| openstackgerrit | Alexey Perevalov proposed openstack/kuryr-kubernetes master: Add SR-IOV binding driver to CNI https://review.openstack.org/512281 | 13:25 |
| openstackgerrit | Peng Liu proposed openstack/kuryr-kubernetes master: Add check for NPWG multi-vif dirver https://review.openstack.org/593062 | 14:23 |
| *** garyloug has quit IRC | 14:45 | |
| *** janki has quit IRC | 14:46 | |
| *** garyloug has joined #openstack-kuryr | 14:51 | |
| openstackgerrit | Alexey Perevalov proposed openstack/kuryr-kubernetes master: Add SR-IOV pod vif driver https://review.openstack.org/512280 | 16:08 |
| openstackgerrit | Alexey Perevalov proposed openstack/kuryr-kubernetes master: Add SR-IOV binding driver to CNI https://review.openstack.org/512281 | 16:08 |
| *** aperevalov has quit IRC | 16:09 | |
| *** garyloug has quit IRC | 16:34 | |
| openstackgerrit | Merged openstack/kuryr-tempest-plugin master: Create service with UDP protocol https://review.openstack.org/585694 | 19:41 |
| openstackgerrit | OpenStack Release Bot proposed openstack/kuryr-libnetwork master: Update reno for stable/rocky https://review.openstack.org/593158 | 20:06 |
| *** pliu has quit IRC | 20:14 | |
| *** rh-jelabarre has quit IRC | 21:15 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!