Wednesday, 2020-01-29

*** armax has quit IRC00:00
*** dtruong has quit IRC00:12
*** dtruong has joined #openstack-lbaas00:12
*** threestrands has joined #openstack-lbaas00:50
*** armax has joined #openstack-lbaas01:04
*** yamamoto has joined #openstack-lbaas01:25
*** sapd1_x has joined #openstack-lbaas01:34
*** spatel has joined #openstack-lbaas01:35
*** mithilarun has quit IRC01:46
*** yamamoto has quit IRC02:19
*** spatel has quit IRC03:06
*** yamamoto has joined #openstack-lbaas03:08
*** yamamoto has quit IRC03:15
*** goldyfruit_ has quit IRC04:04
*** sapd1_x has quit IRC06:17
*** gcheresh_ has joined #openstack-lbaas06:26
*** sapd1_x has joined #openstack-lbaas06:30
*** ccamposr has joined #openstack-lbaas07:12
*** luksky has joined #openstack-lbaas07:20
*** sapd1_x has quit IRC07:32
openstackgerritAnn Taraday proposed openstack/octavia master: Testing  https://review.opendev.org/69721307:40
*** maciejjozefczyk has joined #openstack-lbaas07:54
*** tesseract has joined #openstack-lbaas08:03
*** tkajinam has quit IRC08:08
*** rpittau|afk is now known as rpittau08:08
*** rcernin has quit IRC08:20
*** luksky has quit IRC08:28
*** luksky has joined #openstack-lbaas08:31
*** luksky has quit IRC09:26
*** threestrands has quit IRC09:47
openstackgerritAnn Taraday proposed openstack/octavia master: Testing  https://review.opendev.org/69721310:09
*** luksky has joined #openstack-lbaas10:29
*** psachin has joined #openstack-lbaas10:36
*** pcaruana has quit IRC10:57
*** psachin has quit IRC11:18
openstackgerritAnn Taraday proposed openstack/octavia master: Testing  https://review.opendev.org/69721311:21
*** pcaruana has joined #openstack-lbaas11:40
*** AlexStaf has joined #openstack-lbaas11:46
*** rpittau is now known as rpittau|bbl11:53
openstackgerritAnn Taraday proposed openstack/octavia-tempest-plugin master: TEST  https://review.opendev.org/70478311:56
*** ramishra has quit IRC12:08
*** ramishra has joined #openstack-lbaas12:13
*** salmankhan has joined #openstack-lbaas12:23
*** ramishra has quit IRC12:50
*** goldyfruit_ has joined #openstack-lbaas13:01
*** maciejjozefczyk has quit IRC13:14
*** rpittau|bbl is now known as rpittau13:30
*** ramishra_ has joined #openstack-lbaas13:36
*** maciejjozefczyk has joined #openstack-lbaas13:39
*** strigazi has quit IRC13:56
*** strigazi has joined #openstack-lbaas14:02
*** KeithMnemonic has joined #openstack-lbaas14:20
*** TrevorV has joined #openstack-lbaas14:21
*** spatel has joined #openstack-lbaas14:36
*** spatel has quit IRC14:40
*** ccamposr has quit IRC14:48
*** ccamposr has joined #openstack-lbaas14:48
*** spatel has joined #openstack-lbaas14:48
*** AlexStaf has quit IRC14:51
*** spatel has quit IRC14:54
*** Trevor_V has joined #openstack-lbaas14:55
*** Trevor__V has joined #openstack-lbaas14:57
*** TrevorV has quit IRC14:58
*** Trevor_V has quit IRC15:00
*** TrevorV has joined #openstack-lbaas15:12
*** Trevor__V has quit IRC15:15
*** ataraday_ has joined #openstack-lbaas15:18
*** Trevor_V has joined #openstack-lbaas15:38
*** TrevorV has quit IRC15:41
*** gcheresh_ has quit IRC15:42
rm_work#startmeeting Octavia16:00
openstackMeeting started Wed Jan 29 16:00:02 2020 UTC and is due to finish in 60 minutes.  The chair is rm_work. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: Octavia)"16:00
openstackThe meeting name has been set to 'octavia'16:00
rm_work#chair johnsom16:00
openstackCurrent chairs: johnsom rm_work16:00
rm_work#chair cgoncalves16:00
openstackCurrent chairs: cgoncalves johnsom rm_work16:00
cgoncalveshi16:00
johnsomo/16:00
rm_worko/16:00
haleybhi16:00
ataraday_hi16:00
rm_worklooks like just a few of us today16:02
rm_workor maybe this is the norm? anywho16:02
rm_work#topic Announcements16:02
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"16:02
johnsomThe NDSU students working on TLS ciphers and protocols start today!16:02
rm_workI was about to say that :D16:02
johnsomWe are doing an introduction meeting for them later today.16:02
cgoncalvesawesome! if you are reading this, welcome!!16:03
johnsom#link https://www.openstack.org/foundation/2019-openstack-foundation-annual-report16:03
johnsomWe are also mentioned in the annual report for this effort.16:03
*** ccamposr has quit IRC16:03
cgoncalvesspotlight on the project!16:04
rm_workSo that's all I have... Anyone else have anything to announce?16:04
johnsomNothing else from me16:07
rm_workgoing once... going once... going twice... going three times.... going five times...16:07
rm_work#topic Brief progress reports / bugs needing review16:07
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"16:08
haleyba talkative crowd today16:09
rm_workapparently16:09
cgoncalvesI spent some time reviewing stuff, particularly jobboard patches. many merged16:10
rm_workSo, I've resurrected the patch that allows UDP pool HMs to use other protocols16:10
johnsomSo, failover flow....  It is working in the lab, I am at the "clean it up" and testing phase.16:10
rm_work#link https://review.opendev.org/#/c/589180/16:10
johnsomRecently I have been looking at the SINGLE topology testing.16:10
ataraday_cgoncalves, rm_work thanks a lot for reviews! Just one change letf :)16:11
rm_workWe ran into issues with the UDP healthcheck in our environment (it's ... not a great design, but I guess it's the best we can do generically) so we need to be able to use other types on a UDP LB16:11
johnsomFor SINGLE LBs, I have dropped the outage time down to a second or two for manual failovers. This is a huge improvement.16:11
rm_workataraday_: so close! :D16:11
*** maciejjozefczyk has quit IRC16:11
rm_workjohnsom: o/ does it create an amp in parallel before the delete of the old one?16:12
johnsomRight now I am working to debug an IPv6 DAD issue triggered by my new code to speed up the SINGLE topology failover.16:12
rm_workwhich is what we were avoiding previously due to "possible resource constraints" but i kinda thought was a BS reason16:12
ataraday_rm_work, yeah, just the main change :D16:12
johnsomrm_work, it does build prior to failover, so yes, if there is a quota/capacity constraint it will now fail. This is what also raised this DAD issue.16:13
johnsomDuplicate Address Detection (DAD)16:13
rm_workI just figured your kids came back home and kept interrupting you :D16:13
cgoncalvesataraday_, great work on your patches! you asked a question today on Gerrit if amphorav2 should be default in Ussuri. we could discuss it here today if you'd like16:14
rm_workI have a couple of patches that I worked on that are good to go I think, could just use more reviews and a push :D16:14
rm_work#link https://review.opendev.org/#/c/699521/16:14
rm_work^^ to add more functionality to AZs16:15
rm_work#link https://review.opendev.org/#/c/702535/16:15
johnsomThe rebase is going to be a nightmare I think....16:15
haleybjohnsom: ping me if you need help with DAD, if it's failing is there a loop?16:15
rm_work^^ allow configuring whether you want to force one-armed16:15
johnsomI have also done some significant refactoring around the amphorae driver and backend to clean up some "issues".16:15
johnsomhaleyb I have fixed these issues before. It's a sequencing issue with the new accelerated failover.  I just found it in testing last night, so will look at it and fix it today.16:16
haleyback16:16
johnsomLast time I tested, SINGLE completely rebuilds the amphora in around 30 seconds, Act/Stdby in around 70 seconds. Outage time is a second or less for both.  Switching to VRRP version 3 will drop it even more. I have a followup patch for that started.16:17
cgoncalvesyou're on fire!16:18
johnsomStill fully backport-able. No image roll needed, but would help bring down the SINGLE outage time.16:19
johnsomAnyway, that and reviews have been my focus over the last week.16:20
haleybi'd like to ask for some of my py2 removal patches to get some reviews, we're seeing other repos randomly get bitten as third party library support goes away, would be good to get ahead of it16:21
haleybexcept for the six removal they're all pretty small16:21
haleybjohnsom: should i put some on your list?16:22
johnsomhaleyb I think I have already been bugging you about some of those... grin16:22
johnsomhaleyb But, yes, please make sure they are on the priority list.16:22
ataraday_cgoncalves, It can be discussed on gerrit :) I put question to highlight this point. Should be 'amphorav2' amoung enabled_provider_drivers by default or not16:23
haleybjohnsom: yes, and i think i've re-spun most, i'll verify and add to list16:23
johnsom#link https://etherpad.openstack.org/p/octavia-priority-reviews16:23
johnsomJust in case someone doesn't have it16:23
rm_workalso this one I just rebased:16:25
openstackgerritAdam Harwell proposed openstack/octavia master: Update the lb_id on an amp earlier if we know it  https://review.opendev.org/69808216:25
rm_workwhich was a combination of what ataraday_ and I did independently (though she did it first and I was just blind, lol)16:26
*** numans has quit IRC16:28
rm_workok so I guess it's time for:16:30
rm_work#topic Open Discussion16:30
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"16:30
cgoncalvesI'd like to get input from the team on enabling KVM instead of QEMU, when possible, in the CI jobs16:32
rm_workwe've bounced around on this a lot16:32
johnsomI am planning to finish up the basic cleanup stuffs and dev testing, then I will probably post failover with broken tests for v1 only. Followup will be with fixed tests.16:32
rm_workwe do it, and then it works for a bit, and then jobs start breaking, and then we have to disable it16:32
rm_workwe can try again, but just be aware16:32
rm_workthis'll be like the third time16:32
cgoncalvescontext is that there are some nodepool providers that provide nested virtualization but we are not leveraging because of bugs in the ubuntu kernel16:32
johnsomI am good with turning it on again if we seem to have passing tests across the nodepool providers.16:33
johnsomWe can hope that the kernel bug is now fixed and deployed across the nodepool fleet16:33
cgoncalvesalthough, I think root caused it to one particular provider (vexxhost) having not exact/best-matching CPU models than the actual physical CPU16:34
johnsomWe ran for a year and a half with it on without any issues, so I'm not worried about it in *general*16:34
cgoncalves#link https://review.opendev.org/#/c/702921/16:34
johnsomLast root cause I found in partnership with OVH was a kernel KVM bug with certain guest and host kernel versions.16:34
cgoncalvesnote there's a depends-on for a devstack patch16:34
cgoncalvesin testing, seems to work fine at OVH16:35
cgoncalvesthe problematic one was vexxhost because of the CPU model16:35
johnsomYeah, it's been a long time since we tried it again to see if there is a fix out.16:35
cgoncalvessetting cpu model to host-passthrough in libvirt helped16:35
cgoncalvesthere's more information on the commit message that may better explain the context and proposal16:37
cgoncalvesmaybe I should give folks some time to digest it. we can talk about it again next week or in Gerrit16:38
rm_workits prolly fine to try again16:38
cgoncalvesworks for me. I'll make sure the devstack patch merges16:39
*** luksky has quit IRC16:42
rm_workanything else or should we call it for today?16:43
rm_workok, calling it, thanks folks16:50
rm_work#endmeeting16:50
*** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews"16:50
openstackMeeting ended Wed Jan 29 16:50:26 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-01-29-16.00.html16:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-01-29-16.00.txt16:50
openstackLog:            http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-01-29-16.00.log.html16:50
johnsomo/16:50
*** mithilarun has joined #openstack-lbaas17:08
*** mithilarun has quit IRC17:09
*** ataraday_ has quit IRC17:28
*** rpittau is now known as rpittau|afk17:33
*** salmankhan has quit IRC17:51
*** mithilarun has joined #openstack-lbaas18:00
*** luksky has joined #openstack-lbaas18:09
*** mithilarun has quit IRC18:10
*** mithilarun has joined #openstack-lbaas18:10
*** ramishra_ has quit IRC18:37
*** gcheresh_ has joined #openstack-lbaas18:48
*** tesseract has quit IRC18:59
*** spatel has joined #openstack-lbaas19:02
*** mithilarun has quit IRC19:20
*** mithilarun has joined #openstack-lbaas19:24
*** spatel has quit IRC19:30
*** gcheresh_ has quit IRC19:35
openstackgerritMerged openstack/octavia stable/queens: Add listener and pool protocol validation  https://review.opendev.org/70210219:53
*** gmann is now known as gmann_afk19:57
*** dawzon has joined #openstack-lbaas19:58
cgoncalves^ only took 12 rechecks20:15
rm_work<_<20:20
rm_workdoes that mean every patch we want to merge into queens from now on should also be expected to take 12 rechecks? <_<20:21
cgoncalvesno. could be also 11 or 1320:22
rm_workah you're right20:22
rm_worknever mind then20:22
cgoncalvessorry, bad joke xD20:22
rm_worki thought it was good :D20:23
johnsomI have no idea why that ifup issue is occurring on that queens gate. I think I have tried my ideas that seemed to work local.20:26
*** dawzon has quit IRC20:53
*** gmann_afk is now known as gmann21:17
*** rcernin has joined #openstack-lbaas21:32
*** Trevor_V has quit IRC22:31
*** tkajinam has joined #openstack-lbaas23:07
*** spatel has joined #openstack-lbaas23:14
*** spatel has quit IRC23:19

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!