Wednesday, 2019-09-18

*** goldyfruit_ has joined #openstack-lbaas00:06
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510300:16
*** sapd1_x has joined #openstack-lbaas00:19
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510300:20
openstackgerritMerged openstack/octavia stable/rocky: Fix auto setup Barbican's ACL in the legacy driver.  https://review.opendev.org/66568100:37
*** ccamposr__ has quit IRC01:12
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510301:42
*** sapd1_x has quit IRC01:50
*** yamamoto has joined #openstack-lbaas01:56
*** yamamoto has quit IRC02:21
*** yamamoto has joined #openstack-lbaas02:21
*** ccamposr has joined #openstack-lbaas03:02
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510303:03
*** ccamposr__ has joined #openstack-lbaas03:34
*** ccamposr has quit IRC03:34
*** sapd1_x has joined #openstack-lbaas03:45
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510303:52
*** ricolin has joined #openstack-lbaas03:54
openstackgerritMerged openstack/octavia stable/stein: Fix l7rule API handling of None updates  https://review.opendev.org/67855004:01
*** ramishra has joined #openstack-lbaas04:01
*** ccamposr has joined #openstack-lbaas04:15
*** sapd1_x has quit IRC04:16
*** ccamposr__ has quit IRC04:16
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510304:18
openstackgerritMichael Johnson proposed openstack/octavia-tempest-plugin master: [train][goal] Define new 'octavia-v2-dsvm-noop-api-ipv6-only' job  https://review.opendev.org/68272604:19
openstackgerritMerged openstack/octavia stable/rocky: Fix l7rule API handling of None updates  https://review.opendev.org/67855104:31
*** pcaruana has joined #openstack-lbaas04:46
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510305:07
rm_workcgoncalves: is this a real test fail? https://review.opendev.org/#/c/678557/05:35
rm_workbad rebase maybe?05:36
*** ramishra has quit IRC05:39
openstackgerritAdam Harwell proposed openstack/octavia master: Fix 'additive_only' parameter api-ref  https://review.opendev.org/68162505:50
rm_work^^ just a rebase to fix a weird zuul state05:50
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510305:52
*** gthiemonge has quit IRC06:01
*** gthiemon1e has joined #openstack-lbaas06:01
*** mjozefcz has joined #openstack-lbaas06:01
rm_workjohnsom: wat, you're still working too? :D06:02
rm_workI just +A'd ... 13 patches >_>06:02
rm_workjohnsom: wat are you doing with all those FIXED_IP# vars? trying to debug something? :D06:05
rm_worklol: `Connected to download.fedoraproject.org (2605:bc80:3010:600:dead:beef:cafe:feda) port 443`06:07
rm_worknice IP06:07
rm_workhmm our fedora28 publish is failing because `GET /pub/fedora/linux/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2` is a 40406:08
rm_worklol yeah06:09
rm_workhttps://download-ib01.fedoraproject.org/pub/fedora/linux/releases/28/README06:09
rm_workgonna fix it06:10
rm_worki assume that's a DIB issue? actually maybe someone already has a patch06:13
rm_workor possibly we should move up to fedora 29 or 3006:14
openstackgerritAdam Harwell proposed openstack/octavia master: Bump fedora amp version to fedora30  https://review.opendev.org/68280006:17
rm_workneed to decide the best way to test that06:17
*** ccamposr has quit IRC06:18
*** ccamposr has joined #openstack-lbaas06:19
cgoncalvesrm_work, https://review.opendev.org/#/c/673173/06:21
rm_workah cool k06:22
rm_workdidn't notice you did the switch too06:22
*** gthiemon1e has quit IRC06:22
rm_workwas just looking at adding a non-voting dsvm but you got that too :D06:23
*** gthiemonge has joined #openstack-lbaas06:23
rm_workthough I wish there was a dash "fedora-30" for my OCD06:23
rm_workbecause centos7 never has one either but we use it in our format anyway because it's "$DISTRO-$VERSION"06:23
cgoncalvesI'd back that change06:25
rm_workbut i also don't understand where "octavia-v2-dsvm-py2-scenario-centos-7" comes from06:25
rm_worki mean, I would like to see that added as a nonvoting check06:25
cgoncalvesI'd also back that :)06:26
rm_workahh06:26
rm_worki read that as "back-out"06:26
cgoncalvessorry. bad Engrish06:26
rm_worknah reading comprehension fail06:26
cgoncalvesI'd support that change06:26
rm_workahh it's in tempest-plugin06:26
rm_workso we need to make a patch there to add it06:27
cgoncalvesyes06:27
rm_workk06:27
rm_workwant to do that, depends-on it, and actually make it official?06:27
rm_workor are you still testing06:27
rm_worktimeout is :(06:27
cgoncalvesadding a non-voting fedora job may be reasonable. it seeems to be of interest to Jack, Kong and now also you06:27
cgoncalvesI was interested about it back a few months when we were preparing RHEL806:28
rm_worki mean, if we're publishing an image, we need the test06:30
rm_worki don't care about fedora so much06:30
rm_worki just care about consistency06:30
johnsomYeah, trying to figure out why something that works local doesn’t work in the job....06:33
johnsomClassic works for me situation06:33
johnsomSeems the IPv6 environment variable is the issue.06:34
*** ccamposr has quit IRC06:34
*** ccamposr has joined #openstack-lbaas06:34
johnsomLovely devstack plugin scoping is my guess.06:35
johnsomFyi, fedora cycles often, so if we add a job we are signing up to update it often06:36
openstackgerritCarlos Goncalves proposed openstack/octavia master: WIP: Switch Fedora-based amphora to fedora-minimal  https://review.opendev.org/67317306:36
rm_work:/06:36
openstackgerritCarlos Goncalves proposed openstack/octavia master: WIP: Switch Fedora-based amphora to fedora-minimal  https://review.opendev.org/67317306:36
cgoncalvesgood point06:37
cgoncalveswe're making good progress merging backport patches. thank you, all!06:38
rm_workso should we remove the publish job we have already?06:38
rm_work:/06:38
johnsomI think fedora is questionable.  I would like to see someone willing to sign up to maintain it and had a need. By design it is experimental.06:39
rm_workUnable to connect to mirror.gra1.ovh.openstack.org:http:06:40
rm_workhmmm06:40
rm_workhave we seen issues with their mirrors before?06:40
rm_workjohnsom: yeah my point is that we publish a fedora amp image *now*. not sure how it got there -- but we either need to add a test for it or remove that publish job, IMO06:41
johnsomIf it is DNS related, infra is tracking an issue06:41
rm_workk06:41
cgoncalvesjohnsom, agreed. I thought there would be enough interest now in fedora. Jack, Kong and apparently also Adam but turns out that is not exactly the case06:41
johnsomAgreed, I would pull the publish myself06:41
rm_workanyway, doing *nothing* is not a great option06:41
rm_workmy interest is in us not publishing untested images :D06:41
rm_workwhich has the obvious two options i stated above, lol06:42
cgoncalvesgood PTLing06:42
johnsomrm_work if it was a dns failure, I would flag the job for clark06:42
rm_workjohnsom: it's one that's still in gate06:42
rm_workbut will fail eventually06:42
rm_work67854906:42
rm_workwhere do you see them tracking an issue?06:43
johnsomIRC. I raised it with them06:43
johnsomWe thought it was fixed with a bad router at an provider, but ...06:44
johnsomWait, we have multi-node voting???06:44
johnsomI have seen neutron pain on multi-node06:44
rm_workapparently?06:45
rm_workbut this is unrelated to neutron? unless you mean that neutron kills all outbound networking or something, which is possible i guess lol06:45
johnsomOh, v1 multinode06:45
rm_workahhh yeah this is a stable job06:46
rm_workso v1 and multinode]06:46
johnsomYeah, multi-node worked before zuul v306:46
rm_work<_<06:46
*** gcheresh has joined #openstack-lbaas06:47
*** ccamposr__ has joined #openstack-lbaas06:47
johnsomCan't help you with that one...06:49
*** ccamposr has quit IRC06:50
johnsomYeah, joy, ok, so ENVIRON in awk doesn't work in the job, but works local.  Joy....06:50
johnsomSomething for tomorrow....06:51
johnsomProbably a side effect of this "oscwrap" thing....06:52
rm_workweird06:52
johnsomCatch you all tomorrow.06:53
rm_workkk later06:53
*** rcernin has quit IRC06:57
rm_workoctavia is really ripping up the nodepool right now, lol07:05
rm_workguess i'll crash and let some finish and check again tomorrow07:06
rm_workcgoncalves: i guess if you see rechecks that need to be done later, that'd be good :D07:06
rm_worki know there's at least a couple07:06
*** trident has quit IRC07:08
*** luksky has joined #openstack-lbaas07:13
*** trident has joined #openstack-lbaas07:19
openstackgerritMerged openstack/octavia stable/rocky: Fix template that generates vrrp check script  https://review.opendev.org/68228207:21
openstackgerritMerged openstack/octavia stable/queens: Fix L7 repository create methods  https://review.opendev.org/67856507:21
*** ivve has joined #openstack-lbaas07:32
*** rpittau|afk is now known as rpittau07:32
openstackgerritAndreas Jaeger proposed openstack/octavia stable/rocky: Do not run non-voting jobs in gate  https://review.opendev.org/68281207:32
openstackgerritMerged openstack/octavia stable/stein: elements: add arch property for ``open-vm-tools``  https://review.opendev.org/67853708:00
*** mjblack has quit IRC08:00
*** mjblack has joined #openstack-lbaas08:03
*** tkajinam has quit IRC08:04
openstackgerritMerged openstack/octavia stable/rocky: elements: add arch property for ``open-vm-tools``  https://review.opendev.org/67853808:06
openstackgerritMerged openstack/octavia master: Fix base (VRRP) port abandoned on revert  https://review.opendev.org/67915308:15
openstackgerritAnn Taraday proposed openstack/octavia stable/stein: Fix base (VRRP) port abandoned on revert  https://review.opendev.org/68283308:23
openstackgerritAnn Taraday proposed openstack/octavia stable/rocky: Fix base (VRRP) port abandoned on revert  https://review.opendev.org/68283608:45
openstackgerritMerged openstack/octavia stable/stein: Fix catching driver exceptions  https://review.opendev.org/68177908:45
*** KeithMnemonic1 has joined #openstack-lbaas08:55
*** KeithMnemonic has quit IRC08:55
*** KeithMnemonic1 has quit IRC08:55
*** KeithMnemonic1 has joined #openstack-lbaas08:56
*** pcaruana has quit IRC08:57
*** ramishra has joined #openstack-lbaas08:59
openstackgerritMerged openstack/octavia master: Work around strptime threading issue  https://review.opendev.org/67571708:59
openstackgerritMerged openstack/octavia master: Use dual intermediate CAs for devstack  https://review.opendev.org/67892308:59
openstackgerritMerged openstack/octavia master: Fix 'additive_only' parameter api-ref  https://review.opendev.org/68162508:59
*** pcaruana has joined #openstack-lbaas09:01
*** ramishra_ has joined #openstack-lbaas09:25
*** ramishra has quit IRC09:26
*** ramishra has joined #openstack-lbaas09:26
*** yamamoto has quit IRC09:27
*** ramishra_ has quit IRC09:30
*** tesseract has joined #openstack-lbaas09:44
*** ajay33 has joined #openstack-lbaas09:52
*** yamamoto has joined #openstack-lbaas09:58
*** sapd1_x has joined #openstack-lbaas10:00
*** openstackgerrit has quit IRC10:06
*** yamamoto has quit IRC10:07
*** sapd1_x has quit IRC10:18
*** ramishra has quit IRC10:20
*** salmankhan has joined #openstack-lbaas10:26
*** yamamoto has joined #openstack-lbaas10:42
*** salmankhan1 has joined #openstack-lbaas10:54
*** salmankhan has quit IRC10:56
*** salmankhan1 is now known as salmankhan10:56
mjozefcz\o cgoncalves rm_work, are you folks okey with this patch? https://review.opendev.org/#/c/672477/ :)11:05
*** yamamoto has quit IRC11:12
*** pcaruana has quit IRC11:19
*** psachin has joined #openstack-lbaas11:26
*** pcaruana has joined #openstack-lbaas11:28
*** boden has joined #openstack-lbaas11:53
*** yamamoto has joined #openstack-lbaas11:59
*** yamamoto has quit IRC12:02
*** yamamoto has joined #openstack-lbaas12:02
*** goldyfruit_ has quit IRC12:15
*** openstackgerrit has joined #openstack-lbaas12:20
openstackgerritAnn Taraday proposed openstack/octavia master: Convert listener flows to use provider models  https://review.opendev.org/66023612:21
openstackgerritAnn Taraday proposed openstack/octavia master: Convert pool flows to use dicts  https://review.opendev.org/66538112:21
openstackgerritAnn Taraday proposed openstack/octavia master: Transition member flows to use dicts  https://review.opendev.org/65784212:21
openstackgerritAnn Taraday proposed openstack/octavia master: Transition amphora flows to dicts  https://review.opendev.org/66889812:21
*** redrobot has joined #openstack-lbaas12:26
*** luksky has quit IRC12:27
openstackgerritAnn Taraday proposed openstack/octavia stable/rocky: Fix base (VRRP) port abandoned on revert  https://review.opendev.org/68283612:32
openstackgerritAnn Taraday proposed openstack/octavia master: Convert health_monitor flows to use provider models  https://review.opendev.org/67977912:34
*** yamamoto has quit IRC12:53
openstackgerritAnn Taraday proposed openstack/octavia master: Convert Lb flows to use provider dicts  https://review.opendev.org/67172512:54
openstackgerritAnn Taraday proposed openstack/octavia master: [WIP] Jobboard based controller  https://review.opendev.org/64740612:56
*** yamamoto has joined #openstack-lbaas12:57
*** ramishra has joined #openstack-lbaas13:01
*** ricolin_ has joined #openstack-lbaas13:08
*** ricolin has quit IRC13:11
*** ricolin_ is now known as ricolin13:12
*** luksky has joined #openstack-lbaas13:19
*** Vorrtex has joined #openstack-lbaas13:24
*** goldyfruit_ has joined #openstack-lbaas13:30
*** psachin has quit IRC13:33
openstackgerritMerged openstack/octavia stable/queens: Fix template that generates vrrp check script  https://review.opendev.org/68228313:37
openstackgerritMerged openstack/octavia stable/rocky: Do not run non-voting jobs in gate  https://review.opendev.org/68281213:44
openstackgerritCarlos Goncalves proposed openstack/octavia stable/stein: Enabling SG customization on loadbalancer listerners  https://review.opendev.org/64806913:48
*** yamamoto has quit IRC13:49
*** ajay33 has quit IRC13:51
*** Vorrtex has quit IRC14:20
*** openstackgerrit has quit IRC14:21
*** yamamoto has joined #openstack-lbaas14:28
*** ccamposr__ has quit IRC14:32
*** openstackgerrit has joined #openstack-lbaas14:34
openstackgerritAnn Taraday proposed openstack/octavia master: Use retry for AmphoraComputeConnectivityWait  https://review.opendev.org/66279114:34
*** yamamoto has quit IRC14:37
*** yamamoto has joined #openstack-lbaas14:37
*** yamamoto has quit IRC14:40
*** yamamoto has joined #openstack-lbaas14:40
*** luksky has quit IRC14:48
*** haleyb has joined #openstack-lbaas15:10
openstackgerritCarlos Goncalves proposed openstack/octavia stable/stein: Fix cleanup of expired load balancer entries  https://review.opendev.org/68293015:13
*** ivve has quit IRC15:18
*** goldyfruit_ has quit IRC15:20
*** gcheresh has quit IRC15:28
*** trident has quit IRC15:41
openstackgerritCarlos Goncalves proposed openstack/octavia master: Enable full MAC security by default  https://review.opendev.org/68293215:45
cgoncalveslets see how it goes15:45
*** mjozefcz has quit IRC15:48
*** trident has joined #openstack-lbaas15:53
*** ramishra has quit IRC15:55
*** ataraday_ has joined #openstack-lbaas15:56
johnsomo/16:01
johnsomrm_work Meeting time....16:01
*** ajay33 has joined #openstack-lbaas16:02
johnsom#startmeeting Octavia16:02
*** henriqueof has quit IRC16:02
openstackMeeting started Wed Sep 18 16:02:55 2019 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.16:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:02
*** openstack changes topic to " (Meeting topic: Octavia)"16:02
openstackThe meeting name has been set to 'octavia'16:03
cgoncalveshi16:03
ataraday_hi16:03
ajay33Hi16:03
johnsomI guess our PTL is MIA today. lol16:03
*** rpittau is now known as rpittau|afk16:03
gthiemongehi16:03
johnsom#topic Announcements16:03
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"16:03
johnsomWe are in feature freeze. Please do not merge any feature patches.16:04
johnsomThank you to everyone that helped us get those last few features merged after all of the infra and requirements issues we faced.16:04
johnsomIf you didn't follow the issues, a few weeks before feature freeze Ubuntu released a kernel that paniced if you had a NAT rule (floating IP) and sent a fragmented packet through it. This caused our tempest jobs to fail with "RETRY_LIMIT".16:05
johnsomThis was fixed in the -62 kernel release.16:05
*** yamamoto has quit IRC16:06
johnsomThen requirements had bumped configparser to 4.x.x, but the author pulled the package from pypi after the global-requirements update went out.16:06
*** ccamposr has joined #openstack-lbaas16:06
johnsomThis also broke our jobs.16:06
cgoncalvesthank YOU for the hours put troubleshooting it, and syncing with infra and other teams16:06
johnsomJust glad we got it fixed in time for feature freeze16:07
johnsomRC1 and stable/train branches are next week.16:07
johnsomNext week is RC1 week. We should try to have all of our bug fixes in for that.16:07
johnsomI will not be available to do the release. rm_work or cgoncalves will need to take point on that.16:08
johnsomI can create a stub release patch if you would like.16:08
cgoncalvesany help is welcome :)16:08
johnsomFinally, as I hinted at above. I will not be available for the next two weeks.16:09
cgoncalvesI will coordinate with our PTL during your absence16:09
johnsomOk, I will put up a stub patch (no commit strings) and mark it WIP. Then send you two the link.16:10
johnsomAny other announcements this week?16:10
johnsom#topic Brief progress reports / bugs needing review16:10
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"16:10
johnsomI have posted patches for the PDF docs Train goal. All of them are in pretty good shape except for the Octavia docs.16:11
johnsomThree sections are "missing" from the PDF version. I'm not sure why. I have added it to the "PDF Docs" trouble sheet hoping for some help.16:12
johnsomThe latex seems to have the content, just the latex->pdf process drops them.16:12
gthiemongeFYI I added https://review.opendev.org/#/c/682365/ to the review list16:12
johnsomI think the patches are good enough to merge and then we can bug fix from there.16:12
ataraday_I would like to ask review for16:12
ataraday_#link https://review.opendev.org/#/c/681195/16:12
johnsomI have also picked up the IPv6 goal work. It is almost done, just some devstack plugin strangeness to figure out. Hope to have that done today.16:13
ataraday_And hit this bug16:13
ataraday_#link https://storyboard.openstack.org/#!/story/200656016:13
ataraday_but didn't come with anything on it yet16:13
johnsomTo finish out my week I may try to improve an error message when the user gives us a pass phrase protected pkcs12 bundle.16:15
johnsomWe have seen a few folks get confused by that as barbican does not accept a pass phrase for pkcs12 "secrets".16:16
redrobotđź‘€16:16
colin-have been continuing to test haproxy2.0 with octavia and rm_work's single-process patch. looking good so far and haven't encountered any major issues now that my amp image has the components i wanted (thanks again johnsom)16:17
openstackgerritAnn Taraday proposed openstack/octavia stable/rocky: Fix base (VRRP) port abandoned on revert  https://review.opendev.org/68283616:17
johnsomredrobot Hi. Just mentioning we need to give a better error message when users upload a pkcs12 to barbican that is pass phrase protected.16:17
johnsomredrobot Since secrets don't have a pass phrase parameter.... Which, would kind of be pointless.16:17
cgoncalvescolin-, there's an open issue with the single-process. ataraday_ has a patch for it up for review16:18
cgoncalves#link https://review.opendev.org/#/c/681195/16:18
colin-ah i hadn't done one with multiple https listeners but glad we found it arleady and thanks to ataraday_16:19
johnsomAny other updates today?16:20
cgoncalveswith my stable liaison hat on, I proposed a handful of backport patches. they have been merging rather quick. thank you all for your reviews!16:21
johnsomPlease focus on getting the bug fixes merged for the RC1.16:21
johnsomWe should also strive to get our tempest patches merged. We will want to tag a tempest version at the same time.16:21
cgoncalvesyep16:22
johnsomThat bug fix above is a good example of a patch we really want in....16:22
johnsom#topic Open Discussion16:22
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"16:22
johnsomOther topics for today?16:22
cgoncalvesI have one16:23
cgoncalvesI put up for review a patch that enables full MAC security by default16:23
cgoncalves#link https://review.opendev.org/#/c/682932/16:23
johnsomGo for it16:23
*** gmann is now known as gmann_afk16:23
johnsomDoes that include apparmor or only selinux?16:23
cgoncalvesthus far we have been setting systems to permissive (RHEL, CentOS, Fedora)16:24
cgoncalvesonly SELinux as that is the only place right now we enable/disable16:24
johnsomWe should probably make this generic enough to work for either platform16:25
cgoncalveswe can consider including apparmor, sure16:25
cgoncalvesI'd like to know if this is something folks are positive about or not16:25
johnsomI think our friends at Canonical have the apparmor rules, though we may not have them here.16:26
cgoncalvesproduction clouds usually have selinux/apparmor enabled, so we would be testing code closer to such envs16:26
johnsomI think it is a great idea.16:26
cgoncalvescolin-, would you have some thoughts? :)16:27
cgoncalvesor just don't care16:27
cgoncalves(valid response anyway)16:28
*** goldyfruit_ has joined #openstack-lbaas16:28
cgoncalveshmm, ok. if there's any feedback please feel free to share it at a later time here on the channel or Gerrit16:29
cgoncalvesapparmor is in the package install linux. would someone know what is the default setting (permissive/enforcing)?16:31
cgoncalvess/linux/list/16:31
johnsomI think by default in DIB it's permissive/audit16:31
*** nmagnezi has quit IRC16:31
*** ivve has joined #openstack-lbaas16:32
johnsomOther topics today?16:34
johnsomOk, thanks everyone!16:35
johnsom#endmeeting16:35
*** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews"16:35
openstackMeeting ended Wed Sep 18 16:35:30 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:35
openstackMinutes:        http://eavesdrop.openstack.org/meetings/octavia/2019/octavia.2019-09-18-16.02.html16:35
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/octavia/2019/octavia.2019-09-18-16.02.txt16:35
openstackLog:            http://eavesdrop.openstack.org/meetings/octavia/2019/octavia.2019-09-18-16.02.log.html16:35
*** ccamposr has quit IRC16:49
*** tesseract has quit IRC16:53
*** luksky has joined #openstack-lbaas17:06
*** salmankhan has quit IRC17:09
openstackgerritCarlos Goncalves proposed openstack/octavia stable/stein: Work around strptime threading issue  https://review.opendev.org/68294917:16
openstackgerritCarlos Goncalves proposed openstack/octavia stable/rocky: Work around strptime threading issue  https://review.opendev.org/68295017:16
openstackgerritCarlos Goncalves proposed openstack/octavia stable/queens: Work around strptime threading issue  https://review.opendev.org/68295117:16
*** ataraday_ has quit IRC17:22
*** ricolin has quit IRC17:27
*** yamamoto has joined #openstack-lbaas17:33
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510317:36
*** yamamoto has quit IRC17:38
rm_workUgh yeah sorry lost track of what day it was and stayed up way too late shepherding those patches last night lol18:07
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510318:35
johnsomGreat, now this ipv6 stuff is failing in a new way this morning.18:35
johnsomMore logging! lol18:35
*** openstackgerrit has quit IRC18:37
*** goldyfruit___ has joined #openstack-lbaas18:38
*** goldyfruit_ has quit IRC18:41
*** openstackgerrit has joined #openstack-lbaas18:49
openstackgerritAdam Harwell proposed openstack/octavia master: Validate server_certs_key_passphrase is 32 chars  https://review.opendev.org/67073518:49
cgoncalvesjohnsom, on the positive side, you can add "data analytic expert" to your resume18:56
johnsomLol, that was a previous job....  I'm still looking for the country of "Europe" that kept showing up in those old data sets.... grin18:57
*** ccamposr has joined #openstack-lbaas18:59
openstackgerritAdam Harwell proposed openstack/octavia master: Validate server_certs_key_passphrase is 32 chars  https://review.opendev.org/67073519:00
rm_work^^ could use re +219:00
rm_workhad to fix a pep8 issue from the new rules19:00
cgoncalvesjohnsom, https://www.youtube.com/watch?v=kRh1zXFKC_o19:02
cgoncalvesrm_work, I'll approve once it passes pep819:03
rm_workNo faith! :P19:03
cgoncalveswere you monitoring the job when it entered the gate?19:04
*** gcheresh has joined #openstack-lbaas19:04
rm_workyes19:07
*** boden has quit IRC19:07
rm_worki am basically constantly monitoring stuff in the gates :D19:07
rm_worki am CRAAAAAZY impatient, lol19:08
rm_worki basically stared at gates last night for 3 hours19:08
rm_work>_<19:08
rm_workand watched training videos T_T19:09
rm_workwhich i am still doing19:09
*** Vorrtex has joined #openstack-lbaas19:17
johnsomMe too, staring at gate jobs way more than is healthy19:17
johnsomJoy, so for whatever reason the awk ENVIRON works great locally, but doesn't work at all in the job.....  Well, I can work around that.19:21
*** ajay33 has quit IRC19:21
*** ccamposr__ has joined #openstack-lbaas19:25
zigojohnsom: I'd like to say thank you for your yesterday's advice, now Octavia works so much better.19:26
zigoI just had to failover all the lbs that were in production.19:27
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510319:27
zigoIt's now in my installer ...19:27
zigohttps://salsa.debian.org/openstack-team/debian/openstack-cluster-installer/commit/a9101e4b753fba2728925c6ea237a836a507347119:27
zigo:)19:27
*** ccamposr has quit IRC19:28
johnsomzigo Awesome!  BTW, Stein forward you can use the Amphora config update and not need to failover: https://docs.openstack.org/api-ref/load-balancer/v2/index.html#configure-amphora19:28
zigojohnsom: Our production is still running Rocky ...19:29
johnsomSomething to look forward to then.... grin19:29
zigoI'll probably update it strait with stretch-rocky -> buster-rocky -> buster-stein -> buster-train when train is released.19:29
zigojohnsom: I'm also planning on having official OpenStack Debian images including an amphora image.19:30
zigoFYI, I did my own type of image, without using DIB.19:30
*** gcheresh has quit IRC19:31
johnsomRight, I remember. If those are public, send me a link so I can have it to share if someone asks19:31
zigojohnsom: It's there: https://salsa.debian.org/openstack-team/debian/openstack-debian-images/tree/debian/train/contrib/octavia19:32
openstackgerritMerged openstack/octavia stable/queens: Add warning log if auth_strategy is not keystone  https://review.opendev.org/67854919:32
zigoThough it's included in openstack-debian-images, directly in Debian.19:32
zigoThe Sid version only (in Buster, last february, it didn't work yet)19:33
johnsomOk, bookmarked19:33
johnsomUgh, chrome memory leaks... One window, three tabs (irccloud, etherpad, gmail), was using 4+GB RAM19:40
cgoncalvesI found this week Firefox renders large log files from CI so much faster than Chrome19:45
johnsomYeah, I have noticed chrome to be slow. I should try firefox19:45
johnsomFYI: would be good to get reviews on: https://review.opendev.org/#/c/682148/19:46
*** ccamposr__ has quit IRC19:47
*** gcheresh has joined #openstack-lbaas19:49
johnsomFYI In case anyone is interested: https://www.haproxyconf.com/agenda/19:52
johnsomI have a conflict so will not be attending.19:52
*** ccamposr has joined #openstack-lbaas19:55
*** ccamposr__ has joined #openstack-lbaas19:56
rm_workzigo: just remember that if you opt not to use DIB, you will need to keep up with changes that are made (for example, I have some significant and essential image building changes in my multi-vip patch) because we only commit to updating our DIB elements19:58
rm_workzigo: I know we've stressed to you how much we think that you should use DIB, but at this point I assume your mind is made up and further prodding is counter-productive -- so I just want to make sure you're aware that things might break suddenly, and if so I guess you can ask here and we can try to remember which patches you should look at :D19:59
*** ccamposr has quit IRC20:00
zigorm_work: Thanks for the reminder !20:01
zigorm_work: I told you the problem about DIB and stuff, using pip install.20:01
zigoThat's simply not acceptable from the Debian point of view.20:01
rm_workzigo: I believe we told you that it's totally possible to use binaries (in fact RHEL elements are configured to do this)20:01
zigoWe *must* use stuff in Debian, not taken from a random curl | bash thing.20:01
zigoThat's what pip does from our point of view.20:01
zigoRight.20:02
rm_workyou could simply update the elements for us to give a binary-debian option similar to the binary-rhel option :D20:02
rm_workwhich is a much better tactic IMO20:02
rm_workI can even help you with that if you'd like20:02
rm_workthere's no reason to say that installs MUST use pip -- that is completely untrue20:02
zigoThe issue is, the Debian image will soon be done with FAI (fully automated install) because we want things to be the same for all clouds (ie: aws, gce, azure, etc.), and therefore, I wont even have a say.20:03
zigoI expect to work on this on the next Debian cloud sprint.20:03
rm_worksimply add a section for debian here: https://github.com/openstack/octavia/blob/master/diskimage-create/diskimage-create.sh#L361-L36220:03
zigo(in Boston, at the MIT)20:03
rm_workzigo: who WOULD have a say? We'd love to talk to them in that case20:04
zigoNo-one will have a say, we decided to base all our images on FAI and unify everything...20:04
zigoI would have loved if we decided to use DIB rather than FAI.20:05
zigoThough it's too late to revert that decision.20:05
*** ccamposr__ has quit IRC20:05
*** ccamposr__ has joined #openstack-lbaas20:05
rm_workI feel like in general it's never too late to have a discussion and possibly make exceptions -- whether that's by officially defining an exception (which I agree might be difficult) or just by reclassifying what exactly an amphora image is and what restrictions must be applied to it20:06
rm_workI feel like it should be beyond scope to redefine build procedures for external ecosystems20:07
rm_workespecially when they have their own clearly defined standard already, and that is directly related to real code work20:08
rm_workyou will literally be signing up the debian openstack maintenance team (ie, YOU) to do additional coding work *for an external project*20:08
zigoFor FAI ?20:08
zigoI don't plan to work on that, no.20:09
rm_workAnd I know you view this as non-negotiable but I wonder if others do as well20:09
rm_workWell, someone will need to, or it will continuously break20:09
rm_workwho is going to sign up for that?20:09
zigoThe FAI thing has been maintained by Thomas Lange for over 10 years.20:09
rm_workright but someone will have to maintain the Octavia build system for FAI20:10
zigoI don't think he'll give up.20:10
rm_worknot just FAI itself20:10
rm_workand not just FAI for "debian"20:10
*** pcaruana has quit IRC20:10
zigoRight.20:10
rm_workI'd like to hear from him that he is willing to sign up for this, being fully informed about what that means and what the alternatives are20:11
zigoAnd we want to do that, because we want one single thing to build images, so we can tell that all our images are the same, using the same standards, etc.20:11
zigoThis really makes sense, you know.20:11
zigoSo that Debian users can expect the same thing whatever the cloud they are in.20:11
zigoWell, he's not, I guess that's on me.20:11
rm_workI agree that you think it makes sense. :D20:11
rm_workIf you use DIB for octavia-amp images, then you CAN guarantee users can expect the same thing from whatever cloud they're in20:12
rm_workand not just debian clouds -- ALL clouds that use octavia20:12
rm_workI agree with you about pip installs for debian images BTW -- I would love if you or someone would make the binary stuff be the default for debian20:13
zigoYou don't get it, I think.20:13
*** ccamposr__ has quit IRC20:14
zigoWe're talking about the same image, supported officially by cloud vendors.20:14
zigoLike, AWS, GCE and Azure will all promote the Debian official image we'll provide for them.20:14
*** ccamposr__ has joined #openstack-lbaas20:14
zigoWell, not the exact same image, but as close as possible.20:15
rm_workso if you provide them the one built with DIB... how do you think it would suddenly be different across those clouds, if you provide the same image?20:15
zigoIt works with some kind of templating.20:15
zigoFor lots of reasons, because of the way each tool is building stuff.20:16
zigoBecause each has its own preference.20:16
rm_workwhat you WILL do is make the experience inconsistent in those clouds between debian and everything else (I assume it's still possible for users to choose a rhel or ubuntu environment)20:16
zigoFor example, if we decide to add GNU screen to the image, then it's going to be added everywhere.20:16
rm_workyes, but if you build with DIB... and then release that image... the image won't suddenly mutate when AWS downloads it and GCE downloads it20:17
zigoLet's say we decide to switch from nano as the default editor to vim, same ...20:17
rm_workright, so add that to the DIB build scripts for debian20:17
zigoWe change one line, all image change.20:17
rm_workright, and if you then rebuild with DIB, we use the official debian cloud image base, so we'd get that change20:17
rm_workI don't see the issue?20:17
*** ccamposr has joined #openstack-lbaas20:17
zigoSo, basically, what you're proposing is *MORE* work, as I'd have to follow each and every change made with the FAI image so that the DIB one matches...20:18
*** ccamposr__ has quit IRC20:18
rm_workor if it's specific to octavia-amps, then you could just make that change in the DIB elements for debian, and bam, every image has the change20:18
rm_workerr, no?20:18
zigoWell, yes ! :)20:18
rm_workwe don't bootstrap from 020:18
rm_workwe use the FAI built image as a base20:18
rm_workso anything you changed there will automatically be represented in our image20:18
zigoAs much as I know, DIB does everything from zero, using debootstrap.20:19
rm_workDIB is really just an overlay on a base20:19
rm_workthat is absolutely wrong20:19
rm_workyou can choose20:19
zigoThat's how I built my first amphora.20:19
rm_workwe do all of our images starting from a base cloud image20:19
rm_workthen you chose a strange way to do it :D20:19
zigo:)20:19
rm_workif you look at how all of our current images are built, we define which base image to use20:19
zigoOk.20:20
rm_workwhich in the debian case would be the output from your base FAI cloud image build20:20
rm_workand then we overlay with our elements (add package A, remove package B)20:20
rm_workthat's all20:20
rm_workso the workflow of: start with debian-official FAI built cloud image -> add appropriate packages via apt-install-binaries -> finalize image20:21
rm_workis what you'd want to do for our DIB build20:21
rm_workit's what I would have recommended *regardless*20:21
rm_workand is what our elements are already set up to do20:21
*** ccamposr has quit IRC20:22
zigorm_work: Have you seen that, in Debian, I just have an octavia-agent that I just install?20:22
zigoThat's kind of nice, no?20:22
johnsomNot to derail this discussion, but we are now using a debootstrap method to get the image sizes down.20:22
rm_workjohnsom: well, we don't need to -- and we'd just match what he is doing already by not doing that20:23
rm_worki'd recommend starting with `debian-minimal`20:23
rm_workah hmm, no, debian-minimal does use debootstrap20:23
johnsomYeah, usually the minimal elements are build it up using debootstrap.20:23
rm_workthen we'd just need a debian-cloudimage element or something20:23
johnsomThen, centos for example installs the Octavia RPMs...20:24
rm_workbut that's entirely doable -- that's how ubuntu/rhel/etc work20:24
rm_workor did work until johnsom started hacking at them :D20:24
johnsomYeah, image based is pretty straight forward20:24
rm_workso I'd propose a debian-cloudimage element that mirrors how either fedora or centos work20:25
rm_workand then adding the necessary bits for a binary install to work on debian20:26
rm_workwhich I can probably do, I'm very familiar with debian as well (I practically started my linux journey with it)20:26
rm_worklet me check first to see if there is already an element somewhere for debian-image-based builds20:28
johnsomYeah, it would just be a "debian" element that pulls in the image. then the pkg map would map the octavia repo bits to the debian packages.20:28
rm_workyep20:28
johnsomI know it works because Helion was debian pkg based20:28
rm_workwhich leads to a consistent build that is based on exactly the official FAI images20:28
johnsomWe used DIB20:28
zigorm_work: At this point, I'd have to modify Rocky's Octavia in Buster to make the octavia-agent pacakge work.20:28
zigoWhich I didn't have the time to work on yet ... :(20:28
rm_workzigo: yeah, i'm just saying moving forward this would be the ideal20:28
zigo(ie: only the stein Octavia package works well enough...)20:28
rm_workI would leave rocky as is20:28
rm_workbecause the real issue is with new patches that change how the image builds, and we don't usually backport those anyway20:29
rm_workso whatever you have for rocky should continue to function fine20:29
rm_workjust moving forward, it would be good to look at this method instead20:29
rm_workI *will* commit to helping with this20:29
zigoThe issue is, if we want to provide a Debian *STABLE* image, then the work must be done on Rocky (because that's what Buster has...).20:30
*** gcheresh has quit IRC20:30
rm_workwe can get it working with current stuff, and then we can look at backporting20:30
zigoThat's perfectly fine otherwise, to build a Debian testing weekly image based on whatever's in current testing though.20:30
rm_workI don't see a problem with backporting an additional new build method20:30
zigoWell, *I* must do the work on updating the Octavia package in Buster.20:30
zigo:/20:30
zigo(I mean, backporting my fixes)20:31
rm_workand yes, that allows us to easily test debian in our gate, in scenarios, with official debian images/packages20:31
zigoYou could already do that with the unofficial Buster backports repo.20:31
rm_workour gate tests need to have the *option* of building the amp using agent-from-git20:32
rm_workotherwise they can't test properly20:32
zigoRight.20:32
rm_workbut it would test with the rest of the official ecosystem20:32
rm_workwhich would be much closer to your images20:33
rm_workI'd just like to get these couple of things done, and then take another look and see if it's closer to something you'd be willing to work with20:33
zigoI guess I could try to figure out something.20:33
zigoThis would be after the cloud sprint in Boston then.20:34
rm_workkk20:34
rm_workI'll see if I can get time to do some of the prep work before that so when you're ready we'll be closer20:34
cgoncalvesUgh, parsing feedback from the user survey spreadsheet is a nightmare20:36
johnsomHey, nice. Is this the first time they sent this out public? I don't think I have ever seen one20:38
rm_workerr is this actually necessary? https://review.opendev.org/#/c/682148/20:38
rm_workwouldn't we have noticed before that it needs that to work?20:38
johnsomSo, this confused me too.... It's not necessary for the job to run in the Octavia repo, but it is if we run it in the octavia-lib repo. We didn't see it because we had a Depends-On for the octavia-lib merge20:39
johnsomrm_work https://review.opendev.org/#/c/682120/20:40
rm_workah >_<20:40
rm_workgot it20:40
johnsomI had to phone a friend in infra to figure out what was going on20:40
rm_workok20:40
rm_work+A20:40
rm_workdid you want to try to get this in? https://review.opendev.org/#/c/580778/20:41
johnsomNot critical20:41
cgoncalvesI think our bot reviewers haven't noticed yet they have been voting on backport-candidate label instead of code-review xD20:41
rm_worklol yep20:41
johnsomYeah, we were just commenting on that the other day20:42
rm_workI suggested we add another field and sort it to the top20:42
rm_workthat is a bot-detector20:42
johnsomBot backporting the release bot patch....20:42
cgoncalvesalpha sorting20:42
rm_workyeah20:42
rm_workaIMABot20:42
johnsomMaybe we can get the bots into a loop?20:42
*** ccamposr has joined #openstack-lbaas20:43
johnsomUmm:20:43
*** goldyfruit___ has quit IRC20:44
johnsomhttps://www.irccloud.com/pastebin/6JBokLHG/20:44
johnsomEither the survey got messed up or the report.20:44
rm_workhmmm20:44
rm_workwait where are you looking? where's the survey results?20:44
johnsomopenstack-discuss list20:45
cgoncalveshttp://lists.openstack.org/pipermail/openstack-discuss/2019-September/009501.html20:45
johnsomYeah, I think the csv has some bugs. I see nova columns shifted all over the place20:45
*** ccamposr has quit IRC20:48
johnsomI might play with this later and create an Octavia specific report with cleaned up data.20:49
johnsomcgoncalves See, there is probably a country of Europe in here somewhere.....20:49
cgoncalveslol20:53
cgoncalvesit looks like active-active is the #1 most wanted feature20:54
johnsomNot a huge surprise there20:54
*** goldyfruit___ has joined #openstack-lbaas20:56
cgoncalves67 active-active20:56
cgoncalves28 container-based amphora driver20:57
cgoncalves21 VIP ACL20:57
cgoncalves30 Simultaneous IPv4 and IPv6 VIP20:57
*** nmagnezi has joined #openstack-lbaas20:57
cgoncalves32 Log offloading20:57
cgoncalves36 Statistics (more metrics)20:58
cgoncalves33 HTTP/2 protocol20:59
rm_workit's our highest priority at VZM20:59
cgoncalvesHTTP/2?20:59
johnsomYeah, the new HTTP version. HAProxy 2.0 supports it21:00
johnsomThough it's still stablizing IMO21:00
cgoncalves21 OVN (provider driver). surprising to me21:00
johnsomMe too21:00
rm_workI mean Active-Active21:00
johnsomSo 21 Red Hat employees filled out the survey?21:01
cgoncalvesFWIW, I didn't fill it :)21:02
cgoncalves52 HAProxy Technologies. do you think folks think amphora==HAProxy Technologies?21:04
johnsomOr the old HAProxy driver21:05
johnsomBut maybe they want an official HAProxy Technologies driver....21:05
cgoncalvesor are stuck in the neutron-lbaas era and don't know about the amphora driver21:06
openstackgerritMerged openstack/octavia stable/stein: Fix base (VRRP) port abandoned on revert  https://review.opendev.org/68283321:11
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510321:13
johnsomThis is killing me. So simple yet no-worky21:14
*** goldyfruit_ has joined #openstack-lbaas21:22
*** goldyfruit___ has quit IRC21:25
*** Vorrtex has quit IRC21:31
rm_work>_<22:07
rm_worki can maybe look22:08
*** goldyfruit_ has quit IRC22:15
rm_workneed to get my devstack back up and running lol22:21
rm_workreally? https://review.opendev.org/#/c/679358/ failed AGAIN in the gate? T_T22:31
rm_worksomething died in DIB but i don't see the dib log22:33
rm_worknm found it22:34
rm_workwat22:34
rm_workE: Failed getting release file http://archive.ubuntu.com/ubuntu/dists/xenial/Release22:34
rm_workhttps://52df86e50fd53575865f-ba41f76c71830a91780de57d2c593811.ssl.cf5.rackcdn.com/679358/2/gate/octavia-grenade/ef029d4/logs/dib-build/amphora-x64-haproxy.qcow2.log22:34
*** goldyfruit_ has joined #openstack-lbaas22:40
*** tkajinam has joined #openstack-lbaas23:02
*** luksky has quit IRC23:13
*** rcernin has joined #openstack-lbaas23:16
*** goldyfruit_ has quit IRC23:24
*** goldyfruit_ has joined #openstack-lbaas23:29
johnsomhttps://usercontent.irccloud-cdn.com/file/f7o9nh51/2019%20OpenStack%20User%20Survey.pdf23:47
johnsomSome quick and dirty charting23:47
*** ivve has quit IRC23:48
openstackgerritMichael Johnson proposed openstack/octavia master: Allow IPv6 health network in devstack  https://review.opendev.org/66510323:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!