Wednesday, 2020-07-22

« Tuesday, 2020-07-21 Index Thursday, 2020-07-23 »
*** ccamposr has joined #openstack-lbaas00:09
*** ccamposr__ has joined #openstack-lbaas01:04
*** ccamposr has quit IRC01:07
*** ccamposr has joined #openstack-lbaas01:08
*** ccamposr__ has quit IRC01:09
openstackgerritBrian Haley proposed openstack/octavia master: Remove Neutron SDN-specific code  https://review.opendev.org/71819201:10
openstackgerritBrian Haley proposed openstack/octavia master: Remove Neutron SDN-specific code  https://review.opendev.org/71819201:11
*** also_stingrayza has joined #openstack-lbaas01:22
*** stingrayza has quit IRC01:25
*** ccamposr__ has joined #openstack-lbaas01:26
*** ccamposr has quit IRC01:27
*** ccamposr has joined #openstack-lbaas01:34
*** ccamposr__ has quit IRC01:34
*** ccamposr has quit IRC01:59
*** ccamposr has joined #openstack-lbaas01:59
*** ccamposr__ has joined #openstack-lbaas02:02
*** ccamposr has quit IRC02:03
*** sapd1 has joined #openstack-lbaas02:41
*** psachin has joined #openstack-lbaas03:31
sorrisonjohnsom, rm_work, pulled in all those failover fixes and looks like it's fixed all our issues05:04
sorrison108 amphorae found on load balancer ed632e08-d649-4601-b1cf-84a694f4b272 where two should exist. Repairing.05:04
devfazsorrison: which fixes do you refer to?05:08
sorrisonmainly https://review.opendev.org/#/c/739002/05:15
sorrisonWe were running train and now on ussuri with this patch applied too05:16
*** gcheresh has joined #openstack-lbaas05:28
*** sapd1 has quit IRC05:32
*** gcheresh_ has joined #openstack-lbaas05:39
*** gcheresh has quit IRC05:39
*** maciejjozefczyk_ has joined #openstack-lbaas05:43
*** gcheresh_ has quit IRC05:44
*** sapd1 has joined #openstack-lbaas05:54
*** vishalmanchanda has joined #openstack-lbaas05:55
*** gcheresh_ has joined #openstack-lbaas05:56
*** dayou has quit IRC06:20
*** dayou has joined #openstack-lbaas06:28
*** ltomasbo has joined #openstack-lbaas06:41
*** ltomasbo has left #openstack-lbaas06:41
*** ccamposr has joined #openstack-lbaas06:52
*** ccamposr__ has quit IRC06:54
*** born2bake has joined #openstack-lbaas07:04
*** sapd1 has quit IRC07:10
cgoncalvessorrison, great feedback, thank you!07:11
cgoncalvessorrison, FYI, there is a bug where load balancers fail to create if topology is SINGLE and Nova anti-affinity is enabled -- https://storyboard.openstack.org/#!/story/2007943. I am going to propose a patch later today, just spent too much time yesterday fighting with Glance API not starting07:11
*** sapd1 has joined #openstack-lbaas07:13
*** gcheresh_ has quit IRC07:22
*** gcheresh has joined #openstack-lbaas07:23
*** rcernin has quit IRC07:32
*** ccamposr__ has joined #openstack-lbaas07:45
*** ccamposr has quit IRC07:48
*** gcheresh has quit IRC07:50
*** gcheresh has joined #openstack-lbaas08:01
*** sapd1 has quit IRC08:13
*** sapd1 has joined #openstack-lbaas08:22
*** ccamposr__ has quit IRC08:31
*** also_stingrayza is now known as stingrayza08:40
*** rcernin has joined #openstack-lbaas09:10
*** rcernin has quit IRC09:35
*** sapd1 has quit IRC09:37
openstackgerritCarlos Goncalves proposed openstack/octavia master: Fix missing dependency in amphora create flow  https://review.opendev.org/74238210:01
*** sapd1_y has quit IRC10:14
*** sapd1 has joined #openstack-lbaas10:14
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Enable nova anti-affinity in two-node act-stdby job  https://review.opendev.org/74219910:17
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Add act-stdby scenario jobs to the gate  https://review.opendev.org/74238510:27
*** tkajinam has quit IRC10:59
*** rcernin has joined #openstack-lbaas11:49
openstackgerritCarlos Goncalves proposed openstack/octavia stable/ussuri: Fix missing dependency in amphora create flow  https://review.opendev.org/74240111:57
openstackgerritCarlos Goncalves proposed openstack/octavia stable/train: Fix missing dependency in amphora create flow  https://review.opendev.org/74240211:58
*** maciejjozefczyk_ has quit IRC12:01
*** maciejjozefczyk has joined #openstack-lbaas12:01
*** rouk has quit IRC12:38
*** headphoneJames has quit IRC12:38
*** rm_work has quit IRC12:38
*** headphoneJames has joined #openstack-lbaas12:42
*** rm_work has joined #openstack-lbaas12:42
*** sapd1 has quit IRC12:56
*** rcernin has quit IRC13:32
*** TrevorV has joined #openstack-lbaas13:43
*** ramishra has quit IRC14:22
*** ramishra has joined #openstack-lbaas14:22
*** psachin has quit IRC14:24
*** sapd1 has joined #openstack-lbaas14:37
*** KeithMnemonic has joined #openstack-lbaas14:59
*** ccamposr has joined #openstack-lbaas15:02
*** gcheresh has quit IRC15:44
*** tow has joined #openstack-lbaas15:47
johnsom#startmeeting Octavia16:00
openstackMeeting started Wed Jul 22 16:00:06 2020 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: Octavia)"16:00
openstackThe meeting name has been set to 'octavia'16:00
johnsomHi everyone16:00
rm_worko/16:00
aannuusshhkkaahello!16:00
gthiemongeHi16:00
johnsom#topic Announcements16:01
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"16:01
*** shtepanie has joined #openstack-lbaas16:01
johnsomI don't have much in the announcement category today.16:01
johnsomI will remind everyone that next week is the Victoria milestone 2 week.16:01
johnsom#link https://releases.openstack.org/victoria/schedule.html#v-216:01
cgoncalveshi16:01
johnsomAny other announcements today?16:02
johnsom#topic Brief progress reports / bugs needing review16:02
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"16:02
johnsomI have been focused on doing reviews and working on the failover v2 patch. It is now fully functional (haven't pushed that yet), but I still have some cleanup and test work to do.16:03
gthiemongeI have #link https://review.opendev.org/#/c/735126/ (add a config setting for a default connection_limit in the amphora driver) in review, perhaps we could discuss it later16:05
johnsomAny other updates this week?16:05
cgoncalvesI have posted a patch to fix a bug introduced recently in master that also impacts stable/train. if you run on SINGLE topology and nova anti-affinity enabled, please consider testing https://review.opendev.org/#/q/I757d84e4624f488382ec4ff9af3f489cd9a85ed316:05
johnsomYeah, we should discuss that. Let's do that next16:05
johnsomcgoncalves Thanks!16:05
gthiemongesure16:06
cgoncalveswe also released octavia-lib 2.1.0 and 2.1.116:06
johnsomI think there has been some work on the metrics side as well16:07
*** ZhuXiaoYu has joined #openstack-lbaas16:07
aannuusshhkkaaYeap, we have 3 patches up16:08
aannuusshhkkaa1 - delta changes, 2 - Refactoring driver layer, 3 - Added new metric - response time16:08
aannuusshhkkaaWe still have to merge the three together and in that order16:09
johnsomNice16:09
shtepaniethe delta changes are pretty much complete, as well as the driver layer refactor16:09
johnsom#topic default connection_limit in the amphora driver16:09
*** openstack changes topic to "default connection_limit in the amphora driver (Meeting topic: Octavia)"16:09
johnsomGreg you have the floor16:09
gthiemongethanks16:09
gthiemongefor reference, the story is #link https://storyboard.openstack.org/#!/story/200779416:10
gthiemongebasically, default connection_limit in the api is -1, which is translated to 1,000,000 in the amphora driver16:10
rm_workyeah we ran into this internally -- had to get people to start setting it manually to a lower value as a workaround16:11
rm_workwould be good to get a fix in16:11
gthiemongeit leads to a huge memory consumption in recent haproxy releases, that might trigger bugs when reloading the configuration (combined with some features, such as stateful table in HA)16:11
rm_workit happens very easily with terraform since it does a bunch of stuff back-to-back16:11
johnsomI raised some concern about the proposed patch in  that it secretly changes -1 to some number defined by the operator.16:11
gthiemongeso the idea is to add a default_connection_limit option in octavia that could be 50000 (admin can set it back to 1M if he needs)16:12
rm_workI don't see it being realistic to have it higher honestly16:12
cgoncalvesfirst, thank you Greg for finding and reporting this issue and for proposing a patch!16:12
gthiemongein my last patch, -1 sets 50k in the amphora, but the user doesn't know that it is 50k16:13
johnsomWell, with one core, agreed. But, with future versions multi-core will do more16:13
johnsomAgreed, thank you for working on this.16:14
gthiemonge;-)16:14
cgoncalvesmy concern is about violating the API contract by starting returning a different  connection limit. for example, user sets -1 but API returns 50,000 or whatever is the default16:14
johnsomYeah, it's a tough one. If we don't swap it and show them the number set, people will think whatever the operator set is the limit of the load balancer, which it may not be.16:15
cgoncalvesI have to be consistent and fair with everyone since I pushed back on a Neutron SG API change not so long ago very similar to this case16:16
johnsomWell, that was a bit different IMO as it was changing user input data and this is changing the meaning of -116:17
cgoncalvesit would still change the return value if users set to -116:18
rm_workyeah we should change the default (and note that) but not *translate* -116:19
johnsomI just feel that users should be able to see what the actual value is set to.16:19
johnsomSo I lean towards either taking -1 and returning the user the actual value, both at create and future "show" calls. Or we just eliminate -1 as an option and set some other default value16:20
gthiemonge1. would require to translate the value before entering the amphora driver, right?16:22
johnsomYes, I think so16:22
johnsomI think it's a bit better transition if we still accept -1 but translate it. Just in case someone has coded something like terraform to use -1.16:25
cgoncalves+116:26
rm_worki ... feel like that's unlikely16:27
rm_workbut ok16:28
cgoncalvesI'm just agreeing with continue to accept -116:28
johnsomI'm throwing ideas out. Please if you have input share.16:29
johnsomDo we need a vote?16:31
johnsomgrin16:31
cgoncalvesif we determine 50,000 is an unattainable connection limit amphora load balancers, that would still be an unlimited connection limit16:31
rm_worki believe it is16:31
rm_workbut /shrug16:32
cgoncalvesrm_work, I'm disappointing. you should be screaming "API v3!!"16:32
johnsomYeah, I think 50k is probably a decent default value.16:32
rm_worklol i mean yeah16:32
johnsomI think the only question is how we handle the API part.16:33
johnsomThe current patch leaves -1 in place in all of the API return values, but secretly changes it in the amphora to 50k16:33
cgoncalveswe may need to consider a fix needs to be backported to stable branches, so either we come up with *one* solution that is backportable or *two* solutions (master / stable branches)16:34
cgoncalvesjohnsom, it is secretly changing it to 1M today16:35
johnsomYeah, which is effectively "unlimited" lol But, hey with 2.2 you might be able to throw enough cores at it....16:35
cgoncalvesright, that is why I mentioned that we may need to come up with two solutions: one for master and another for stable branches16:36
johnsom1. Leave the patch as is and secretly set the value to whatever the operator picks.16:36
johnsom2. Change -1 to the value the operator picks.16:37
johnsom#vote Which path should we take? 1, 216:37
johnsom#startvote Which path should we take? 1, 216:37
openstackBegin voting on: Which path should we take? Valid vote options are 1, 2.16:37
openstackVote using '#vote OPTION'. Only your last vote counts.16:37
johnsomlol, it's been to long since we needed to vote.16:38
cgoncalves#vote 116:38
johnsom#vote 216:38
gthiemonge#vote 116:39
cgoncalvesoption 1 allows us to still take option 2. taking option 2 today is irreversible, I think16:39
gthiemonge(i'm lazy)16:39
johnsomaannuusshhkkaa, shtepanie, rm_work?16:39
johnsomAnyone else that may be in the meeting...16:40
aannuusshhkkaai dont think i have enough knowledge about the consequences to vote..16:40
ZhuXiaoYu#vote 116:40
rm_workhmm16:41
shtepaniei dont have enough knowledge / context either16:41
rm_worksorry doordash arrived16:41
johnsomlol16:41
rm_workuhh16:42
rm_work#vote 216:42
rm_worki think i get it16:42
cgoncalvesdoordash also gets to vote if they use octavia16:42
johnsomTrue16:42
johnsomOk, I think that is enough time and everyone that I know is in the meeting.16:43
johnsom#endvote16:43
openstackVoted on "Which path should we take?" Results are16:43
openstack1 (3): ZhuXiaoYu, cgoncalves, gthiemonge16:43
openstack2 (2): rm_work, johnsom16:43
johnsomSo it looks like folgers crystals it is.16:44
johnsomThanks Greg!16:44
johnsom#topic Open Discussion16:44
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"16:44
johnsomOther topics today?16:44
gthiemongeThanks Folks!16:44
ZhuXiaoYuExcuse me. About my Active/Active spec, I want to if there is16:45
ZhuXiaoYuanything else we need to provide16:45
johnsomAh, yes. That needs some reviews I suspect.16:45
johnsom#link https://review.opendev.org/72386416:45
cgoncalves#link https://review.opendev.org/#/c/723864/16:45
johnsomYeah, that one.16:46
johnsomI had put this on the meeting agenda a few weeks back. It doesn't look like it got the reviews it needs.16:46
cgoncalvesI have only read quickly the first version, sorry16:46
johnsom#action Please review the Act/Act spec https://review.opendev.org/72386416:47
ZhuXiaoYuWe are working on a POC right now, it's coming up in August.16:47
johnsomIt got an update last week, so is due for another review pass16:48
cgoncalves#link https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_14a/723864/5/check/openstack-tox-docs/14aaa63/docs/contributor/specs/version1.1/alternative-active-active-l3-distributor.html16:48
johnsomZhuXiaoYu Thank you for reminding us16:48
cgoncalves^ HTML rendered page16:48
johnsomAny other items today?16:50
rm_workoh did we link the delta patch?16:50
rm_workit's ready to go16:50
rm_workfailing on the docs issue16:51
rm_workbut we'll wrap that today hopefully?16:51
johnsomIt wasn't linked, but mentioned16:51
rm_work(the octavia-lib docs issue, not with the delta patch)16:51
rm_work#link https://review.opendev.org/#/c/740815/16:51
johnsomOk then, thanks everyone for the good discussion today.16:52
johnsom#endmeeting16:52
*** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews"16:52
openstackMeeting ended Wed Jul 22 16:52:53 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:52
openstackMinutes:        http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-07-22-16.00.html16:52
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-07-22-16.00.txt16:52
openstackLog:            http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-07-22-16.00.log.html16:52
*** sapd1 has quit IRC17:03
*** ccamposr has quit IRC17:41
rm_workwait so did cgoncalves say we did an octavia-lib release already?17:48
rm_workor do we still need another one to fix the doc issue17:48
johnsomI think so17:48
johnsomYeah, it is in 2.1.117:49
rm_worknice17:54
rm_workok so rechecks17:54
cgoncalvesrm_work, wait for https://review.opendev.org/#/c/742422/ to merge first17:55
rm_workah k17:55
*** TMM has quit IRC18:34
*** TMM has joined #openstack-lbaas18:34
*** ZhuXiaoYu has quit IRC18:44
*** maciejjozefczyk has quit IRC19:26
*** also_stingrayza has joined #openstack-lbaas19:34
*** stingrayza has quit IRC19:36
openstackgerritBrian Haley proposed openstack/octavia master: Remove Neutron SDN-specific code  https://review.opendev.org/71819219:51
openstackgerritMichael Johnson proposed openstack/octavia master: Update amphora v2 for the failover refactor  https://review.opendev.org/73829219:56
johnsomThat one seems to be fully functional. I just want to do some testing and cleanup.19:57
*** TrevorV has quit IRC19:59
cgoncalvesrm_work, go for recheck20:06
rm_worksweet thx20:06
johnsomlol, my failover patch ran just before that landed.20:31
*** gcheresh has joined #openstack-lbaas20:46
*** gcheresh has quit IRC21:07
*** shtepanie has quit IRC21:41
*** vishalmanchanda has quit IRC22:01
*** ccamposr has joined #openstack-lbaas22:45
*** rcernin has joined #openstack-lbaas22:52
*** rcernin has quit IRC22:58
*** rcernin has joined #openstack-lbaas23:01
*** rcernin has quit IRC23:04
*** rcernin has joined #openstack-lbaas23:05
johnsomWell, that was a bummer, the config file flipped back to the v1 driver, so I was testing the wrong code this morning. sigh23:07
*** born2bake has quit IRC23:30
johnsomI was like, hmm, didn't touch that code. Then looked at the log: neutronclient.common.exceptions.OverQuotaClient: Quota exceeded for resources: ['security_group']23:40
« Tuesday, 2020-07-21 Index Thursday, 2020-07-23 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!