Wednesday, 2018-12-19

*** ccamposr has quit IRC00:01
*** PagliaccisCloud has joined #openstack-lbaas00:41
*** sapd1 has joined #openstack-lbaas00:45
*** PagliaccisCloud has quit IRC00:48
*** abaindur has quit IRC00:53
*** abaindur has joined #openstack-lbaas00:54
*** sapd1 has quit IRC01:20
*** colby_ has quit IRC01:45
rm_workjohnsom: does building an amp image really require root? <_<01:49
rm_worki never really noticed01:50
rm_worki guess it has to mount stuff01:51
openstackgerritMerged openstack/octavia master: Change openstack-dev to openstack-discuss  https://review.openstack.org/62200501:57
*** dayou has quit IRC01:59
johnsomI want to say it didn’t originally, that it used the userspace tools, but I guess DIB has changed.02:03
*** sapd1 has joined #openstack-lbaas02:04
*** dayou has joined #openstack-lbaas02:06
*** sapd1 has quit IRC02:09
openstackgerritMerged openstack/octavia-tempest-plugin master: Adds a mixed IPv4/IPv6 members traffic test  https://review.openstack.org/61140502:14
*** colby_ has joined #openstack-lbaas02:17
rm_workhmmm02:20
rm_worki wonder if there's a way around that02:20
*** psachin has joined #openstack-lbaas02:57
*** yangjianfeng has joined #openstack-lbaas03:11
*** yangjianfeng has quit IRC03:14
*** yamamoto has quit IRC03:34
*** yamamoto has joined #openstack-lbaas03:34
*** yamamoto has quit IRC03:36
*** hongbin has joined #openstack-lbaas03:51
*** sapd1 has joined #openstack-lbaas04:07
*** beno12345 has joined #openstack-lbaas04:26
beno12345hi, just updated from 2.0.1 to 2.0.2 (queens) and previously had 3 health-manager processes...now theres 11 and its making repeated connections to rabbitmq as the neutron user which dont seem to close.....we idled at 400 rabbitmq connections previously...now its hitting 1100...restart health manager drops them all but they quickly come back....any changes that could be related?04:27
openstackgerritGerman Eichberger proposed openstack/octavia master: Amphora logging  https://review.openstack.org/62483504:49
xgermanbeno12345: the only thing I know which is messing with the queue is the event streamer (see https://github.com/openstack/octavia/blob/master/etc/octavia.conf#L100) this should be noop04:52
xgermannot sure how you would end up with 11 processes since we only start the number defined in the configs + had/backported the process patch for a long while04:53
beno12345zgerman...yeah, just noticed we had it set to queue_event_streamer...changed tov noop and not it doesnt make the rabbitmq connections (guess this is no longer needed and not using neutron-lbaas)05:04
beno12345which config option to set the number of health-manager processes....i couldnt find anything that looked related05:05
*** hongbin has quit IRC05:40
*** dims has quit IRC06:27
*** dims has joined #openstack-lbaas06:28
*** gcheresh has joined #openstack-lbaas06:32
*** dims has quit IRC06:33
*** dims has joined #openstack-lbaas06:34
*** celebdor has joined #openstack-lbaas06:38
*** yboaron has joined #openstack-lbaas06:39
*** PagliaccisCloud has joined #openstack-lbaas06:55
*** beno12345 has quit IRC07:10
*** etp has joined #openstack-lbaas07:35
*** celebdor has quit IRC07:41
*** sapd1 has quit IRC07:49
*** pcaruana has joined #openstack-lbaas08:03
*** rpittau has joined #openstack-lbaas08:04
*** sapd1 has joined #openstack-lbaas08:06
*** yboaron has quit IRC08:15
*** pcaruana has quit IRC08:24
*** pcaruana has joined #openstack-lbaas08:33
*** pcaruana has quit IRC08:41
openstackgerritOpenStack Proposal Bot proposed openstack/octavia-dashboard master: Imported Translations from Zanata  https://review.openstack.org/62611508:42
*** velizarx has joined #openstack-lbaas08:50
*** rcernin has quit IRC08:51
*** celebdor has joined #openstack-lbaas08:54
*** yboaron has joined #openstack-lbaas09:00
*** ccamposr has joined #openstack-lbaas09:13
*** abaindur has quit IRC09:25
*** sapd1 has quit IRC09:34
*** sapd1 has joined #openstack-lbaas09:35
openstackgerritCarlos Goncalves proposed openstack/octavia master: Set same debug log level as per DevStack local.conf  https://review.openstack.org/62613209:56
nmagnezicgoncalves, how did we live without ^^ ?? :D10:08
cgoncalveswe never had to debug octavia. it is so perfect and flawless10:09
nmagneziTrue.10:09
*** salmankhan has joined #openstack-lbaas10:15
*** salmankhan1 has joined #openstack-lbaas10:23
*** salmankhan has quit IRC10:24
*** salmankhan1 is now known as salmankhan10:24
openstackgerritMerged openstack/octavia-dashboard master: Imported Translations from Zanata  https://review.openstack.org/62611510:26
*** psachin has quit IRC10:29
openstackgerritCarlos Goncalves proposed openstack/octavia master: Fixing data model to_dict() recursive function  https://review.openstack.org/60538210:30
*** sapd1 has quit IRC11:10
*** sapd1 has joined #openstack-lbaas11:17
*** salmankhan has quit IRC11:35
*** phuoc_ has joined #openstack-lbaas11:43
*** phuoc has quit IRC11:47
*** phuoc_ has quit IRC11:47
*** phuoc_ has joined #openstack-lbaas11:47
*** rpittau is now known as rpittau|lunch12:19
*** celebdor has quit IRC12:27
*** hogepodge has quit IRC12:29
*** hogepodge has joined #openstack-lbaas12:30
*** velizarx has quit IRC12:31
*** velizarx has joined #openstack-lbaas12:48
*** salmankhan has joined #openstack-lbaas12:50
*** sapd1 has quit IRC13:03
*** rpittau|lunch is now known as rpittau13:19
*** celebdor has joined #openstack-lbaas13:27
openstackgerritZhaoBo proposed openstack/octavia master: Support Host header inject for healthmonitor HTTP 1.1 health check  https://review.openstack.org/62618314:52
*** amuller has joined #openstack-lbaas15:14
openstackgerritZhaoBo proposed openstack/python-octaviaclient master: Add new options to HealthMonitor CLI  https://review.openstack.org/62619015:15
*** gcheresh has quit IRC15:32
*** velizarx has quit IRC15:42
openstackgerritJoe Talerico proposed openstack/octavia master: Fix config options  https://review.openstack.org/62622716:03
*** yboaron_ has joined #openstack-lbaas16:11
*** yboaron has quit IRC16:14
*** ccamposr has quit IRC16:32
dulekHey folks, do you have a minimum supported protocol set to consider an Octavia driver valid?16:52
johnsomHmmm, interesting question. Currently no.16:53
*** yboaron_ has quit IRC16:55
dulekjohnsom: Okay, just for context - we simply don't want to maintain this: https://github.com/openstack/kuryr-kubernetes/blob/8f871ee23476ddfe009c5c25fda07a8a01379669/kuryr_kubernetes/controller/drivers/lbaasv2.py#L40-L4316:55
dulekjohnsom: Especially if someone uses some homegrown driver. ;)16:55
dulekjohnsom: At least TCP seems to be lowest common denominator?16:56
johnsomTypically yes.16:56
*** rpittau has quit IRC17:06
xgermanjohnsom: https://review.openstack.org/#/c/624835/17:26
xgermanin my devstack:17:26
xgermanstack@octavia:~$ ls -l /var/log/octavia-amphora.log17:26
xgerman-rw-r----- 1 syslog adm 1209 Dec 19 12:24 /var/log/octavia-amphora.log17:26
xgermanstack@octavia:~$17:26
xgermanso maybe I need to touch that before I let it loose17:27
xgerman?17:27
xgermannever mind17:33
openstackgerritGerman Eichberger proposed openstack/octavia master: Amphora logging  https://review.openstack.org/62483517:33
*** gcheresh has joined #openstack-lbaas17:39
*** salmankhan has quit IRC18:05
*** gcheresh has quit IRC18:08
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Fix dependency on requests library version  https://review.openstack.org/62583618:12
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Fix dependency on requests library version  https://review.openstack.org/62583618:40
cgoncalveshow many PS do I need for a one-liner patch -_-18:41
amullercgoncalves: certainly you intended to file a story and attach it to the commit message :)18:53
cgoncalvesamuller, psssst! I thought of that on PS2. I AM sure no one cares for the tempest plugin18:59
amullerbad habit19:00
openstackgerritCarlos Goncalves proposed openstack/octavia-tempest-plugin master: Fix dependency on requests library version  https://review.openstack.org/62583619:09
cgoncalvesyou win19:09
*** yboaron_ has joined #openstack-lbaas19:12
amullerit's a good story :)19:38
*** yboaron_ has quit IRC19:43
*** gcheresh has joined #openstack-lbaas19:45
johnsom#startmeeting Octavia20:00
openstackMeeting started Wed Dec 19 20:00:04 2018 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.20:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.20:00
*** openstack changes topic to " (Meeting topic: Octavia)"20:00
openstackThe meeting name has been set to 'octavia'20:00
johnsomHi all20:00
xgermano/20:00
nmagnezio/20:00
johnsomThis might be it this week. I know some folks are out.20:00
johnsom#topic Announcements20:01
*** openstack changes topic to "Announcements (Meeting topic: Octavia)"20:01
johnsomDenver Summit call-for-papers is now open, deadline January 23rd.20:01
johnsom#link http://lists.openstack.org/pipermail/openstack-discuss/2018-December/001091.html20:01
johnsomTime to start thinking about sessions we would like to propose20:01
xgermanlogging ;-)20:01
xgermanflavors ;-)20:01
johnsomI hope to be there, so I am willing to help with presentations as well.20:02
nmagneziWhat not to log.. :D20:02
*** PagliaccisCloud has quit IRC20:02
xgerman+120:02
johnsomWe should still have the option for project update and on boarding sessions.20:03
xgermanyeah, I hope qulaity of talks goes up now as it’s co-located with the PTG again20:04
johnsomThe other announcement I have is I am cancelling next weeks meeting as many folks are on holiday.20:04
johnsom+1 for that20:04
nmagneziWeeks?20:04
nmagneziNot just one week?20:04
johnsomOne week20:04
johnsomOctavia weekly IRC meeting is cancelled for December 26th.20:04
nmagnezioh, I read this wrong20:04
nmagneziIgnore me20:04
xgermanyeah, I am back 1/220:04
johnsomI probably wrote it wrong, so all fair20:05
johnsomYeah me too.20:05
johnsomI'm off Friday through the 2nd. But I should be reachable if there is something important.20:05
johnsomAny other announcements this week?20:05
johnsom#topic Brief progress reports / bugs needing review20:06
nmagneziNothing on my end. Mostly focusing on pushing reviews so we can shrink the list one that etherpad20:06
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)"20:06
johnsomYeah, my week has been all about reviews, fixing patches, rebasing, and gate work.20:07
johnsomThanks folks for working on the patch list! I feel like we are making progress20:07
xgermanhttps://review.openstack.org/#/c/624835/ - works ok in devstack now trying to make gates give us amp logs20:07
xgermanplease review20:07
nmagnezixgerman, will do. Please also add it here20:08
nmagnezi#link https://etherpad.openstack.org/p/octavia-priority-reviews20:08
nmagnezippl mostly look there20:08
xgermanyeah, will do once it does the amp logs in the gate20:08
johnsomWe also released stable branch packages this week.20:09
johnsomI also updated OSA to pull those SHAs.20:09
xgerman+100020:09
nmagneziyup20:09
nmagnezi#link https://review.openstack.org/#/c/625144/20:09
nmagneziThanks for that20:09
johnsomNot that I can get an OSA AIO install to work, but that is a different issue.20:10
xgermanwhile you are there: https://review.openstack.org/#/c/603228/20:10
xgermanotherwise it’s a wasteland20:10
johnsomAny other updates this week?20:11
johnsomI am still planning to try to have the base flavors stuff done by the end of the week20:11
xgermanyeah20:11
johnsom#topic Open Discussion20:12
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)"20:12
johnsomOk, any other topics for this week?20:12
xgermandon’t think so but we have seen bizarre errors lately - must be the season20:12
xgermanmostly nova though ;-)20:13
johnsomYeah, I have seen some strange things from a bunch of services recently20:13
*** brtknr has quit IRC20:13
xgerman+120:14
nmagneziWe had strange breakages in our downstream CI, stuff like https://review.openstack.org/#/c/625836/20:14
johnsomI need to figure out what is up with the bionic gate all of sudden. It's dying on an rsyslog package issue that doesn't make any sense.20:14
xgermanyeah, hope we don’t come back from the holidays and everyhting is red20:15
johnsomWell, it's good we are finding things. It means the testing is working.... grin20:15
johnsomOk, anything else today?20:16
nmagneziOne question20:16
johnsomSure20:16
nmagneziAbout the centos gate20:16
nmagneziWhich is currently non voting20:17
nmagneziI'm not up to speed with this, what do we have left to do in order for it to vote again?20:17
johnsomRight as it's broken again20:17
nmagneziyeah I just didn't follow this recently as cgoncalves did20:17
nmagneziBut he's not here at the moment20:17
johnsomYeah, so for us to consider it back to voting, I would like to see a good track record for it passing here: http://zuul.openstack.org/builds?job_name=octavia-v2-dsvm-py2-scenario-centos-720:18
nmagneziack. Will keep an eye on that. thanks!20:19
nmagneziNo further questions20:19
johnsomYeah, I would hope that we would have some code contributors that are supporting that gate. For example figuring out what is wrong with it now.20:20
nmagneziDefinitely the reason I asked this :)20:20
johnsomSadly I don't have cycles to focus there....  Just to much going on20:20
johnsomOk, any other topics?20:21
johnsomOk, thanks folks! enjoy the week without an IRC meeting next week.20:22
johnsom#endmeeting20:22
*** openstack changes topic to "Discussions for Octavia | Stein priority review list: https://etherpad.openstack.org/p/octavia-priority-reviews"20:22
openstackMeeting ended Wed Dec 19 20:22:49 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)20:22
openstackMinutes:        http://eavesdrop.openstack.org/meetings/octavia/2018/octavia.2018-12-19-20.00.html20:22
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/octavia/2018/octavia.2018-12-19-20.00.txt20:22
openstackLog:            http://eavesdrop.openstack.org/meetings/octavia/2018/octavia.2018-12-19-20.00.log.html20:22
*** PagliaccisCloud has joined #openstack-lbaas20:41
*** gcheresh has quit IRC20:48
rm_workoh whoops21:23
johnsomMorning21:44
johnsomgrin21:44
xgerman:-)22:04
xgermananyone knows why this https://review.openstack.org/#/c/624835/10/zuul.d/jobs.yaml  ain’t work - or is enough friends with the infra people that they answer questions ;-)22:05
*** rcernin has joined #openstack-lbaas22:19
lxkongguys, any suggsetion to deal with lbs in immutable status(e.g. pending_update)? I think the lbs will be left in pending_update status when being failed over but suddenly the octavia-worker is restarted or killed.22:30
johnsomYes, if the control plane processes are killed via -9 and not gracefully shutdown you can end up with objects stuck in pending_* states.22:31
johnsomBest option is to modify their state to ERROR in the DB and fire a failover API call22:33
lxkongjohnsom: will gracefully shutdown give any help?22:34
lxkongthe process will wait until the failover is done?22:34
johnsomYes, if you do a graceful shutdown (-15) it will finish open actions prior to shutting down and will not leave things in odd states22:35
openstackgerritMichael Johnson proposed openstack/octavia-tempest-plugin master: Add traffic tests using an IPv6 VIP  https://review.openstack.org/61198022:35
*** rcernin has quit IRC22:36
lxkongjohnsom: ok, thanks for the suggestion22:37
*** rcernin has joined #openstack-lbaas22:37
xgermanand it turns out I am blind: http://logs.openstack.org/35/624835/10/check/octavia-v2-dsvm-scenario/2993dba/ has the amp-logs but we also will need it for the non-legacy jobs.22:53
johnsomYeah, you really shouldn't mess with the legacy jobs. They are only there for v1 api, etc.22:54
xgermanalso #infra says we should labe our jobs legacy and non-legacy to be less ocnfusing22:54
johnsomWell, they are in two completely different repos, so....22:55
xgermanI went with https://review.openstack.org/#/c/612622/16/zuul.d/jobs.yaml — but there was only legacy22:56
johnsomOur jobs are all in tempest-plugin22:56
xgermanwell, I shall, make change there, too, then22:57
*** rcernin has quit IRC22:57
*** rcernin has joined #openstack-lbaas22:58
*** rcernin has quit IRC22:59
openstackgerritGerman Eichberger proposed openstack/octavia-tempest-plugin master: Save amphora logs in gate  https://review.openstack.org/62640623:02
xgerman^^ and here it is23:02
*** rcernin has joined #openstack-lbaas23:07
*** PagliaccisCloud has quit IRC23:16
*** rcernin has quit IRC23:38
*** rcernin has joined #openstack-lbaas23:41

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!