Thursday, 2020-07-16

« Wednesday, 2020-07-15 Index Friday, 2020-07-17 »
*** rcernin has quit IRC00:06
*** rcernin has joined #openstack-lbaas00:13
*** yamamoto has quit IRC00:20
*** yamamoto has joined #openstack-lbaas00:38
*** rouk has joined #openstack-lbaas00:40
*** wuchunyang has quit IRC00:58
*** shtepanie has quit IRC01:00
*** yamamoto has quit IRC01:06
*** yamamoto has joined #openstack-lbaas01:08
openstackgerritMichael Johnson proposed openstack/octavia master: Update amphora v2 for the failover refactor  https://review.opendev.org/73829201:21
*** tkajinam has quit IRC01:55
*** tkajinam has joined #openstack-lbaas01:55
*** sapd__x has joined #openstack-lbaas02:39
*** psachin has joined #openstack-lbaas03:39
*** armax has quit IRC03:45
*** armax has joined #openstack-lbaas04:07
*** gcheresh has joined #openstack-lbaas05:46
*** wuchunyang has joined #openstack-lbaas05:52
*** wuchunyang has quit IRC06:17
*** wuchunyang has joined #openstack-lbaas06:36
*** maciejjozefczyk has joined #openstack-lbaas07:01
*** born2bake has joined #openstack-lbaas07:12
*** ramishra has quit IRC07:17
*** wuchunyang has quit IRC07:20
*** ataraday_ has joined #openstack-lbaas07:27
*** ramishra has joined #openstack-lbaas07:38
openstackgerritAdam Harwell proposed openstack/octavia master: Use routed network filter if it exists  https://review.opendev.org/70615307:40
*** rcernin has quit IRC07:48
*** rcernin has joined #openstack-lbaas08:14
*** stingrayza has joined #openstack-lbaas08:23
*** happyhemant has joined #openstack-lbaas08:23
*** also_stingrayza has quit IRC08:24
*** ataraday_ has quit IRC09:04
*** rcernin has quit IRC09:07
*** ataraday_ has joined #openstack-lbaas09:48
*** dmellado has quit IRC09:59
*** dmellado has joined #openstack-lbaas10:07
openstackgerritCarlos Goncalves proposed openstack/octavia stable/train: Remove install DIB from Git option  https://review.opendev.org/74142110:12
openstackgerritCarlos Goncalves proposed openstack/octavia stable/ussuri: Migrate grenade job to native Zuul v3  https://review.opendev.org/72674110:14
*** rcernin has joined #openstack-lbaas10:15
openstackgerritAdam Harwell proposed openstack/octavia master: Refactoring amphora stats driver interface  https://review.opendev.org/73711110:15
rm_workjohnsom: ^^ if you have a minute, new review on that, there's some stuff i am uncertain about and could use your thoughts10:16
openstackgerritAdam Harwell proposed openstack/octavia master: Refactoring amphora stats driver interface  https://review.opendev.org/73711110:17
openstackgerritAdam Harwell proposed openstack/octavia master: Refactoring amphora stats driver interface  https://review.opendev.org/73711110:22
openstackgerritAdam Harwell proposed openstack/octavia master: Refactoring amphora stats driver interface  https://review.opendev.org/73711110:29
rm_worklol sorry spam, keep forgetting things10:29
cgoncalvesnever apologize for working hard ;)10:29
rm_workwell I feel bad for hammering zuul too :D10:30
rm_worki just keep typing git-review too fast lol10:30
rm_work"oh wait, one more thing" "oh crap, forgot a release note" heh10:30
*** rcernin has quit IRC10:30
rm_workcould use your thoughts on that too, i just kinda guessed wildly where to put the driver interface10:31
rm_workhopefully the new locations make sense10:31
cgoncalvesI've admittedly been neglecting this RFE. I think I get the goal but better review the code10:35
rm_workwell this is step one which is... almost a no-op10:37
rm_workit's making the driver interfaces make more sense10:37
rm_worknext we can write a driver for some common format like maybe graphite format?10:39
rm_workand have it just output to a configured endpoint10:39
rm_workone question I have is whether we will provide an option for each LB to let the user specify a routable stats endpoint or something so we could export the metrics for each LB or something10:40
rm_workbecause we still need to expose this stuff somehow10:40
cgoncalvesI thought stats drivers would be responsible for exporting, for example, to Logstash?10:44
cgoncalvess/Logstash/Prometheus/10:46
*** wuchunyang has joined #openstack-lbaas10:50
rm_workyes10:51
rm_workexactly10:51
rm_work(BTW Prometheus supports Graphite format)10:51
rm_workso "a driver for some common format like maybe graphite format" + "have it just output to a configured endpoint" == support for Prometheus export :D10:52
cgoncalvesup to drivers to do whichever data handling/transformations they want10:54
*** wuchunyang has quit IRC10:55
*** wuchunyang has joined #openstack-lbaas10:55
openstackgerritAdam Harwell proposed openstack/octavia master: Failover stop threshold  https://review.opendev.org/65681110:55
rm_workjust arebase10:55
rm_workcgoncalves: right, so i'm saying the next step for us (me and anushka and stephanie) to do is write the driver i'm describing :)10:56
*** wuchunyang has quit IRC10:56
cgoncalvesrm_work, yep. I'm just saying I don't think having an option to let tenant users specify a routable stats endpoint is something we should be offering10:59
rm_workhmm11:04
rm_workso then the metrics are for administrators only?11:04
rm_workor the cloud operator should be expected to somehow stand up a user-accessible Prometheus/other system?11:04
rm_workwould you say that the API is still the only way for USERS to retrieve these metrics? and if so, what is the point, admins aren't going to really care about most of this...11:05
rm_workwhat we want to expose is something like what AWS ELB exposes to users, and allow for essentially something like CloudWatch capability11:06
cgoncalvesyes to the latter. additionally, couldn't we expose stats via the REST API? we have stats APIs already. sorry if this was discussed in previous meetings and I missed it11:06
rm_workI mean, we do expose stats via the rest API11:06
rm_workthe ones from the DB11:06
rm_workwhich are not time-series11:06
rm_workwhich means it is very awkward11:07
cgoncalvesrm_work, do you have a link to AWS ELB handy?11:07
rm_workhttps://i.imgur.com/qcWWQYZ.png11:08
rm_workthat's what they expose11:08
rm_workour metrics are similar (though we are missing a couple things11:08
rm_work)11:08
rm_workah i didn't take a screenshot of the main dashboard... hmm11:09
cgoncalvesIMO auto-scaling capabilities should be up to e.g. Senlin where Senlin11:09
cgoncalves... where Senlin would consume data from a stats driver11:09
rm_workhttps://www.bluematador.com/blog/how-to-monitor-amazon-elastic-load-balancing-with-cloudwatch maybe11:10
cgoncalvesI mean, not directly from a stats driver but from somewhere where the driver publishes the data to like a message bus11:10
rm_workyeah like monasca11:10
cgoncalvesI guess11:10
rm_workwhich would probably be a driver we'd want11:10
rm_work`TargetResponseTime` is one we want to add11:10
rm_workwhich comes from haproxy as `rtime`11:11
rm_workaverage member response time11:11
rm_workbut i think i'm talking more about like, monasca/ceilometer + aodh11:12
cgoncalvesrm_work, off-topic: https://review.opendev.org/#/c/741147/11:12
rm_workooo11:12
rm_worki like that, as we run cent8 amps :D11:12
cgoncalvesthanks for +W11:13
*** sapd__x has quit IRC11:28
rm_workany more to review?11:35
rm_workI can review11:35
cgoncalvesrm_work, https://etherpad.opendev.org/p/octavia-priority-reviews11:38
cgoncalvesthere are a bunch of trivial ones, don't be alarmed :)11:38
* rm_work is alarmed11:39
*** yamamoto has quit IRC11:49
*** wuchunyang has joined #openstack-lbaas12:06
rm_workrocky gates still broken?12:07
cgoncalvesrm_work, yes12:09
*** wuchunyang has quit IRC12:10
*** takamatsu has joined #openstack-lbaas12:11
*** happyhemant has quit IRC12:12
rm_workcgoncalves: https://review.opendev.org/#/c/738797/12:20
*** maciejjozefczyk has quit IRC12:21
*** rcernin has joined #openstack-lbaas12:21
*** maciejjozefczyk has joined #openstack-lbaas12:21
rm_workand question on Use retry for AmphoraComputeConnectivityWait -- https://review.opendev.org/#/c/739771/12:24
cgoncalvesrm_work, stable/stein octavia-v2-dsvm-scenario is Bionic: https://zuul.opendev.org/t/openstack/build/b742896f223542d88add4872ab4815d3/log/zuul-info/zuul-info.controller.txt#1012:26
cgoncalvesrm_work, patch AmphoraComputeConnectivityWait was to resolve merge conflict with the refactor failover flow backport12:27
rm_workok12:27
rm_workcgoncalves: ahh it's about the controller, not the amp <_< k12:27
cgoncalvesoh, you mean the amphora is xenial. that's a bug IMO12:27
rm_workmy mind went to amp12:27
rm_worksince i was just looking at the cent8 one12:27
rm_workwait12:27
rm_workis the cent8 one for the amp or the controller system <_<12:27
*** sapd__x has joined #openstack-lbaas12:28
cgoncalvesrm_work, centos8 is both controller and amp12:28
rm_workok cool12:28
*** yamamoto has joined #openstack-lbaas12:29
rm_worklots of +2s12:30
rm_workeither ya'll are just writing good patches, or I'm going soft >_>12:30
cgoncalvesI doubt it's the former xD12:31
*** yamamoto has quit IRC12:55
*** spatel has joined #openstack-lbaas13:04
openstackgerritMerged openstack/octavia master: Add non-voting octavia-v2-dsvm-scenario-centos-8  https://review.opendev.org/74114713:05
cgoncalvesCI jobs are ridiculously fast now \o/13:09
cgoncalvesoctavia-grenade isn't on nest-virt nodeset yet though13:10
openstackgerritMerged openstack/octavia-tempest-plugin master: Set test_basic_traffic as smoke test  https://review.opendev.org/73383213:14
rm_work:/13:24
rm_workgotta do that :D13:24
*** rcernin has quit IRC13:31
*** TrevorV has joined #openstack-lbaas13:32
*** sapd__x has quit IRC13:34
openstackgerritCarlos Goncalves proposed openstack/octavia master: Use octavia nodeset in octavia-grenade  https://review.opendev.org/74145113:47
openstackgerritBrian Haley proposed openstack/octavia-lib master: Update test-requirements to match Octavia repo  https://review.opendev.org/74145513:51
haleybrm_work: thanks for all the +2's, but seem we're caught in a pep8 failure now, that ^^ fixed it locally13:52
rm_workerg k13:55
rm_worki was wondering what was up with that13:55
rm_workon the lib side13:55
haleybit was either isort or pylint14:00
*** ianychoi has joined #openstack-lbaas14:05
*** ramishra has quit IRC14:10
*** ramishra has joined #openstack-lbaas14:12
*** gcheresh has quit IRC14:12
*** TrevorV has quit IRC14:15
openstackgerritBrian Haley proposed openstack/octavia-lib master: Update test-requirements to match Octavia repo  https://review.opendev.org/74145514:16
*** t0rrant has joined #openstack-lbaas14:24
openstackgerritMerged openstack/octavia-tempest-plugin master: Set image driver to noop in noop jobs  https://review.opendev.org/73801514:26
*** TrevorV has joined #openstack-lbaas14:46
*** tkajinam has quit IRC14:47
*** sapd__x has joined #openstack-lbaas14:55
*** yamamoto has joined #openstack-lbaas14:55
*** devfaz has quit IRC14:55
*** irclogbot_0 has quit IRC14:55
*** devfaz has joined #openstack-lbaas14:57
*** irclogbot_0 has joined #openstack-lbaas14:57
*** mloza has joined #openstack-lbaas14:59
openstackgerritMerged openstack/octavia-lib master: Update test-requirements to match Octavia repo  https://review.opendev.org/74145515:01
*** spatel has quit IRC15:04
johnsomFYI, the OpenStack 10 year zoom call is on now: https://etherpad.opendev.org/p/tTP9ilsAaJ2E8vMnm6uV15:08
*** ataraday_ has quit IRC15:14
*** johnthetubaguy has quit IRC15:16
*** johnthetubaguy has joined #openstack-lbaas15:20
openstackgerritCarlos Goncalves proposed openstack/octavia-lib master: Add NotFound exception class  https://review.opendev.org/73807515:21
*** maciejjozefczyk has quit IRC15:23
*** yamamoto has quit IRC15:30
openstackgerritMerged openstack/octavia-lib master: Add NotFound exception class  https://review.opendev.org/73807515:52
openstackgerritCarlos Goncalves proposed openstack/octavia-lib master: Switch to victoria jobs  https://review.opendev.org/73954215:56
johnsomFYI, Mark just said the October summit is going to be virtual16:04
*** sapd__x has quit IRC16:05
*** psachin has quit IRC16:12
*** johnthetubaguy has quit IRC16:17
*** johnthetubaguy has joined #openstack-lbaas16:20
*** sapd__x has joined #openstack-lbaas16:24
rm_workahh thanks16:52
rm_workwas busy making breakfast and missed it T_T RIP16:52
rm_workjohnsom: any comments on https://review.opendev.org/#/c/737111/ for stuff I can fix before I fix that stupid single pep8 error? lol16:54
johnsomHaven't looked at that in a bit. I can take a quick pass here soon-ish16:56
rm_workkk i'll just make another patchset because why not16:58
openstackgerritAdam Harwell proposed openstack/octavia master: Refactoring amphora stats driver interface  https://review.opendev.org/73711116:58
cgoncalvesOK, octavia-grenade down to 1h02 with nested virt. blazing fast, too, considering it runs devstack twice, runs nova, neutron, glance, etc smoke and octavia scenario tests17:01
rm_workjohnsom:  do you remember the reason why we switched everything to the os.fdopen stuff like on the left side of https://review.opendev.org/#/c/740815/1..7/octavia/amphorae/backends/health_daemon/health_daemon.py ?17:03
rm_workgthiemonge told them to switch to just open() but I think that's not right17:04
johnsomrm_work Greg asked me that this week. I have no idea why you would do that on a read. We do it on a write as you can set/maintain the permissions.17:04
rm_workAH, ok so on reads it's fine17:04
rm_workthat was me just cargo-cult copy/pasting that from other stuff because i remembered at some point we went through and switched a bunch out17:05
johnsomYeah, on read I can't think of any difference between just open RDONLY and this os.fdopen thing17:05
rm_workk17:05
johnsomWrite matters though17:05
rm_workyeah17:06
rm_workdo you think the simple file-based persistence mechanism there should be good?17:06
rm_workI was kinda curious if oslo_config does writebacks to the config file17:06
rm_workif so that'd be cute to just put the "last values" into CONF :D17:06
rm_workbut i don't think it does...17:07
johnsomfile based persistence?17:07
rm_workit's the thing that review was doing17:10
rm_workfor deltas17:10
johnsomI guess I have not read that far yet17:11
johnsomrm_work Ok, read through that patch what file based persistence?17:19
rm_workwe store off the current counters to a file, every time we calculate the deltas17:19
johnsomWhere?17:19
rm_workline 24517:20
johnsomNot in 73711117:20
rm_workno, the one i linked about the file thing17:20
rm_work74081517:20
johnsomYou asked me to look at 73711117:20
rm_workheh i was mentioning that as i had just linked it and you'd commented17:20
rm_workso i figured you had it open :D17:20
rm_workno worries17:21
rm_worki'll try not to be distracting lol17:21
johnsomOk, looking at this one17:21
johnsomWhy would we need to persist it? Why can't it just be in memory?17:21
rm_workif the agent reloads itself17:23
rm_workit'd lose the "last known value"17:23
rm_workif we only keep it in memory17:23
rm_workso then the first delta would be ... the entire counter17:23
johnsomYeah, that is true17:24
*** yamamoto has joined #openstack-lbaas17:27
johnsomrm_work What about if the agent sees >0 it starts the deltas from the current number? Yeah, you would drop a few bytes, but then there is no file I/O17:29
rm_workwell17:30
rm_workyeah i guess the agent would ALWAYS be started before haproxy :D17:31
rm_workbut this is only one write every 10s?17:31
rm_workdo you think it'll be an issue?17:31
johnsomIt's not huge for sure, just thinking if it can be avoided17:32
johnsomI am commenting that you should not open/close all of the time.17:32
johnsomBut, yeah, I guess I wasn't thinking about 10 sec intervals. Though that is configurable.17:33
johnsomGave you some comments17:37
johnsomrm_work BTW: https://review.opendev.org/#/c/737191/17:40
rm_workyeah i guess this is one case where ... leaving open the file handle isn't ... bad17:43
rm_workyep looks good17:48
rm_work+W17:48
johnsomThanks.17:50
rm_workyou said "No code here to read this on startup"??17:52
rm_workwhat is that referring to17:52
johnsomYou are only writing to the file, there is nothing that ever reads it17:52
rm_workget_counters()17:52
rm_workcalled from calculate_stats_deltas()17:53
johnsomWhy not only read it on startup?17:54
rm_workit does17:55
rm_workonly once17:55
rm_workit's a singleton17:55
johnsomSo, wait, you are comparing to the original value on every loop? the singleton value?17:57
rm_work?17:58
rm_workit loads up that file once, the first time17:59
rm_workthen it continually updates the dict17:59
rm_workand stores it off in case of crash17:59
johnsomMaybe I am not on my game this morning, but I don't see it setting counters back with the new value18:03
*** yamamoto has quit IRC18:04
rm_work14818:06
rm_workyou commented18:06
rm_workit's ... yes and no18:06
johnsomYeah, you get it, then set it back to the value you just got18:06
rm_workthat's a no-op if it's already there, and if it hit the case where it's anew listener, it sets that {} from the get() default18:07
rm_workthen on 15618:07
johnsomOh, I see18:08
rm_workwe could do some other logic like "if listener_id not in counters: counters[listener_id] = listener_counters" but I think that check is basically the same number of cycles lol18:08
rm_workyeah, 99% of the time it's a no-op18:08
* rm_work shrugs18:08
*** sapd__x has quit IRC18:11
openstackgerritAnushka Singh proposed openstack/octavia master: Change amphora statistics to use deltas  https://review.opendev.org/74081518:22
openstackgerritMerged openstack/octavia-tempest-plugin master: Fix availability zone API tests  https://review.opendev.org/73719118:46
*** andrein has quit IRC19:14
*** nicolasbock has quit IRC19:14
*** andrein has joined #openstack-lbaas19:15
*** nicolasbock has joined #openstack-lbaas19:15
*** t0rrant has quit IRC19:27
*** mchlumsky0 has joined #openstack-lbaas19:35
*** mchlumsky has quit IRC19:37
*** mchlumsky0 is now known as mchlumsky19:37
openstackgerritMerged openstack/octavia master: Fix listener update with SNI certificates  https://review.opendev.org/71279019:56
*** shtepanie has joined #openstack-lbaas20:01
*** yamamoto has joined #openstack-lbaas20:03
*** vesper11 has quit IRC20:20
*** vesper11 has joined #openstack-lbaas20:21
*** TrevorV has quit IRC20:27
openstackgerritAdam Harwell proposed openstack/octavia master: Refactoring amphora stats driver interface  https://review.opendev.org/73711120:35
*** yamamoto has quit IRC20:43
*** wuchunyang has joined #openstack-lbaas21:01
*** wuchunyang has quit IRC21:06
openstackgerritBrian Haley proposed openstack/octavia master: Remove Neutron SDN-specific code  https://review.opendev.org/71819221:23
*** yamamoto has joined #openstack-lbaas21:38
openstackgerritBrian Haley proposed openstack/octavia master: Remove Neutron SDN-specific code  https://review.opendev.org/71819222:42
*** rcernin has joined #openstack-lbaas22:43
*** rcernin has quit IRC22:47
*** rcernin has joined #openstack-lbaas22:54
*** tkajinam has joined #openstack-lbaas23:02
*** wuchunyang has joined #openstack-lbaas23:08
*** wuchunyang has quit IRC23:12
*** devfaz has quit IRC23:16
*** irclogbot_0 has quit IRC23:16
*** devfaz has joined #openstack-lbaas23:22
*** irclogbot_0 has joined #openstack-lbaas23:22
openstackgerritAdam Harwell proposed openstack/octavia master: Change amphora statistics to use deltas  https://review.opendev.org/74081523:33
« Wednesday, 2020-07-15 Index Friday, 2020-07-17 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!