Thursday, 2017-08-24

*** JudeC has quit IRC00:02
*** leitan has joined #openstack-lbaas00:08
*** rcernin has quit IRC00:12
*** yuanying_ has joined #openstack-lbaas00:58
*** yuanying has quit IRC01:02
*** yuanying has joined #openstack-lbaas01:02
*** yuanying_ has quit IRC01:02
*** sticker has joined #openstack-lbaas01:05
*** JudeC has joined #openstack-lbaas01:08
*** JudeC has quit IRC01:19
*** JudeC has joined #openstack-lbaas01:30
*** dayou has quit IRC01:41
*** yuanying has quit IRC01:50
*** dougwig has quit IRC02:00
*** yuanying has joined #openstack-lbaas02:01
*** dayou has joined #openstack-lbaas02:16
*** leitan has quit IRC02:39
*** leitan has joined #openstack-lbaas02:39
*** leitan has quit IRC02:44
*** fnaval has quit IRC02:48
*** links has joined #openstack-lbaas03:18
*** JudeC has quit IRC03:27
*** sanfern has joined #openstack-lbaas03:42
*** JudeC has joined #openstack-lbaas04:09
*** sanfern has quit IRC04:11
*** sanfern has joined #openstack-lbaas04:12
*** sanfern has quit IRC05:09
*** sanfern has joined #openstack-lbaas05:10
*** rcernin has joined #openstack-lbaas05:11
*** rcernin has quit IRC05:27
*** armax has quit IRC05:28
openstackgerritOpenStack Proposal Bot proposed openstack/octavia master: Updated from global requirements  https://review.openstack.org/49605205:56
*** rcernin has joined #openstack-lbaas06:12
*** eezhova has joined #openstack-lbaas06:20
*** pcaruana has joined #openstack-lbaas06:21
openstackgerritOmer Anson proposed openstack/octavia master: devstack: Allow 3rd party neutron backends deployment  https://review.openstack.org/49620506:25
*** slaweq has joined #openstack-lbaas06:32
*** aojea has joined #openstack-lbaas06:39
*** gcheresh_ has joined #openstack-lbaas06:55
*** eezhova has quit IRC07:07
*** aojea has quit IRC07:10
*** tesseract has joined #openstack-lbaas07:20
*** eezhova has joined #openstack-lbaas07:47
*** tesseract-RH has joined #openstack-lbaas07:49
*** tesseract-RH has quit IRC07:50
*** tesseract-RH has joined #openstack-lbaas07:50
*** tesseract has quit IRC07:52
*** tesseract-RH has quit IRC08:01
*** tesseract-RH has joined #openstack-lbaas08:02
*** yamamoto has joined #openstack-lbaas08:10
*** aojea has joined #openstack-lbaas08:22
*** gcheresh_ has quit IRC08:24
*** gcheresh has joined #openstack-lbaas08:26
*** reedip has quit IRC08:31
openstackgerritJude Cross proposed openstack/octavia-tempest-plugin master: Create scenario tests for members  https://review.openstack.org/49713908:35
*** JudeC has quit IRC08:48
*** aojea has quit IRC08:50
*** slaweq has quit IRC09:11
*** slaweq has joined #openstack-lbaas09:12
*** gcheresh has quit IRC09:13
*** yamamoto has quit IRC09:16
*** slaweq has quit IRC09:16
*** yamamoto has joined #openstack-lbaas09:17
*** gcheresh has joined #openstack-lbaas09:17
*** tesseract-RH has quit IRC09:18
*** tesseract has joined #openstack-lbaas09:23
*** yamamoto has quit IRC09:27
*** yamamoto has joined #openstack-lbaas09:29
*** yamamoto has quit IRC09:30
*** aojea has joined #openstack-lbaas09:39
*** reedip has joined #openstack-lbaas09:51
*** slaweq has joined #openstack-lbaas09:59
*** aojea has quit IRC10:04
*** yamamoto has joined #openstack-lbaas10:31
*** yamamoto has quit IRC10:36
*** krypto has joined #openstack-lbaas11:17
*** yamamoto has joined #openstack-lbaas11:33
*** yamamoto has quit IRC11:38
openstackgerritOpenStack Proposal Bot proposed openstack/octavia master: Updated from global requirements  https://review.openstack.org/49605211:44
*** sanfern has quit IRC12:08
*** yamamoto has joined #openstack-lbaas12:34
*** coreycb has joined #openstack-lbaas12:38
*** yamamoto has quit IRC12:41
*** atoth has joined #openstack-lbaas12:47
*** sanfern has joined #openstack-lbaas12:49
*** reedip has quit IRC13:01
*** catintheroof has joined #openstack-lbaas13:02
*** reedip has joined #openstack-lbaas13:13
*** krypto has quit IRC13:24
*** krypto has joined #openstack-lbaas13:24
*** fnaval has joined #openstack-lbaas13:31
*** yamamoto has joined #openstack-lbaas13:36
*** krypto has quit IRC13:38
*** yamamoto has quit IRC13:42
*** leitan has joined #openstack-lbaas13:44
*** krypto has joined #openstack-lbaas13:45
*** links has quit IRC13:46
*** slaweq has quit IRC14:29
*** gcheresh has quit IRC14:29
*** yamamoto has joined #openstack-lbaas14:38
*** armax has joined #openstack-lbaas14:42
*** yamamoto has quit IRC14:44
*** krypto has quit IRC15:01
*** krypto has joined #openstack-lbaas15:01
*** eezhova has quit IRC15:09
*** jniesz has joined #openstack-lbaas15:17
*** sanfern has quit IRC15:26
*** cody-somerville has joined #openstack-lbaas15:31
*** yamamoto has joined #openstack-lbaas15:40
*** yamamoto has quit IRC15:45
*** eezhova has joined #openstack-lbaas15:46
*** amotoki is now known as amotoki__away15:47
*** leitan has quit IRC15:53
*** rcernin has quit IRC15:54
*** pcaruana has quit IRC15:57
*** ianychoi has joined #openstack-lbaas16:00
*** tesseract has quit IRC16:03
*** openstackgerrit has quit IRC16:04
*** leitan has joined #openstack-lbaas16:04
*** links has joined #openstack-lbaas16:06
*** kbyrne has quit IRC16:09
*** krypto has quit IRC16:09
*** kbyrne has joined #openstack-lbaas16:10
*** tongl has joined #openstack-lbaas16:28
*** links has quit IRC16:38
*** yamamoto has joined #openstack-lbaas16:41
*** yamamoto has quit IRC16:47
*** JudeC has joined #openstack-lbaas16:58
*** sanfern has joined #openstack-lbaas17:26
*** rcernin has joined #openstack-lbaas17:27
*** eezhova has quit IRC17:34
*** openstackgerrit has joined #openstack-lbaas17:41
openstackgerritMerged openstack/octavia master: Updated from global requirements  https://review.openstack.org/49605217:41
*** yamamoto has joined #openstack-lbaas17:43
*** catinthe_ has joined #openstack-lbaas17:44
*** catintheroof has quit IRC17:45
*** eezhova has joined #openstack-lbaas17:46
*** eezhova has quit IRC17:47
*** yamamoto has quit IRC17:48
*** eezhova has joined #openstack-lbaas17:48
*** SumitNaiksatam has joined #openstack-lbaas17:56
*** sanfern has quit IRC18:00
*** sanfern has joined #openstack-lbaas18:09
*** dasanind has joined #openstack-lbaas18:32
rm_workjohnsom: FYI I'm going to be working on the API for the maintenance mode thing we discussed a week or so ago18:43
rm_workprobably starting next week18:43
johnsomOk18:43
rm_workdoing it upstream, any suggestions?18:43
rm_workIt's going to be an "operator" command, we decided those should not be under any different namespace or anything18:44
rm_workright? just ... use permissions to lock them down18:44
johnsomRight18:44
rm_workk18:44
rm_workso we should have an entire cycle to figure out how we want to structure those18:44
rm_workprobably I'll bring it up at the PTG18:44
rm_workI'd like to get a number of admin things in this cycle18:44
*** yamamoto has joined #openstack-lbaas18:44
rm_workanother one i'll probably do is a thing to force recycle the spares pool (for when we release a new image)18:45
rm_workthough maybe that could be done via the failover API?18:45
johnsomNo, that would need to be another path.  failover works at LB level18:45
rm_workk18:46
rm_worktoying with `/maintenance` and `/spares/recycle`18:46
rm_workwe might have more spares operations? like, maybe GET for them to show count and readyness18:47
*** yamamoto has quit IRC18:50
johnsomYeah, my initial thought is /amphora but that is too driver specific IMO18:58
rm_workyeah19:05
rm_worki wonder if this *will be* very driver specific though19:06
rm_workto be honest the amount that i've thought it through for the big picture is <_<19:06
rm_worknot a lot19:06
rm_workmaybe it is deserving a spec?19:06
rm_workbut ... due to internal pressures i'm going to be doing code within a short time no matter what :(19:07
rm_workso i might just be looking at doing some rewriting later if people review it and say "nope"19:07
*** eezhova has quit IRC19:13
*** SumitNaiksatam has quit IRC19:15
*** sanfern has quit IRC19:16
*** sanfern has joined #openstack-lbaas19:17
*** sanfern has quit IRC19:25
*** eezhova has joined #openstack-lbaas19:27
*** sanfern has joined #openstack-lbaas19:27
*** gcheresh has joined #openstack-lbaas19:35
openstackgerritJude Cross proposed openstack/octavia-tempest-plugin master: Create scenario tests for members  https://review.openstack.org/49713919:42
*** yamamoto has joined #openstack-lbaas19:46
*** yamamoto has quit IRC19:51
*** yamamoto has joined #openstack-lbaas19:51
*** fnaval has quit IRC19:54
openstackgerritMerged openstack/octavia master: Add a CLI section to the Octavia docs  https://review.openstack.org/49694019:57
*** aojea has joined #openstack-lbaas20:06
*** eezhova has quit IRC20:13
jnieszthoughts on adding an additional configuration option to change the behavior of amp_boot_network_list20:14
jnieszinstead of attaching to all networks in the list20:14
jnieszbalance across amp_boot_network_list based on ip availability20:14
johnsomjniesz So, we have discussed the option to have many routable networks for the lb-mgmt-net.20:14
jnieszour main usecase is for regions that can have 1000+ SLB's20:15
jnieszand network team doesn't like giving out large networks, due to broadcasts and size of L2 domain20:16
johnsomRight, that many ports on a network with neutron can get "interesting"20:16
jnieszoptimally we would add a pool of /24 nets20:16
jnieszwhen one fills up we can add another20:16
jnieszwastes less IPv4 space as well20:16
johnsomYeah, you can always use IPv6 for the mgmt net, but that may not be possible at all deployments20:17
jnieszyea we are limitied in certain networks to IPv420:17
jnieszIPv6 is definitely preference for lb management network20:17
jnieszwhen possible20:17
johnsomOk, so some thoughts.  Neutron has a subnetpool interface, but I'm not super familiar with it.  So I'm not sure if that would help here or not.20:19
johnsomLooks like that might not be flexible enough for our needs20:23
johnsomSo, thinking about this, I think we would need a few things:20:23
jnieszor maybe address scopes20:24
johnsom1. A new config setting for a list of networks that will be selected from for the lb-mgmt-net20:25
johnsom2. Change the flow to select one of these networks to pass to the compute driver.20:26
johnsom3. Update the create flow to be able to retry with an additional network selected from the list should that network be full at boot time.20:27
johnsom4. Maybe do a check during selection as well, but we must handle the boot error case as it could get used between the check and the boot call.20:27
jnieszyea i think doing a check would be good and then if failed retry as well20:28
johnsomrm_work Any thoughts on this?  I know we have talked about it before20:28
johnsomBack when we started neutron had problems with networks having more than 200 ports.  I'm sure that is better now, but was a concern that led to this discussion.20:29
johnsomThe assumption that the networks are routable with the controllers will need to be captured in the docs/release notes too20:30
xgerman_I think a list and selecting at random sounds good20:31
jnieszinstead of random what about doing a neutron net-ip-availability-list20:33
jnieszto avoid trying full network20:33
rm_workintelligence sounds good there20:33
johnsomYeah, that neutron extension might make selection a bit more "balanced'20:34
rm_workthough internally what we do is ... i don't provide a network at all... nova schedules me "one" by default and it is guaranteed to be routable20:34
rm_workand that's that20:34
johnsomYou are using "auto" for nova?20:34
johnsomI think we need to make sure we do a check that the extension is available in neutron, use it if available, use random otherwise.20:35
johnsomWe already have code in the network driver to check for available extensions20:36
johnsomjniesz So it sounds like support for the idea20:38
johnsomThere is kind of a roadmap of thoughts on the how part.20:38
jnieszi thought nova scheduling based on ip availability was coming as part of pike20:39
jnieszi need to check on that20:39
johnsomYeah, I see the "auto" option, but it just grabs a network from the project, probably not ideal.20:40
jnieszyea it would have to select from specific list20:40
johnsomSo to get started, a spec would be good.  I think we laid most of it out.20:42
johnsomIf nova can handle it, that is the best option, then we don't have to deal with the retry code20:44
*** yamamoto has quit IRC20:45
jnieszagreed, that would make coding simpler20:46
*** gcheresh has quit IRC20:47
rm_workjniesz: that's a local patch20:51
rm_workbecause of our network architecture, we can't allow binding of arbitrary networks20:51
rm_workwe actually don't support tenant networks at all20:51
rm_workat no point can i tell nova to bind any network20:51
rm_work:(20:52
rm_workbut it makes this specific thing a non-problem for me20:52
johnsomlol20:52
*** catinthe_ has quit IRC20:53
jnieszthese would be admin networks that only the service tenant could deploy onto20:53
*** catintheroof has joined #openstack-lbaas20:54
johnsomCorrect20:56
jnieszhttp://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/neutron-routed-networks.html20:58
*** catintheroof has quit IRC20:58
*** sanfern has quit IRC20:59
*** sanfern has joined #openstack-lbaas21:02
johnsomHmmm, if that works, that would be a great docs article to add to our admin guide21:04
jnieszplus if somebody is doing pure routed L3 networks21:04
jnieszand there is no L3 across racks21:04
jnieszwould need something like that for the lb management network21:05
jnieszI mean no L2 across racks21:05
rm_workjniesz: that is my case21:05
rm_workno L2 across racks21:05
rm_workwe're pure routed L321:05
jnieszwhat do you do for lb management?21:06
rm_worknova on each cell knows what network it has access to, so it schedules everything on that21:06
rm_workthere are no tenant networks21:06
rm_workso our mgmt network is not strictly "private"21:06
rm_workit is the same as the VIP network21:07
rm_workand we use FLIPs to actually expose LBs21:07
rm_workwhen a user creates a LB, they pass a *FLIP* network-id as "vip_network_id" and we create a FLIP for them and associate it21:07
rm_workjniesz: we use this driver: https://review.openstack.org/#/c/435612/21:09
jnieszinteresting, we are kind of cheating with EVPN VXLAN in ToR21:10
rm_workwhat I need to do is actually make sure that runs properly in devstack and set up devstack to actually use that driver in the tests :P21:10
rm_workjniesz: ok so apparently, that routed-networks thing was essentially born at our request to handle our exact topology, lol21:15
rm_workjniesz: unfortunately we're just too far behind to run it yet <_<21:15
rm_workbut it WILL be what we run, I hope, eventually :321:16
jnieszdo you know if nova actually implemented their part for routed network scheduling in pike?21:16
rm_worki do not21:16
jnieszlooks like neutron part is done21:16
rm_worki was told the nova piece was supposed to land in *newton*21:16
rm_workso21:16
rm_workmaybe it got delayed and delayed and delayed?21:16
openstackgerritMichael Johnson proposed openstack/octavia-dashboard master: Remove neutron-lbaas checks and rename "slug"  https://review.openstack.org/49764221:17
jnieszhttps://blueprints.launchpad.net/neutron/+spec/routed-networks21:17
jnieszI think Pike was target21:17
rm_workhmm k21:17
rm_workhttps://blueprints.launchpad.net/nova/+spec/neutron-routed-networks21:18
rm_work"With https://review.openstack.org/#/c/299591/ being merged I'm considering this partially complete for the Newton release, any remaining work for neutron routed networks should happen in Ocata. -- mriedem 20160802"21:18
*** rcernin has quit IRC21:19
rm_workjniesz: if you have L3 routing I'm curious if you have any interest whatsoever in the FLIP driver I'm working with21:21
rm_work*if you're using L3 routing instead of L221:21
openstackgerritJude Cross proposed openstack/octavia-tempest-plugin master: Create scenario tests for health monitors  https://review.openstack.org/49764321:21
rm_workbut I guess you said you're faking it21:21
rm_work?21:21
jnieszyea, we are using provider networks, but those networks are extended across racks with our ToR21:21
rm_workI am not clear on exactly how "EVPN VXLAN in ToR" makes things work :)21:21
jnieszwe need diffferent security zones21:22
rm_work<-- not a networking person21:22
rm_workyeah we do security zones via AZ21:22
rm_workor rather21:22
rm_worksorry no21:22
rm_workwe have *different clouds* for security zones21:22
rm_worklol21:22
jnieszon our new L3 network we have multiple security zones21:22
jnieszso it is converged21:22
jnieszand the provider network determines the zone21:23
johnsomWow, I really confused gerrit on that last patch.  I think the code is right, but the diff is really scrambled21:23
rm_work>_>21:23
rm_workjohnsom: looks sane to me? https://review.openstack.org/#/c/497642/ ?21:23
johnsomYeah, at the top, look at the file with "C" tag21:24
rm_workoh just the one21:25
jnieszrm_work I'll be at the PTG, we can probably discuss more there21:25
rm_workkk21:25
rm_workjniesz: i wish i was more of a networking person <_<21:25
johnsomSay the load balancer core21:25
rm_worki just find it hard to *care* what happens below a certain layer (which happens to be 7)21:25
jnieszha21:25
rm_workwhich makes it difficult to pick up21:25
jnieszlayer 7 always blames layer 321:26
rm_workwell maybe layer 3 should get its shit together :P21:26
* rm_work shrugs21:26
rm_workjohnsom: yeah that's funny i guess it's less characters to describe copying urls.py with changes to panels.py, than to describe the actual changes to panel.py? <_<21:28
johnsomYeah, I git mv'd the directory and made edits, so...21:28
jnieszhave to sign off for the day21:28
jnieszttyl21:28
johnsomo/21:28
*** jniesz has quit IRC21:31
*** yamamoto has joined #openstack-lbaas21:46
*** aojea has quit IRC21:47
*** aojea has joined #openstack-lbaas21:47
*** eandersson has joined #openstack-lbaas21:50
*** yamamoto has quit IRC21:51
*** aojea has quit IRC21:51
johnsomYeah, that code seems to be working just fine21:59
*** catintheroof has joined #openstack-lbaas21:59
johnsomOne more step towards an octavia-dashboard22:01
*** leitan has quit IRC22:02
*** leitan has joined #openstack-lbaas22:02
*** leitan has quit IRC22:02
*** leitan has joined #openstack-lbaas22:02
*** leitan has quit IRC22:03
*** yamamoto has joined #openstack-lbaas22:47
*** yamamoto has quit IRC22:53
rm_workjohnsom: does our mark_amp_health_busy thing ACTUALLY get a lock?!23:19
rm_worki'm a little confused23:19
rm_workhttps://github.com/openstack/octavia/blob/master/octavia/controller/worker/tasks/database_tasks.py#L7723:20
johnsomYes, it is an auto commit23:24
johnsomThere aren't actions to chain in a transaction so it doesn't need the autocommit=False lock sessions we have used in other places23:25
rm_workhmm k23:30
rm_workso err23:30
rm_workin a sec can you review this and see if i'm doing this locking properly23:31
johnsomSure, NP23:31
rm_workerr if we do a with_for_update() query...23:36
rm_workand multiple processes do it at the same time23:36
rm_workdo they wait?23:36
rm_workor does the DB reject all after the first?23:36
rm_workjohnsom: ^^23:39
*** catintheroof has quit IRC23:46
openstackgerritAdam Harwell proposed openstack/octavia master: WIP: Floating IP Network Driver (spans L3s)  https://review.openstack.org/43561223:47
rm_workjohnsom: https://review.openstack.org/#/c/435612/44..45/octavia/amphorae/drivers/health/flip_utils.py23:47
rm_workThe idea is that all the HMs will receive this packet around the same time, and I want to prevent as many as possible from sending the message23:47
rm_workI think this should block the majority, but if the message is delayed for some and the operation is very short on the one that got it early, i guess it could still run multiple times <_<23:48
johnsomThey should wait for a period of time, then if the lock is there too long the first one wins23:49
rm_workhmm k23:49
rm_workwell, i am not good at this23:49
rm_workso I hope I got it mostly23:49
*** yamamoto has joined #openstack-lbaas23:49
johnsomrm_work Can I pull this down and make some changes?23:53
rm_workif you want :)23:53
*** tongl has quit IRC23:53
johnsomMight be easier than trying to talk you through it23:54
*** yamamoto has quit IRC23:55
rm_worklol yeah i won't argue if you just want to fix it23:55
rm_worklets me zoom with JudeC right now on getting his tempest tests running :P23:55
JudeCo/23:55
rm_workone step closer to having our own proper testing23:56
JudeCStill need someone to take a look at my code and point out all the stuff I did wrong :P.23:56
rm_workI will definitely be looking shortly ^_^23:57
johnsomYeah, I was going to do that today, but dashboard got me sucked in.  So much to fix there23:59
*** yuanying_ has joined #openstack-lbaas23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!