Tuesday, 2017-05-30

*** yuanying has quit IRC00:00
*** yuanying has joined #openstack-lbaas00:01
*** cpuga has joined #openstack-lbaas00:18
*** sanfern has quit IRC00:47
*** yamamoto_ has joined #openstack-lbaas01:19
*** sanfern has joined #openstack-lbaas01:41
*** leitan has joined #openstack-lbaas01:55
*** leitan_ has joined #openstack-lbaas01:58
*** leitan has quit IRC02:01
*** ipsecguy_ has joined #openstack-lbaas02:04
*** ipsecguy has quit IRC02:06
*** leitan has joined #openstack-lbaas02:30
*** leitan__ has joined #openstack-lbaas02:31
*** leitan_ has quit IRC02:34
*** leitan has quit IRC02:35
*** yamamoto_ has quit IRC03:06
*** cpuga has quit IRC03:06
*** cpuga has joined #openstack-lbaas03:07
*** gans has joined #openstack-lbaas03:07
*** links has joined #openstack-lbaas03:09
*** cpuga has quit IRC03:11
*** sanfern has quit IRC03:13
*** sanfern has joined #openstack-lbaas03:14
*** sanfern has quit IRC03:14
*** cpuga has joined #openstack-lbaas03:19
*** armax has joined #openstack-lbaas03:23
gansnmagnezi, your code is working good (y)03:26
*** links has quit IRC03:32
*** cpuga has quit IRC03:48
*** cpuga has joined #openstack-lbaas03:48
*** links has joined #openstack-lbaas03:49
*** cpuga has quit IRC03:55
*** redrobot has quit IRC03:56
*** leitan__ has quit IRC03:56
*** leitan has joined #openstack-lbaas03:57
*** leitan has quit IRC03:57
*** redrobot has joined #openstack-lbaas03:57
*** redrobot is now known as Guest7234503:57
*** yamamoto_ has joined #openstack-lbaas03:58
*** aojea has joined #openstack-lbaas03:59
*** Dinesh_Bhor has quit IRC04:02
*** aojea has quit IRC04:04
*** dayou has quit IRC04:08
*** krypto has joined #openstack-lbaas04:14
*** amotoki is now known as amotoki_away04:15
*** dayou has joined #openstack-lbaas04:16
*** amotoki_away is now known as amotoki04:17
*** gcheresh has joined #openstack-lbaas04:20
*** amotoki is now known as amotoki_away04:30
*** dineshbhor has joined #openstack-lbaas04:33
*** amotoki_away is now known as amotoki04:33
*** dineshbhor has quit IRC04:37
*** Dinesh_Bhor has joined #openstack-lbaas04:37
*** JudeC has joined #openstack-lbaas04:38
JudeCAnyone around?04:38
*** sanfern has joined #openstack-lbaas04:40
*** cpuga has joined #openstack-lbaas04:51
*** cpuga has quit IRC05:00
*** amotoki is now known as amotoki_away05:02
*** amotoki_away is now known as amotoki05:10
*** armax has quit IRC05:22
*** cpuga has joined #openstack-lbaas05:23
*** aojea has joined #openstack-lbaas05:39
*** cpuga has quit IRC05:43
*** krypto has quit IRC05:44
*** cpuga_ has joined #openstack-lbaas05:45
*** gcheresh has quit IRC05:52
*** aojea has quit IRC05:54
*** aojea has joined #openstack-lbaas05:54
*** aojea has quit IRC05:58
*** Guest72345 has quit IRC06:04
*** JudeC has quit IRC06:10
*** rcernin has joined #openstack-lbaas06:14
nmagnezigans, good to know :-)06:27
nmagnezixgerman, rm_work, johnsom ^06:27
*** cpuga_ has quit IRC06:28
*** cpuga has joined #openstack-lbaas06:28
*** cpuga has quit IRC06:33
*** gcheresh has joined #openstack-lbaas06:51
*** krypto has joined #openstack-lbaas07:04
*** tesseract has joined #openstack-lbaas07:08
rm_worknmagnezi: yeah I saw07:10
rm_workI'll look deeper in the morning07:10
*** aojea has joined #openstack-lbaas07:31
*** pcaruana has joined #openstack-lbaas07:33
*** krypto has quit IRC07:34
rm_worknmagnezi: i know the interface naming is a bit wonky but it SHOULD work with the old-style interface names set, which we do07:36
rm_worki'm unclear as to why it would have broken now (though I guess it obviously DID?)07:37
*** pcaruana has quit IRC07:39
*** pcaruana has joined #openstack-lbaas07:39
openstackgerritAdam Harwell proposed openstack/octavia master: Pool name/desc needs to be "" when empty, not null  https://review.openstack.org/46778008:11
*** openstackgerrit has quit IRC08:18
*** gans819 has joined #openstack-lbaas08:31
*** gans has quit IRC08:34
*** gans819 has quit IRC08:49
*** gans819 has joined #openstack-lbaas08:55
*** gans819 has quit IRC09:23
*** todin has quit IRC09:27
*** jmccrory has quit IRC09:27
*** afranc has quit IRC09:27
*** dayou has quit IRC09:27
*** links has quit IRC09:27
*** nmagnezi has quit IRC09:27
*** jidar has quit IRC09:27
*** raginbajin has quit IRC09:27
*** todin has joined #openstack-lbaas09:49
*** jmccrory has joined #openstack-lbaas09:49
*** afranc has joined #openstack-lbaas09:49
*** dayou has joined #openstack-lbaas10:13
*** links has joined #openstack-lbaas10:13
*** nmagnezi has joined #openstack-lbaas10:13
*** jidar has joined #openstack-lbaas10:13
*** raginbajin has joined #openstack-lbaas10:13
*** kbyrne has quit IRC10:17
*** kbyrne has joined #openstack-lbaas10:18
*** yamamoto_ has quit IRC10:31
*** sanfern has quit IRC10:39
*** gans has joined #openstack-lbaas10:52
*** csomerville has joined #openstack-lbaas11:05
*** cody-somerville has quit IRC11:06
*** yamamoto has joined #openstack-lbaas11:12
*** gans has quit IRC11:16
*** aojea has quit IRC11:49
*** chlong has joined #openstack-lbaas11:58
*** chlong has quit IRC12:01
*** links has quit IRC12:10
*** atoth has quit IRC12:23
*** mestery has quit IRC12:24
*** mestery has joined #openstack-lbaas12:27
*** atoth has joined #openstack-lbaas12:38
*** atoth has quit IRC12:39
*** atoth has joined #openstack-lbaas12:40
*** leitan has joined #openstack-lbaas12:51
*** chlong has joined #openstack-lbaas13:08
*** gans has joined #openstack-lbaas13:17
*** gcheresh has quit IRC13:35
*** aojea has joined #openstack-lbaas13:43
*** armax has joined #openstack-lbaas13:47
*** cpuga has joined #openstack-lbaas13:48
*** cpuga_ has joined #openstack-lbaas13:49
*** cpuga has quit IRC13:53
leitanHi guys, im working with octavia now, i have created a vxlan network on a tenant where i created my loadbalancer, i tested the LB from inside the namespace and everything is working ok, but when i go to associate a floating ip , to publish de loadbalancer into my network, the LB port does not appear, cause the ports are created with neutron/service user i guess, how its supposed this to be handled in a normal scenario ?14:00
leitanshould i suppose to create the LB directly into the public network ??? (i dont find this friendly due to ip usage)14:03
leitanlet me known when you can jump in, rm_work nmagnezi johnsom xgerman :) thanks14:04
*** reedip_ has joined #openstack-lbaas14:18
*** gans has quit IRC14:24
leitanignore this, i was missing the router to the external network from the loadbalancer network, still not working but im taking a look at it14:30
*** sanfern has joined #openstack-lbaas14:34
*** gcheresh has joined #openstack-lbaas14:51
*** rcernin has quit IRC15:09
*** armax has quit IRC15:12
*** armax has joined #openstack-lbaas15:14
*** gcheresh has quit IRC15:22
*** catintheroof has joined #openstack-lbaas15:25
*** catintheroof has quit IRC15:25
*** catintheroof has joined #openstack-lbaas15:26
*** yamamoto has quit IRC15:28
*** yamamoto has joined #openstack-lbaas15:28
*** yamamoto has quit IRC15:33
*** gans has joined #openstack-lbaas15:50
*** cody-somerville has joined #openstack-lbaas15:50
*** cody-somerville has quit IRC15:50
*** cody-somerville has joined #openstack-lbaas15:50
*** csomerville has quit IRC15:53
*** sanfern has quit IRC15:58
*** aojea has quit IRC15:58
*** sanfern has joined #openstack-lbaas15:59
*** aojea has joined #openstack-lbaas15:59
*** gans has quit IRC16:01
*** tesseract has quit IRC16:03
*** aojea has quit IRC16:03
*** gans has joined #openstack-lbaas16:07
*** gans has quit IRC16:07
*** catintheroof has quit IRC16:08
*** catintheroof has joined #openstack-lbaas16:09
*** armax has quit IRC16:32
*** yamamoto has joined #openstack-lbaas16:35
*** armax has joined #openstack-lbaas16:36
*** JudeC has joined #openstack-lbaas16:39
*** pcaruana has quit IRC16:43
*** reedip_ has quit IRC16:46
*** yamamoto has quit IRC16:47
*** cpuga_ has quit IRC17:08
*** cpuga has joined #openstack-lbaas17:09
*** harlowja has joined #openstack-lbaas17:14
*** sshank has joined #openstack-lbaas17:19
rm_workleitan: are you in a real cloud or just devstack?17:19
leitanrm_work: real cloud17:19
rm_worki assume a real cloud17:19
rm_workk17:19
rm_workyeah, so i did run into one issue where for some reason associating a FLIP wasn't actually bringing the interface up17:19
rm_workbut i think it was because our internal networking is a little odd17:20
rm_workit's supposed to automatically create an interface for it in the netns17:20
leitanrm_work: i debugged this a little17:20
leitanwhen i list the neutron port17:20
leitanand list the floating ips17:20
rm_workyeah, debug mode for the amp agent is a good idea, and looking at those logs17:20
leitani see that its actually asociated with the vip port17:20
rm_workjohnsom: BTW, o/ welcome back17:20
rm_workleitan: that's correct17:21
leitanbut, theres no nat rule on any of the router namespace17:21
leitanwhere the amphorae instances are running17:21
johnsomo/  Still digging out from being gone17:21
rm_workah, yeah we don't touch the routers17:21
rm_workwe assume the networking layer is all routable correctly17:21
rm_workjohnsom: ok when you get done with that, i have a list of prioritized reviews for you17:22
rm_work;)17:22
leitanrm_work: ok, so then the LB_NETWORK where i create the loadbalancers, CANT be a isolated vxlan and then associate a floating ip to the vip17:22
leitanthat is not going to work17:22
rm_workhmmm17:23
leitani assume that is going to work just like the agent mode, where you can do that17:23
leitanbuy that doesnt use allowed address pairs17:23
leitanso maybe theres a thing there17:23
leitani dont know17:23
johnsomWhy would it not work???  (other than DVR mode bugs in neutron)17:23
leitandont know, its not working for me17:24
leitanjust debugging17:24
leitaneverything worked perfectly, but associating a floating ip to the private isolated vip is not working17:24
johnsomMake sure your router is not set for "distributed" (DVR), if it is, there are known bugs in the DVR code for floating IPs.17:24
leitanall my routers are DVR on the setup17:24
leitandvr_snat and dvr17:25
rm_work:(17:25
johnsomYeah, that is the problem then.  With DVR routers the floating IPs don't pass traffic17:25
johnsomIt's a known bug in DVR17:25
leitana bug with DVR + allowed address pairs ?17:25
johnsomYes17:25
leitancrap17:26
rm_workyes. >_>17:26
rm_workcrap is approximately correct17:26
johnsomI think Kevin was putting some effort into getting that finally fixed in this release, but I'm not up to date on their efforts17:26
johnsomThat bug has been open since like mitaka17:26
rm_workkevinbenton is looking into it for us? :P maybe we are saved17:26
leitanok, ill do this, ill create the LB on the public network, to se if it works, allocating the ips on a flat space17:26
rm_worktrust in the kevinbenton17:26
johnsomYeah, Kevin is the man.17:27
leitanjust to test everything, that should work, because it worked correctyle on the vlxan network17:27
johnsomYeah, other thing to test, is set it up on a non dvr router17:27
leitanjohnsom: yes but i dont have available agents to schdule a non dvr17:27
leitani can set then a business rule to enforce that public LBs should be created on public net and consume 3 ips17:28
* leitan cries while the PM slap him in the face17:28
*** catintheroof has quit IRC17:29
leitanin kevinbenton we trust then17:30
johnsomhttps://bugs.launchpad.net/neutron/+bug/158369417:30
openstackLaunchpad bug 1583694 in neutron "[RFE] DVR support for Allowed_address_pair port that are bound to multiple ACTIVE VM ports" [Wishlist,In progress] - Assigned to Swaminathan Vasudevan (swaminathan-vasudevan)17:30
leitanin the meantime, ill try it that way17:30
leitansumming up some "it affects me"17:31
johnsomThis is the last bug I know was tracking the exact issue.17:31
johnsomThere may be others17:31
leitanthanks guys you saved me from a lot of unnecesary debugging johnsom rm_work17:34
johnsomSorry the answer is "it's broke"17:34
leitanjohnsom: hey, no worries, i know how it goes been deploying Openstack since Cactus release17:35
johnsomOuch17:35
rm_workyeah that sounds spiky17:36
rm_workjohnsom: so you think you'll be done catching up by sometime in July? :P17:37
johnsomHa, well, I still need to catch up on my OpenStack-dev e-mail, so....17:38
johnsomItching to look into this interface thing.  We SET that name, so puzzled what the issue is17:38
johnsomrm_work do you have an etherpad list for me?17:40
rm_workjohnsom: well that's the #1 priority anyway17:42
johnsomI'm not real excited about the 3GB thing either.  It's a penalty for the other OS's.17:43
rm_workonly in devstack17:43
rm_workanyone making their own image should be able to do whatever17:43
johnsomYeah17:43
rm_worki dunno17:44
johnsomI wonder what the difference is in gate test time...17:44
rm_worki'm kinda on the fence but17:44
rm_worki feel like in general we tend towards LCD17:44
johnsomI lean toward encouraging the cloud images to be smaller...  Leaving it default 2gb, but printing warnings it RHEL is selected, making the gates use 3gb if it is RHEL.  (though our centos gate isn't failing...)17:46
johnsomI mean, in reality we can't use RHEL in the gates, so...17:46
*** leitan has quit IRC17:51
*** yamamoto has joined #openstack-lbaas17:53
*** aojea has joined #openstack-lbaas17:53
sanfernDo we have tempest scenario test cases and Rally test cases to test Octavia ? Any document around it17:54
johnsomRally, no.17:54
johnsomTempest yes, but they are old and poor.  I have created a repository to start a "reboot" of the tempest tests for Octavia17:54
johnsomhttps://github.com/openstack/octavia-tempest-plugin17:55
johnsomWhat we currently have for tempest test for Octavia are:17:55
johnsom1. The neutron-lbaas tests, since they use the Octavia driver: https://github.com/openstack/neutron-lbaas/tree/master/neutron_lbaas/tests/tempest17:56
johnsom2. The Octavia tempest tests, v1 API: https://github.com/openstack/octavia/tree/master/octavia/tests/tempest17:56
sanfernok thanks johnsom, i will look into that and run those in our environment17:57
johnsom3. some patches for V2 API, but still the old bad code: https://review.openstack.org/#/c/413697 and https://review.openstack.org/#/c/42982517:57
sanfernok17:57
johnsomThe patches need some work I think.17:58
sanfernjohnsom, Are we planning to add Rally test cases?17:58
*** JudeC has quit IRC17:59
johnsomThere has been debate on the value.  What are you looking for out of Rally tests?17:59
*** JudeC has joined #openstack-lbaas17:59
sanferncreating 100 LBs and collect the VIP response time18:00
johnsomOk18:01
*** leitan has joined #openstack-lbaas18:01
johnsomI'm open to adding Rally tests in addition to tempest.  We just need to understand the value and what we can run on the OpenStack gate hosts.18:02
sanfernI want to benchmark results whenever we bringup the new cloud18:03
*** blogan has joined #openstack-lbaas18:12
*** openstackgerrit has joined #openstack-lbaas18:17
openstackgerritMerged openstack/octavia master: Update example config to have more correct keystone_authtoken example  https://review.openstack.org/46577218:17
openstackgerritOpenStack Proposal Bot proposed openstack/octavia master: Updated from global requirements  https://review.openstack.org/46747318:20
*** gcheresh has joined #openstack-lbaas18:23
rm_workjohnsom: sooo with https://review.openstack.org/#/c/468963/ i don't understand WHY it fixes the gate but it does?18:27
rm_workjohnsom: i can say, we are broken ATM for some reason18:27
johnsomYeah, I'm stacking now to take a look at it.18:28
johnsomSomething just is odd since we assign those names as we add interfaces to the namespace.18:28
*** armax has quit IRC18:37
leitanGuys is there any quick api call to know wich amphorae is the master without checking the vrrp config in the instance ?18:43
johnsomNo, not currently.  As it turns out keepalived makes this a bit hard to identify who is currently the master and it's logging is sometimes not accurate.18:44
johnsomIf it is a fresh pair you can query the database18:45
leitanits a fresh pair indeed18:45
johnsomBut if there has been a failover we currently don't have a way to tell.  It18:45
johnsomIt is in the amp info API, but we found we didn't have a good way to get reliable info.18:45
leitansince from the controllers you have trusted ssh access to the amphorae what about some kind of polling from the healthmanager that actually parse the ip command to get wich one is advertising the vip ?18:48
leitanand refreshing that in a field on the loadbalancer details18:49
johnsomWell, the agent would just do that, no need for remote query18:49
*** sanfern has quit IRC18:49
leitanyes youre right, but the "ip method" sounds too awful ?18:50
johnsomThe issue is we also have another little bug, where at some points, the IP is listed on the interface even though it's not the active master.  Since it doesn't impact functionality it has been super low on the fix list18:50
leitanjohnsom: i see18:50
leitanmaybe checking the access_log from haproxy, if its growing in a timelapse of X minutes, its the active18:51
*** sshank has quit IRC18:58
leitanor on the keepalived template that is built, an script that actualy leaves a true or false in a path when for example is seeing logs on the access of the balancer18:59
leitansorry, just thinking about options :)19:00
*** yamamoto has quit IRC19:00
*** yamamoto has joined #openstack-lbaas19:05
*** leitan has quit IRC19:13
*** leitan has joined #openstack-lbaas19:15
*** yamamoto has quit IRC19:25
*** JudeC has quit IRC19:32
*** ianychoi_ has joined #openstack-lbaas19:36
*** ianychoi has quit IRC19:38
*** armax has joined #openstack-lbaas20:02
*** JudeC has joined #openstack-lbaas20:24
*** yamamoto has joined #openstack-lbaas20:26
*** yamamoto has quit IRC20:34
*** PagliaccisCloud has joined #openstack-lbaas20:39
*** cody-somerville has quit IRC20:42
*** cody-somerville has joined #openstack-lbaas20:45
*** cody-somerville has quit IRC20:45
*** cody-somerville has joined #openstack-lbaas20:45
*** gcheresh has quit IRC20:46
*** atoth has quit IRC20:50
*** atoth has joined #openstack-lbaas21:02
*** leitan has quit IRC21:04
*** KeithMnemonic has joined #openstack-lbaas21:13
johnsomUgh, I keep getting an image from the 10th, which runs fine.  too many cached versions.  Cleaned up now, trying again21:13
*** cpuga has quit IRC21:14
*** aojea has quit IRC21:17
*** sshank has joined #openstack-lbaas21:28
*** cpuga has joined #openstack-lbaas21:49
*** sshank has quit IRC21:50
*** leitan has joined #openstack-lbaas22:07
openstackgerritJude Cross proposed openstack/octavia master: Add filtering and field selection to API  https://review.openstack.org/46927522:11
*** JudeC has quit IRC22:26
*** JudeC has joined #openstack-lbaas22:45
rm_workjohnsom: how goes that...22:52
johnsomWell, if I build an image with the May 10th 875cb2db03f4a5551f823e82de3afd18566bdf53 commit the agent is fine.22:53
rm_workyeah k22:53
rm_workhmm22:53
rm_workso it's not something upstream22:53
rm_workbut something internally?22:54
rm_workexcept22:54
johnsomI see that the interface moved into the namespace has the wrong name22:54
rm_workhow would that have merged22:54
johnsomI am thinking it's maybe a pyroute2 issue?22:54
johnsomStill digging22:54
johnsomIt's just odd22:55
rm_workand why would nmagnezi's fix work22:56
johnsomWell, I think he is preserving the interface name from outside the namespace to inside, but I think it will have problems22:57
rm_worki'm actually unclear as to why it had to be done the way you did it22:57
rm_workand not the way he's doing it22:57
johnsomThis is why I forced the names inside the namespace22:57
rm_workk well, once we get this fixed, it'd be nice to merge ... err... basically the whole client22:58
rm_workand then a bunch of my bugfixes22:58
johnsomWell, for one thing think about this scenario:22:58
johnsomamp has eth1 (mgmt) and eth2 (VIP)  We move eth2 into netns as eth2.  World happy.22:59
johnsomThen we add member network, which when hotplugged comes in as eth2 since in the default namespace there is no longer an eth2.  Now migrate that into the netns and you have a conflict.23:00
johnsomBasically the numbering changes based on what is plugged and unplugged in what order23:00
johnsomSo, I chose to "manage" the naming in the netns23:01
rm_workah, k23:01
rm_workdoes your code actually solve for that either though?23:02
rm_worki only saw one constant for naming that23:02
rm_workotherwise it seems the same?23:02
rm_workanywho, if you make a patch that makes the gate work, i'll +2 it :)23:02
johnsomYes, the VIP is always first and doesn't come/go.  So VIP is always eth1 in my naming/numbering scheme.  I.e. the OS could call that port em2000 and I will always name it eth1 in the netns23:03
rm_workbut then if you add another member network23:03
rm_workcouldn't it also get em2000, and then it'd be moved into the netns as ???23:04
rm_workand even if it gets moved as em200023:04
rm_workdoesn't the second member network suffer from the same issue?23:04
johnsomThey are moved in as eth2, 3, 4, 5 etc.  All tracked by MAC23:04
rm_workah23:04
johnsom /var/lib/octavia/plugged_interfaces23:04
rm_worki need to reread the code i guess, didn't catch that23:04
rm_workah i guess he didn't touch that in his patch23:04
*** sshank has joined #openstack-lbaas23:23
openstackgerritJude Cross proposed openstack/octavia master: Add filtering and field selection to API  https://review.openstack.org/46927523:24
*** leitan has quit IRC23:31
*** leitan has joined #openstack-lbaas23:36
johnsomYeah, it is definitely an issue related to pyroute2.  The command line still works, but I can reproduce the bug via pyroute223:41
cpugajohnsom: is tomorrow's meeting still schedule to occur?23:43
johnsomYes, we will have a meeting tomorrw23:43
cpugawould it be possible to add "flavors" as a topic for tomorrow23:43
johnsomSure, NP23:44
cpugak, thx23:44
rm_workoh i need to add a topic...23:47
rm_workor two...23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!