| *** madhu_ak has joined #openstack-lbaas | 00:03 | |
| *** madhu_ak_ has joined #openstack-lbaas | 00:06 | |
| xgerman | so dougwigs lb create is spawning an amphora... | 00:07 |
|---|---|---|
| johnsom_ | Cool. I have stack going again | 00:08 |
| *** madhu_ak has quit IRC | 00:09 | |
| xgerman | my stack’s neywork is hosed | 00:12 |
| *** mestery has quit IRC | 00:14 | |
| rm_work | T_T | 00:19 |
| *** madhu_ak_ has quit IRC | 00:48 | |
| *** apuimedo has quit IRC | 00:53 | |
| *** sbalukoff has quit IRC | 01:16 | |
| *** fnaval has quit IRC | 01:21 | |
| *** sbalukoff has joined #openstack-lbaas | 01:28 | |
| *** mixos has joined #openstack-lbaas | 01:41 | |
| *** mixos has quit IRC | 01:56 | |
| *** mixos has joined #openstack-lbaas | 02:05 | |
| *** mixos is now known as mixos-away | 02:05 | |
| *** mixos-away is now known as mixos | 02:06 | |
| *** clev has joined #openstack-lbaas | 02:12 | |
| openstackgerrit | Brandon Logan proposed stackforge/octavia: Fixes health monitor create issues https://review.openstack.org/183379 | 02:14 |
| openstackgerrit | Brandon Logan proposed stackforge/octavia: Adding sudo permissions to SSH Driver commands https://review.openstack.org/180336 | 02:22 |
| blogan | rm_work: ping | 02:27 |
| blogan | dougwig: ping | 02:29 |
| blogan | lol xgerman: ping | 02:30 |
| blogan | johnsom_ping | 02:30 |
| dougwig | blogan: super ack | 02:30 |
| blogan | everyone who said something ot me earlyie: ping | 02:30 |
| blogan | dougwig: you can review that too, but i'm not sure what adam was talking about | 02:30 |
| dougwig | blogan: the added line 288, shouldn't that have a sudo/ | 02:32 |
| dougwig | ? | 02:32 |
| blogan | self._execute_command does that, by default it runs as root | 02:33 |
| blogan | err adds sudo | 02:33 |
| dougwig | ok. odd default, though. | 02:34 |
| blogan | lol i just realized that | 02:34 |
| blogan | respin | 02:34 |
| blogan | of course most of these will be run as root | 02:35 |
| openstackgerrit | Brandon Logan proposed stackforge/octavia: Adding sudo permissions to SSH Driver commands https://review.openstack.org/180336 | 02:43 |
| blogan | dougwig; ^^ | 02:43 |
| dougwig | Sec, Mac updating. I waited so long it's like 9 patches | 02:48 |
| xgerman | k, will check right now | 02:55 |
| xgerman | will leave +A to dougwig | 02:56 |
| xgerman | ping me if you need me again ;-) | 02:57 |
| *** ajmiller has joined #openstack-lbaas | 02:59 | |
| openstackgerrit | Merged stackforge/octavia: Adding sudo permissions to SSH Driver commands https://review.openstack.org/180336 | 03:01 |
| *** mixos has quit IRC | 03:27 | |
| *** Aish has joined #openstack-lbaas | 03:43 | |
| *** fnaval has joined #openstack-lbaas | 03:46 | |
| *** HenryG has quit IRC | 03:51 | |
| *** clev is now known as clev-away | 04:03 | |
| *** clev-away is now known as clev | 04:03 | |
| openstackgerrit | Phillip Toohill proposed stackforge/octavia: Preparing for tempest testing https://review.openstack.org/172199 | 04:54 |
| openstackgerrit | Phillip Toohill proposed stackforge/octavia: Adds scenario test https://review.openstack.org/182554 | 04:55 |
| *** clev is now known as clev-away | 05:12 | |
| *** blogan_ has joined #openstack-lbaas | 05:19 | |
| *** clev-away is now known as clev | 05:20 | |
| *** clev is now known as clev-away | 05:21 | |
| *** blogan_ has quit IRC | 05:28 | |
| *** clev-away is now known as clev | 05:31 | |
| *** HenryG has joined #openstack-lbaas | 05:38 | |
| *** blogan_ has joined #openstack-lbaas | 05:43 | |
| *** mestery has joined #openstack-lbaas | 05:44 | |
| *** clev is now known as clev-away | 05:47 | |
| *** mestery has quit IRC | 05:53 | |
| *** mestery has joined #openstack-lbaas | 05:53 | |
| *** mestery has quit IRC | 06:13 | |
| *** Aish has quit IRC | 06:34 | |
| *** blogan_ has quit IRC | 06:43 | |
| *** jschwarz has joined #openstack-lbaas | 06:45 | |
| *** jschwarz has quit IRC | 06:50 | |
| *** Guest19895 has quit IRC | 07:21 | |
| *** chlong has quit IRC | 07:32 | |
| *** apuimedo has joined #openstack-lbaas | 08:09 | |
| *** woodster_ has quit IRC | 08:10 | |
| *** fnaval has quit IRC | 08:47 | |
| *** bedis_ is now known as bedis | 08:54 | |
| openstackgerrit | Anand Shanmugam proposed openstack/neutron-lbaas: Adding code to prevent vip port deletion from port api https://review.openstack.org/176016 | 10:59 |
| *** rdekel has joined #openstack-lbaas | 12:47 | |
| *** rm_you| has joined #openstack-lbaas | 12:48 | |
| *** rm_you has quit IRC | 12:51 | |
| *** rdekel has quit IRC | 12:55 | |
| *** rm_you|wtf has joined #openstack-lbaas | 13:13 | |
| *** rm_you| has quit IRC | 13:16 | |
| *** woodster_ has joined #openstack-lbaas | 13:41 | |
| *** apuimedo has quit IRC | 13:53 | |
| *** amotoki has joined #openstack-lbaas | 14:16 | |
| *** clev-away is now known as clev | 15:03 | |
| *** Aish has joined #openstack-lbaas | 15:22 | |
| *** Aish has quit IRC | 15:22 | |
| *** Aish has joined #openstack-lbaas | 15:28 | |
| *** mlavalle has joined #openstack-lbaas | 15:40 | |
| *** Aish has left #openstack-lbaas | 15:45 | |
| *** apuimedo has joined #openstack-lbaas | 15:49 | |
| *** mixos has joined #openstack-lbaas | 15:51 | |
| mixos | https://review.openstack.org/#/c/181609/ <== need one more core review. :- ) | 15:56 |
| *** shakamunyi has quit IRC | 16:00 | |
| *** barra204 has quit IRC | 16:00 | |
| *** madhu_ak has joined #openstack-lbaas | 16:03 | |
| *** sbalukoff has quit IRC | 16:06 | |
| johnsom_ | blogan So I am stuck with network issues at the moment. If I set VIP network as public I can't talk to the amp any longer, if I use internal net I have to use a namespace to access, if I use LB network the secondary IP doesn't come up in the amp so haproxy won't bind. | 16:10 |
| johnsom_ | My next attempt is going to be setting up another custom network and try that as the vip network | 16:10 |
| *** mlavalle has quit IRC | 16:12 | |
| mixos | @johnsom_ is it possible to use private IP for VIP and create router connecting your private net and pub net. | 16:12 |
| mixos | ? | 16:12 |
| mixos | then use floating IP assignment to your VIP ? | 16:13 |
| johnsom_ | Yeah, floats is an option, we just aren't setting up floats in the code yet, so would prefer to have a cleaner solution. It might come down to setting up floats manually for now | 16:14 |
| *** hitalia has joined #openstack-lbaas | 16:27 | |
| johnsom_ | It looks like to me the amp doesn't have a route back | 16:27 |
| johnsom_ | It's router is missing a route or port | 16:28 |
| mixos | In openstack, I was not able to assign pub IP to a VM directly. it always require VM to have private IP so that I can associate FIP to it. | 16:37 |
| mixos | If I want to connect pub IP(FIP) , a router was required. maybe Octavia works different way.. | 16:38 |
| *** apuimedo has quit IRC | 16:43 | |
| *** hitalia has quit IRC | 16:45 | |
| blogan | johnsom_: back | 16:48 |
| blogan | johnsom_: i haven't run into the issue of the amp not being able to bind | 16:48 |
| blogan | mixos: thats what i've run into as well, but some deployments we shoudl be able to assign pub IP | 16:50 |
| blogan | but as far as I've known the way to get a public ip in a typical devstack is to point a floatingip to a non-public network | 16:50 |
| mixos | blogan: yes. | 16:51 |
| johnsom_ | So I setup another network using the same script that sets up the lb-mgmt lan. Still the amp isn't responding on it. I can see the request come in with tcpdump, just not seeing the response | 16:54 |
| mixos | @johnsin_ unless your vm setup routing rule between two different network, two private network would not know how to talk to each other. A router is still required. In the router, you may be able to add interfaces of all of private networks. If VIP, LB VM and pool(+backend) in same network, you would not have this issue. I guess. | 16:59 |
| mixos | johnsin_ --> johnsom_ | 17:00 |
| johnsom_ | Yeah, I added a port on the router | 17:01 |
| dougwig | johnsom_: for demo, you could launch a browser inside the namespace. | 17:03 |
| johnsom_ | Yeah, that is the fall back if I can't crack this issue today | 17:03 |
| *** hitalia has joined #openstack-lbaas | 17:12 | |
| *** Guest77307 has joined #openstack-lbaas | 17:17 | |
| *** sbalukoff has joined #openstack-lbaas | 17:19 | |
| *** Guest77307 is now known as redrobot | 17:20 | |
| *** hitalia has quit IRC | 17:24 | |
| *** redrobot has left #openstack-lbaas | 17:26 | |
| *** redrobot has joined #openstack-lbaas | 17:26 | |
| *** mestery has joined #openstack-lbaas | 17:26 | |
| *** redrobot has quit IRC | 17:29 | |
| *** amotoki has quit IRC | 17:33 | |
| *** redrobot has joined #openstack-lbaas | 17:36 | |
| blogan | johnsom_: amp isn't responding to vip>? | 17:36 |
| *** redrobot is now known as Guest63193 | 17:36 | |
| blogan | argh bbiab | 17:36 |
| johnsom_ | Yeah, I don't see the answer going out the default route interface | 17:37 |
| *** Guest63193 is now known as redrobot | 17:39 | |
| johnsom_ | What I don't get is the lb-mgmt net works but lb-vip doesn't even though I used the same setup script. | 17:42 |
| *** barclaac has joined #openstack-lbaas | 17:46 | |
| *** bharath has joined #openstack-lbaas | 17:55 | |
| *** openstackgerrit has quit IRC | 18:22 | |
| *** openstackgerrit has joined #openstack-lbaas | 18:23 | |
| *** shakamunyi has joined #openstack-lbaas | 18:37 | |
| *** barra204 has joined #openstack-lbaas | 18:37 | |
| *** mixos is now known as mixos-away | 18:44 | |
| *** mixos-away is now known as mixos | 18:57 | |
| johnsom_ | I am pretty sure it is a routing issue as if the default route is changed from 192.168.0.1 to 10.1.1.1 (my vip network) the vip ips start working but the lb-mgmt ip stops working | 19:03 |
| johnsom_ | Anyway, I have scrambled my devstack networking trying things, so restacking | 19:04 |
| ptoohill | so having both the routes breaks it? | 19:07 |
| ptoohill | the devstack plugin inserts that route for the mgmt net | 19:08 |
| ptoohill | I cant even stack, not sure whats going on today :/ | 19:08 |
| blogan | johnsom_: an alternative i may investigate is to drop the allowed address pairs implementation and just expose the amphora's port's ip on that network, so instead of the vip being a different ip it'll be teh same ip that the amphora's is on the vip network | 19:16 |
| *** hitalia has joined #openstack-lbaas | 19:26 | |
| johnsom_ | Not sure that will change things. I started up a simple web server listening on all IPs and neither of the VIP addresses responded | 19:36 |
| *** Varun_Lodaya has joined #openstack-lbaas | 19:40 | |
| *** apuimedo has joined #openstack-lbaas | 19:47 | |
| *** barra204 has quit IRC | 19:48 | |
| *** shakamunyi has quit IRC | 19:48 | |
| openstackgerrit | German Eichberger proposed stackforge/octavia: Implements the haproxy amphora agent api server https://review.openstack.org/160034 | 19:54 |
| blogan | johnsom_: yeah pretty sure its a routing issue on the amphora | 19:55 |
| blogan | johnsom_: err maybe not | 19:56 |
| xgerman | pick one | 19:57 |
| *** mlavalle has joined #openstack-lbaas | 20:06 | |
| *** mixos is now known as mixos-away | 20:22 | |
| *** mixos-away is now known as mixos | 20:39 | |
| *** mlavalle has quit IRC | 20:47 | |
| openstackgerrit | German Eichberger proposed stackforge/octavia: haproxy reference amphora REST API client https://review.openstack.org/171172 | 20:51 |
| xgerman | dougwig: how do I fix this: | 21:31 |
| xgerman | https://www.irccloud.com/pastebin/jSd6uHBg | 21:31 |
| blogan | johnsom_: figured the issue out | 21:35 |
| blogan | johnsom_: not sure there is a way around it | 21:35 |
| johnsom_ | Oh... | 21:35 |
| johnsom_ | I was thinking about doing a bridge interface onto a vip network. Haven't had a chance to try it yet though. | 21:36 |
| xgerman | blogan - I am curious… so what’s the root cause | 21:36 |
| blogan | the default gw on the amphora goes through the ssh interface, so when we make requests over the customer data network it attempts to reply back through that ssh interface | 21:39 |
| xgerman | mmh, but we bind the harpy to the vip interface? | 21:39 |
| blogan | xgerman: we do but any requests going to that interface will be replied back on to the mgmt interface | 21:41 |
| xgerman | ok,mmh, this is confusing me… | 21:43 |
| blogan | so if you change the default route in the amphora to go over the vip network, then the requests will work, but you won't be able to ssh into the instance from the global/default namespace | 21:44 |
| blogan | kind of a either or thing | 21:44 |
| xgerman | ok, there must be a third way | 21:44 |
| blogan | well there's ways to get around it | 21:45 |
| xgerman | static route? | 21:46 |
| xgerman | since we know the ips of the controller? | 21:46 |
| blogan | well the problem is that we are trying to ssh over a different netowrk than we are tryign to make http requests over it | 21:47 |
| blogan | but from teh same ip | 21:47 |
| blogan | so in my setup, from the default namesapce, i'm ssh'in ginto 192.168.0.4, which is the ip of the interface on the amphora | 21:48 |
| xgerman | gotcha - | 21:48 |
| blogan | the amphora's default gw is 192.168.0.1 over the .4 interface, which works | 21:48 |
| blogan | when i curl 10.0.0.5 (the lb's vip address), it gets to the 10.0.0.5 interface on the amphora, but replies back on the 192.168.0.4 interface bc of the default route | 21:49 |
| blogan | the ssh and curl requests are both coming from the same ip 172.24.4.1 | 21:49 |
| xgerman | mmh | 21:51 |
| blogan | so when teh amphora sees it needs to respond back to the 172.24.4.1 ip, it ALWAYS goes over the mgmt interface and gw, even if it is an http request | 21:51 |
| blogan | which then has the wrong source ip | 21:51 |
| xgerman | makes sense | 21:51 |
| blogan | for an http request | 21:51 |
| blogan | so you can test this out by logging into the amphora and changing the default route to go through the vip network, and then curl works | 21:52 |
| blogan | but then you have to ssh to the amphora through the router namespace or the dhcp namespace | 21:53 |
| xgerman | maybe our mgmt network needs to be in a namespace | 21:53 |
| blogan | anyway this definitely needs some more discussion | 21:53 |
| xgerman | ? | 21:53 |
| xgerman | yep | 21:53 |
| blogan | xgerman: thats somethign i was thinking about, we could create an interface in the global namespace with a different ip and bridge that to another interface in a namespace and then that would solve it | 21:54 |
| blogan | but this is more of a how do we deploy octavia than an octavia creating the correct routes problem | 21:55 |
| blogan | well we may have to think about the routes problem too | 21:55 |
| xgerman | agreed | 21:55 |
| blogan | anyway i feel much better knowing what it is now | 21:55 |
| ptoohill | so would we be updating drivers to change route in amp? | 21:55 |
| blogan | sad that its not a simple solution | 21:55 |
| ptoohill | maybe temp? | 21:56 |
| ptoohill | well | 21:56 |
| xgerman | nah, I think we need to think that more throigh | 21:56 |
| blogan | ptoohill: if we change the default route int he driver then we won't be able to ssh into the amphora unless we go through namespace | 21:56 |
| ptoohill | yea | 21:56 |
| ptoohill | Im just thinking how to (hack if needed) to get my tests working | 21:56 |
| blogan | going through namespace requires knowledge of what routers/dhcp namespaces exist | 21:56 |
| blogan | good thing there's a summit to talk about these things | 21:57 |
| xgerman | I am thinking the same | 21:57 |
| ptoohill | how do you plan to do it for demo? | 21:57 |
| xgerman | fake it | 21:57 |
| blogan | just curl over a namespace | 21:57 |
| xgerman | it’s all video — so we can use special effects | 21:57 |
| ptoohill | fair enough | 21:57 |
| ptoohill | lens flare | 21:57 |
| ptoohill | LOTS of lens flare | 21:57 |
| xgerman | :-) | 21:58 |
| ptoohill | are you sure theres no way with flows to manipulate this? thats what ovs is for | 21:58 |
| ptoohill | had same type of scenario during my ryu testing | 21:59 |
| ptoohill | though, i manually built the flows and not quite sure how to do the same here | 21:59 |
| xgerman | yeah, you can use flows since we are on different ports (unless somebody needs an lb on 22) | 22:00 |
| xgerman | or 8443 on REST | 22:00 |
| blogan | we would need ovs running on the amphora | 22:00 |
| blogan | which i dont want to do | 22:00 |
| ptoohill | ah | 22:00 |
| ptoohill | :/ | 22:00 |
| blogan | plus its not a problemw ith octavia in my mind | 22:01 |
| ptoohill | what is it then? | 22:01 |
| ptoohill | neutron? | 22:02 |
| ptoohill | networking in general? | 22:02 |
| xgerman | devstack | 22:02 |
| ptoohill | well bummer | 22:04 |
| blogan | ah there is a way to do this without iptables, ip rules | 22:09 |
| blogan | can basically say any traffic you receive on this interface, go back out that interface | 22:09 |
| blogan | that same interface | 22:09 |
| ptoohill | nice | 22:09 |
| blogan | it is some setup though | 22:09 |
| blogan | this is probably somethign we want anyway | 22:12 |
| xgerman | +1 | 22:14 |
| *** mixos has quit IRC | 22:24 | |
| *** mixos has joined #openstack-lbaas | 22:27 | |
| blogan | i'll try to get a patch out tonight, if i can get this working | 22:27 |
| johnsom_ | I am about ot try hacking up a bring into a vip network | 22:29 |
| *** madhu_ak has quit IRC | 22:29 | |
| *** apuimedo has quit IRC | 22:34 | |
| openstackgerrit | German Eichberger proposed stackforge/octavia: haproxy reference amphora REST API client https://review.openstack.org/171172 | 22:40 |
| dougwig | xgerman: you have to submit a patch against openstack/requirements | 22:42 |
| xgerman | I was afraid of that | 22:43 |
| xgerman | ok, will do | 22:43 |
| *** clev is now known as clev-away | 22:51 | |
| *** mixos has quit IRC | 23:09 | |
| rm_work | reviewing all the discussion today and it seems to be primarily networking mumbo-jumbo O_o | 23:10 |
| *** mlavalle has joined #openstack-lbaas | 23:15 | |
| xgerman | https://review.openstack.org/183729 | 23:19 |
| xgerman | dougwig ^^ | 23:19 |
| *** clev-away is now known as clev | 23:21 | |
| *** hitalia has quit IRC | 23:23 | |
| *** hitalia has joined #openstack-lbaas | 23:26 | |
| openstackgerrit | min wang proposed stackforge/octavia: Continue fixing Octavia complexity issues https://review.openstack.org/183732 | 23:32 |
| *** clev is now known as clev-away | 23:39 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!