| *** puranamr has joined #akanda | 01:15 | |
| *** puranamr has quit IRC | 01:40 | |
| openstackgerrit | Adam Gandelman proposed stackforge/akanda-appliance: Introduces advanced service drivers to akanda-appliance https://review.openstack.org/226119 | 01:46 |
|---|---|---|
| openstackgerrit | Adam Gandelman proposed stackforge/akanda-rug: Start of the LBAAS driver https://review.openstack.org/225369 | 01:48 |
| *** elo has joined #akanda | 05:10 | |
| *** elo has quit IRC | 05:23 | |
| *** elo has joined #akanda | 05:57 | |
| *** elo has quit IRC | 06:07 | |
| *** elo has joined #akanda | 07:31 | |
| *** elo has quit IRC | 08:42 | |
| openstackgerrit | venkatamahesh proposed stackforge/akanda-rug: Fix the sphinx build path in setup.cfg https://review.openstack.org/226694 | 10:59 |
| *** skamithi13 has joined #akanda | 13:58 | |
| skamithi13 | was listening to a YouTube video on Akanda talking about the data path..it was brief but can the akanda stuff run in a container and not a VM? | 14:00 |
| *** stanchan has joined #akanda | 14:23 | |
| ryanpetrello | adam_g_ any idea if the stable/kilo requirements snafu is resolved? | 14:49 |
| ryanpetrello | or markmcclain | 14:49 |
| markmcclain | skamithi13: yes... with driver base either or | 14:50 |
| markmcclain | ryanpetrello: last I looked no | 14:50 |
| *** puranamr has joined #akanda | 15:35 | |
| *** jordantardif has joined #akanda | 15:41 | |
| *** puranamr has quit IRC | 15:47 | |
| *** puranamr has joined #akanda | 15:50 | |
| ryanpetrello | adam_g markmcclain with a vanilla devstack install | 16:09 |
| ryanpetrello | I can boot a tenant VM | 16:10 |
| ryanpetrello | and it's not reachable on the tenant network from the router VM | 16:10 |
| ryanpetrello | i.e., `ping 192.168.0.2` doesn't wor | 16:10 |
| ryanpetrello | *work | 16:10 |
| ryanpetrello | oddly, if I assign it a floater, I can ping the floater from within the router VM | 16:10 |
| ryanpetrello | no TCP traffic at all from the router to tenants over 192.168 | 16:13 |
| ryanpetrello | adam_g mentioned you'd seen some similar issue recently, markmcclain? | 16:13 |
| *** elo has joined #akanda | 16:14 | |
| davidlenwell | adam_g: I've confirmed that the following config will stand up trunk devstack with lbassv2 and you can tell it to make lb's and watch the event go by.. | 16:38 |
| davidlenwell | https://etherpad.openstack.org/p/lbaasv2-devstack-local.conf | 16:38 |
| *** seanmwinn has joined #akanda | 16:46 | |
| *** puranamr has quit IRC | 16:47 | |
| *** cleverdevil has joined #akanda | 16:56 | |
| *** skamithi13 has quit IRC | 17:27 | |
| *** skamithi13 has joined #akanda | 17:27 | |
| adam_g | ryanpetrello, i havent had time to detangle it yet | 17:33 |
| adam_g | ryanpetrello, (the stable/kilo or the connectivity thing) | 17:33 |
| adam_g | ryanpetrello, re connectivity, took a quick look last night and noticed (like you) the router's pots on that net are not bound | 17:34 |
| davidlenwell | adam_g: ryanpetrello this connectivity thing you've been looking into.. is that only happening on stable/kilo or is that happening in master as well? | 17:43 |
| adam_g | davidlenwell, master as well, the default installation | 17:43 |
| davidlenwell | okay.. just wanted to be clear about what I was trying to reproduce | 17:44 |
| openstackgerrit | venkatamahesh proposed stackforge/akanda: Fix the sphinx build path in .gitignore file https://review.openstack.org/226900 | 17:58 |
| adam_g | davidlenwell, a simple test case is to boot a tenant vm connected to the router, ssh to the router and try pinging the VMs address | 18:02 |
| davidlenwell | k.. thanks for that .. im stacking now | 18:02 |
| adam_g | davidlenwell, one thing i noticed last night is that the rourter VM's port for the interface on that network is makred as DOWN and not bound | 18:03 |
| davidlenwell | hmm... okay .. thanks | 18:03 |
| ryanpetrello | yep, noticed the same | 18:03 |
| adam_g | ive been creating ports for the LBAAS VMs using the same create code we use for the router ports and noticed the same there | 18:03 |
| adam_g | so it may be something todo with that | 18:03 |
| adam_g | https://git.openstack.org/cgit/stackforge/akanda-rug/tree/akanda/rug/api/neutron.py#n377 | 18:05 |
| adam_g | creates the port with no associated fixed IPs | 18:05 |
| adam_g | i wonder if theres something we need to do to actually bind it to the VM, since its not mapping via fixed ip anymore | 18:05 |
| adam_g | ryanpetrello, https://review.openstack.org/226913 | 18:09 |
| adam_g | should pass /w that | 18:09 |
| ryanpetrello | cool | 18:10 |
| ryanpetrello | I'll +2 when it passes | 18:10 |
| *** elo has quit IRC | 18:18 | |
| openstackgerrit | Merged stackforge/akanda-rug: Fix the sphinx build path in setup.cfg https://review.openstack.org/226694 | 18:42 |
| ryanpetrello | doesn't look to me like a security group issue; at least, I can `Q_USE_SECGROUP=False` in local.conf and it doesn't resolve the issue | 18:51 |
| ryanpetrello | yep | 19:01 |
| ryanpetrello | ports in question have binding:vif_type = binding_failed | 19:01 |
| adam_g | ryanpetrello, im also realizing the vm doesnt get any ports created for the routers internal ports | 19:28 |
| adam_g | do you know off the top of your head if the <kilo code does that? | 19:28 |
| ryanpetrello | the vrrp port stuff doesn't really exist before Kilo, does it? | 19:28 |
| ryanpetrello | (you mean what we're currently using?) | 19:28 |
| adam_g | ryanpetrello, yeah, i know the vrrp stuff doesnt. im just trying to grok what (in Nova's POV) the router VM gets in terms of ports | 19:30 |
| ryanpetrello | adam_g: it's a security group issue | 19:37 |
| ryanpetrello | I set `/etc/neutron/plugins/ml2/ml2_conf.ini:firewall_driver = neutron.agent.firewall.NoopFirewallDriver` | 19:37 |
| ryanpetrello | w/ devstack it defaults to `/etc/neutron/plugins/ml2/ml2_conf.ini:firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver` | 19:38 |
| ryanpetrello | changing this causes tenant network traffic to work | 19:38 |
| adam_g | ryanpetrello, hm | 19:38 |
| adam_g | security should be turned off for all the VRRP ports | 19:38 |
| ryanpetrello | yep | 19:38 |
| ryanpetrello | with that change, I can ssh into my tenant VMs now | 19:38 |
| ryanpetrello | doesn't look like it is, then | 19:38 |
| ryanpetrello | at the very least, it's a security group issue | 19:39 |
| adam_g | ryanpetrello, were those VMs able to get dhcp from the router prior to disabling that? | 19:39 |
| ryanpetrello | looks like it's cirros and using config drive | 19:39 |
| adam_g | so adding an intenral pot to the router on create gets me DHCP | 19:39 |
| adam_g | but no traffic after that | 19:39 |
| adam_g | maybe secuirty groups post-DHCP | 19:40 |
| adam_g | oh, nvm | 19:42 |
| ryanpetrello | I'll test this once more | 19:42 |
| adam_g | red herring (that works without the port as well) | 19:42 |
| ryanpetrello | but I'm pretty certain it's a security group issue | 19:42 |
| ryanpetrello | this is the first time I've been able to get it to work | 19:42 |
| ryanpetrello | and was after changing the ml2_conf.ini firewall_driver | 19:42 |
| * adam_g looks that way too | 19:45 | |
| ryanpetrello | yea, I'll test once more w/ a fresh ./stack.sh to confirm | 19:46 |
| * davidlenwell is also restacking with that | 19:46 | |
| ryanpetrello | if that's the case, given what adam_g said "security should be turned off for all the VRRP ports" | 19:48 |
| ryanpetrello | I'm not sure how ^^ works, but it doesn't seem to be true | 19:48 |
| ryanpetrello | at least for the internal ports | 19:48 |
| adam_g | ryanpetrello, https://git.openstack.org/cgit/stackforge/akanda-neutron/tree/akanda/neutron/plugins/ml2_neutron_plugin.py#n87 | 19:49 |
| adam_g | that is clearly working on the MGT ports--without that filter we wouldn't be able to reach the API service there | 19:49 |
| adam_g | that said, enabling the Noop driver doesnt seem to do the trick for me, unless im not restarting the right things after | 19:50 |
| ryanpetrello | okay | 19:50 |
| adam_g | im stil l able to serve DHCP via the internal network, so it does smell firewall-y | 19:50 |
| ryanpetrello | I'll tinker some more and make sure that's it | 19:50 |
| ryanpetrello | are you at least able to reproduce my issue on vanilla kilo? | 19:50 |
| *** jordantardif has quit IRC | 19:52 | |
| adam_g | ryanpetrello, no, i dont have a kilo up now. but im able to reproduce notbeing able to reach a tenant VM from the router via L2 | 19:53 |
| ryanpetrello | k | 19:53 |
| davidlenwell | my last restack was with stable kilo.. and yes I was able to reproduce | 19:54 |
| ryanpetrello | okay | 19:54 |
| adam_g | ryanpetrello, you familiar are you with the juno ports config? | 19:54 |
| adam_g | that made sense.. | 19:54 |
| ryanpetrello | heh | 19:54 |
| davidlenwell | I've been poking at it for an hour or so.. no real answers yet .. but this scroll back between you two has given me some more ideas on things to try | 19:54 |
| ryanpetrello | with our hacky frankenstein one, yes | 19:55 |
| ryanpetrello | though to my knowledge the stable/juno branch you all have is vrrp | 19:55 |
| ryanpetrello | and I didn't think it was any different from what's in stable/kilo | 19:55 |
| adam_g | oh, yah, i guess it is | 19:56 |
| adam_g | my hunch is that this the fact that we bring up the router address on an interface in the VM, with a different mac address than the actual neutron port for that address, is causing traffic to get blocked on anti-spoofing | 19:58 |
| adam_g | in theory adding an allowed address pair for the spoofer's port and the real one should fix that but i havne't been able to | 19:58 |
| ryanpetrello | yep, if I stick a `Q_USE_SECGROUP=False` in local.conf, it works; without it, tenant traffic is busted | 20:06 |
| ryanpetrello | if I reset to `firewall_driver=neutron.agent.linux.iptables_firewall.IptablesFirewallDriver` in ml2_conf.ini, restart neutron services, delete the router VM (and let the rug spawn a new one), the tenant network doesn't work on VMs I boot (can't ping 192.168.x.x) | 20:09 |
| ryanpetrello | yep, this is 100% the issue | 20:14 |
| ryanpetrello | I've been able to swap the config value back and forth and restart neutron a few times now | 20:14 |
| ryanpetrello | and make it break/work | 20:14 |
| ryanpetrello | adam_g davidlenwell ^ | 20:14 |
| ryanpetrello | now as for the proper way to fix it, or why it's not working :) | 20:15 |
| ryanpetrello | that I haven't dug into yet | 20:15 |
| davidlenwell | so setting the firewall to noop worked for you ? | 20:16 |
| davidlenwell | I'm not sure I would call that a fix .. but it points us at the right direction.. I had simular thoughts to adam_g about the mac address / anti spoofing protection that is probably being triggered when you are using the iptables firewall.. but why it just started catching that I don't know | 20:17 |
| adam_g | davidlenwell, im not certain this was ever confirmed to have worked | 20:18 |
| davidlenwell | ahh | 20:18 |
| davidlenwell | okay.. I have not eaten since 7am.. so im gonna stop and eat | 20:19 |
| adam_g | davidlenwell, im gonna leave this connectivity issue to you, i need to spend the rest of the day on the other stuff | 20:21 |
| davidlenwell | adam_g: okay | 20:22 |
| davidlenwell | ryanpetrello: I'll be back at my desk in an hour or so.. | 20:26 |
| ryanpetrello | yea | 20:29 |
| ryanpetrello | my best guess is that this just never actually worked | 20:29 |
| ryanpetrello | and setting firewall to noop isn't really a "fix" for production | 20:29 |
| ryanpetrello | more me just saying, "Look, it's a firewall issue" | 20:29 |
| adam_g | yea | 20:38 |
| *** puranamr has joined #akanda | 20:57 | |
| *** seanmwinn has quit IRC | 20:58 | |
| *** jordantardif has joined #akanda | 21:06 | |
| *** jordantardif has quit IRC | 21:09 | |
| *** jordantardif has joined #akanda | 21:10 | |
| *** puranamr has quit IRC | 21:12 | |
| davidlenwell | okay ryanpetrello I can confirm your finding that enabling and dissabling the iptables firewall driver breaks/fixes things | 21:42 |
| davidlenwell | I'm digging into why right now.. specifically troubleshooting iptables and the config it gets handed | 21:43 |
| davidlenwell | will keep you posted | 21:43 |
| *** skamithi14 has joined #akanda | 21:48 | |
| *** skamithi13 has quit IRC | 21:50 | |
| *** openstackgerrit has quit IRC | 22:16 | |
| *** openstackgerrit has joined #akanda | 22:16 | |
| *** jordantardif has quit IRC | 22:58 | |
| *** skamithi14 has quit IRC | 23:55 | |
| *** skamithi13 has joined #akanda | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!