| *** blp has quit IRC | 00:14 | |
| *** armax has joined #openstack-neutron-ovn | 01:22 | |
| *** russellb changes topic to "http://networking-ovn.readthedocs.org/ -=- OVN meeting Thursdays 1700 UTC #openvswitch" | 13:26 | |
| russellb | docs now published | 13:27 |
|---|---|---|
| russellb | could use some more love | 13:27 |
| mestery | russellb: Nice! | 13:37 |
| mestery | russellb: Back to trying on fedora this morning, was knocked out by this nasty cold I've had for a week | 13:37 |
| mestery | Bah | 13:37 |
| russellb | mestery: i'm with you | 13:37 |
| russellb | had a cold for 2 weeks now, saw dr yesterday though | 13:37 |
| russellb | got antibiotic ... developed a sinus infection | 13:37 |
| russellb | fun times. | 13:37 |
| russellb | and by fun i mean "fun" | 13:37 |
| russellb | mestery: recreating the setup is "working" for me, but I don't think the network is working like I think it is | 13:38 |
| russellb | it might be an illusion | 13:38 |
| russellb | but in any case, I'm not getting nova errors | 13:38 |
| mestery | oh man | 13:38 |
| mestery | I'm likely to go into the doctor in a bit too | 13:38 |
| mestery | Tired of being sick :( | 13:38 |
| russellb | ++ | 13:39 |
| russellb | mestery: being a PTL will do that :-p | 13:39 |
| mestery | lol, no kidding, and coupled with 4 kids ... | 13:40 |
| mestery | need to get healthy before my insane 2 weeks of travel coming up too | 13:40 |
| openstackgerrit | Merged stackforge/networking-ovn: docs: Remove content from usage.rst https://review.openstack.org/179261 | 13:40 |
| russellb | today was my kid's last day of day care | 13:41 |
| russellb | i'm hoping that cuts down on how much crap he brings home, for his sake and mine :) | 13:41 |
| mestery | nice! | 13:41 |
| openstackgerrit | Merged stackforge/networking-ovn: devstack: Fix git repo name assumptions. https://review.openstack.org/179262 | 13:41 |
| openstackgerrit | Merged stackforge/networking-ovn: devstack: Give example of custom ovs git repo. https://review.openstack.org/179263 | 13:41 |
| mestery | I remember that when my daughter (first kid) stopped going to daycare | 13:41 |
| mestery | It helped a lot | 13:41 |
| openstackgerrit | Russell Bryant proposed stackforge/networking-ovn: docs: Add testing howto doc. https://review.openstack.org/179264 | 13:43 |
| openstackgerrit | Russell Bryant proposed stackforge/networking-ovn: Move TODO out of published docs. https://review.openstack.org/179384 | 13:43 |
| openstackgerrit | Russell Bryant proposed stackforge/networking-ovn: docs: Add testing howto doc. https://review.openstack.org/179264 | 13:46 |
| russellb | mestery: for your ACK at some point :-) https://review.openstack.org/#/c/175954/ | 13:51 |
| *** openstackgerrit has quit IRC | 13:51 | |
| mestery | russellb: Nice! I saw Octavia wants to do the same, so cool! | 13:51 |
| russellb | makes sense | 13:51 |
| *** openstackgerrit has joined #openstack-neutron-ovn | 13:51 | |
| russellb | otherwiseguy: didn't want to +2 https://review.openstack.org/#/c/178540/6 ? | 14:23 |
| otherwiseguy | Had a comment on patch 5, but doesn't really matter one way or the other. Can always just do another patch. | 14:24 |
| otherwiseguy | russellb:^ | 14:24 |
| russellb | oh sorry | 14:25 |
| otherwiseguy | russellb: no need. it was really just a very small 'would it be more consistent if we...' kind of thing. | 14:27 |
| *** shettyg has joined #openstack-neutron-ovn | 14:43 | |
| openstackgerrit | Russell Bryant proposed stackforge/networking-ovn: devstack: Update integration bridge config https://review.openstack.org/179399 | 14:55 |
| *** gsagie_ has joined #openstack-neutron-ovn | 14:58 | |
| gsagie_ | russellb : any help needed in the multinode devstack? | 15:01 |
| openstackgerrit | Russell Bryant proposed stackforge/networking-ovn: Drop version= from setup.cfg. https://review.openstack.org/179405 | 15:02 |
| gsagie_ | nice job on the docs | 15:03 |
| russellb | thanks! | 15:03 |
| russellb | re: multi-node, feel free to help if you'd like | 15:04 |
| russellb | there's not much code needed | 15:04 |
| russellb | it's a bit of code and a lot more testing and docs work | 15:04 |
| gsagie_ | np, was just wondering if you got to it, if not i can start on sunday, if you did i can start testing and see if anything needs a fix | 15:05 |
| russellb | the code needed is to make it possible to configure what services get run locally, only one node should run ovn-northd | 15:05 |
| russellb | and on that node, ovsdb-server needs to serve the additional DBs | 15:05 |
| russellb | and the other nodes need to know where ovn-northd is running | 15:05 |
| russellb | that kind of config flexibility | 15:05 |
| gsagie_ | ok | 15:06 |
| russellb | gsagie_: whatever i get done i'll get up on gerrit so you can see what i have | 15:07 |
| gsagie_ | ok cool, will continue from there | 15:08 |
| russellb | i need to do some more work to better verify that the OVN datapath is working as expected | 15:08 |
| russellb | and document how to check it | 15:08 |
| russellb | gsagie_: you +2'd this patch before, can you re-add? https://review.openstack.org/179264 | 15:13 |
| gsagie_ | reading changes | 15:14 |
| russellb | thanks | 15:18 |
| otherwiseguy | russellb: the threads I create in that patch are only created after the up event is triggered, then it creates the thread just to run the notification which quickly exits (unless there is some kind of deadlock or something). | 15:33 |
| otherwiseguy | (with that said, I started working on the thread/queue implementation) :p | 15:33 |
| russellb | otherwiseguy: oh nevermind. | 15:33 |
| russellb | whatever you think | 15:34 |
| otherwiseguy | russellb: playing with threads while cloud-init builds fail. :p | 15:34 |
| * mestery back from doctor and trying OVN on fedora again | 15:44 | |
| gsagie_ | russellb : i think you noticed yesterday that ovn-northd didn't update the port status ? | 15:47 |
| gsagie_ | if there is a bug in OVN maybe we should hold with otherwiseguy patch, as the port will never get updated to up in nova | 15:48 |
| otherwiseguy | gsagie_: would be fine with me as I'm playing around with threading stuff on it. | 15:48 |
| otherwiseguy | Someone can just click 'rebase' :p | 15:48 |
| openstackgerrit | Gal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions https://review.openstack.org/178540 | 15:50 |
| gsagie_ | yeah, just think i saw him says something about it yesterday but maybe it works, didn't check it yet myself | 15:50 |
| gsagie_ | otherwiseguy, russell the proposed patch with otherwiseguy comment addressed ;) | 15:51 |
| otherwiseguy | gsagie_: didn't we talk about you working too much? :p | 15:51 |
| russellb | gsagie_: yeah, in my first devstack install, the ports weren't being set to up, i need to debug it | 15:56 |
| russellb | instead i re-created my env and documented it along the way | 15:56 |
| openstackgerrit | Terry Wilson proposed stackforge/networking-ovn: Add port 'up' notification https://review.openstack.org/178826 | 15:58 |
| otherwiseguy | doh, that worked, but I can see it is broken. | 15:59 |
| otherwiseguy | updating | 15:59 |
| openstackgerrit | Terry Wilson proposed stackforge/networking-ovn: Add port 'up' notification https://review.openstack.org/178826 | 16:14 |
| otherwiseguy | BTW, if anyone needs a quick script to play around with creating an ovs port to trigger an 'up' notification based on a neutron port-create $net --name $name: https://goo.gl/wEsu7s e.g. neutron port-create private myport1 && ./up.sh myport1. | 16:17 |
| openstackgerrit | Gal Sagie proposed stackforge/networking-ovn: Reduce the number of OVSDB transactions https://review.openstack.org/178540 | 16:27 |
| *** gsagie_ has quit IRC | 16:37 | |
| *** ajo has joined #openstack-neutron-ovn | 16:45 | |
| *** markmcclain has quit IRC | 16:49 | |
| *** HenryG is now known as floccinaucinihil | 16:54 | |
| *** floccinaucinihil is now known as HenryThe8th | 16:55 | |
| *** markmcclain has joined #openstack-neutron-ovn | 17:03 | |
| *** blp has joined #openstack-neutron-ovn | 17:32 | |
| russellb | mestery: think you could look over my devstack testing guide? want to get it published so I can share. :-) https://review.openstack.org/#/c/179264 | 17:46 |
| mestery | russellb: Ack! I'll do that now. | 17:46 |
| mestery | russellb: I'm still getting odd nova errors in my setup which I can't get past :( | 17:46 |
| russellb | mestery: odd, but doesn't sound ovn related :-/ | 17:46 |
| mestery | Nope, which is all the more frustrating :( | 17:46 |
| russellb | mestery: if you want to give me access i can dig into it though | 17:46 |
| *** ajo has quit IRC | 17:47 | |
| mestery | russellb: Cool, thanks! Give me a few more minutes, I'm trying one more thing | 17:47 |
| russellb | k | 17:47 |
| mestery | In the meantime, I'm looking at your reviews now | 17:47 |
| russellb | i rebuilt my env and it worked again *shrug* | 17:47 |
| russellb | well, "worked" | 17:47 |
| russellb | behaved the same way as before anyway | 17:47 |
| mestery | lol | 17:48 |
| mestery | Good datapoint! | 17:48 |
| openstackgerrit | Merged stackforge/networking-ovn: Move TODO out of published docs. https://review.openstack.org/179384 | 17:49 |
| *** shettyg has quit IRC | 18:03 | |
| *** ajo has joined #openstack-neutron-ovn | 18:05 | |
| *** shettyg has joined #openstack-neutron-ovn | 18:06 | |
| mestery | russellb: So, even before launching a VM, I get this error in my n-cpu log: | 18:09 |
| mestery | 2015-05-01 17:54:41.857 ERROR nova.compute.manager [req-35c4e807-46b1-40de-9b29-b9a17e327ec2 None None] No compute node record for host ovn | 18:09 |
| mestery | russellb: I suspect that's the root of my problem | 18:09 |
| russellb | yeah | 18:09 |
| russellb | that's the error you were getting on boot too | 18:09 |
| russellb | i'll dig into it, got sidetracked by ovn bug i noticed | 18:09 |
| mestery | no worries | 18:10 |
| mestery | I'm getting pulled in 3 directions ATM as well | 18:10 |
| *** shettyg has quit IRC | 18:11 | |
| russellb | sounds about right | 18:12 |
| russellb | well, fixed an ovn bug making ovn-controller die in my test env | 18:23 |
| russellb | yay. | 18:24 |
| russellb | blp: this killed ovn-controller :) http://openvswitch.org/pipermail/dev/2015-May/054928.html | 18:25 |
| russellb | even though apparently i can't spell in my commit message ... and typoed the config variable name | 18:25 |
| mestery | lol | 18:29 |
| russellb | blp: and I think I figured out why things "worked" before fixes ... if the above killed ovn-controller right away, and I also hadn't updated our code to configure br-int with fail-mode=secure, I think everything on the switch was just happily talking to each other with the default switch config, heh | 18:32 |
| mestery | russellb: Getting further with my setup | 18:33 |
| mestery | Now seeing this: tError: Unable to add port tap0f222df0-7a to OVS bridge br-int: Operation not permitted | 18:33 |
| mestery | libvirtError: Unable to add port tap0f222df0-7a to OVS bridge br-int: Operation not permitted | 18:33 |
| russellb | progress indeed | 18:33 |
| mestery | Yup | 18:33 |
| mestery | oh no | 18:35 |
| mestery | is it apparmor? | 18:35 |
| mestery | May 1 18:16:48 localhost kernel: [251495.793655] type=1400 audit(1430504208.416:47): apparmor="DENIED" operation="exec" profile="/usr/sbin/libvirtd" name="/usr/local/bin/ovs-vsctl" pid=8661 comm="libvirtd" requested_mask="x" denied_mask="x" fsuid=0 ouid=0 | 18:35 |
| * mestery cries a little | 18:35 | |
| russellb | lolol | 18:36 |
| russellb | security FTMFL! | 18:36 |
| * mestery sheds tears of uncertainty | 18:36 | |
| * mestery boots a VM with APPARMOR TURNED OFF | 18:36 | |
| mestery | Whew | 18:37 |
| mestery | it still failed | 18:37 |
| mestery | lol | 18:37 |
| * mestery reboots his devstack VM | 18:41 | |
| *** ajo has quit IRC | 18:44 | |
| * mestery burned apparmor to the ground | 18:47 | |
| *** shettyg has joined #openstack-neutron-ovn | 19:02 | |
| openstackgerrit | Russell Bryant proposed stackforge/networking-ovn: devstack: Run ovn-controller with sudo. https://review.openstack.org/179464 | 19:16 |
| russellb | 2015-05-01T19:21:59.096Z|00015|pipeline|INFO|Dropped 59 log messages in last 60 seconds (most recently, 1 seconds ago) due to excessive rate | 19:22 |
| russellb | yay | 19:22 |
| mestery | cool! | 19:22 |
| mestery | after burning apparmor to the ground ... MY VMS BOOT! | 19:24 |
| mestery | and my pings work! | 19:25 |
| mestery | WOOT! | 19:25 |
| mestery | russellb: ^^^^^ | 19:25 |
| * mestery dances the happy dance | 19:25 | |
| russellb | \o/ | 19:28 |
| mestery | However, when I dump the flows on br-int, I only see the NORMAL action | 19:29 |
| mestery | which explains why it's working | 19:29 |
| mestery | But likely not the right thing | 19:29 |
| russellb | mestery: yeah so i was about to break some news to you haha | 19:29 |
| mestery | ubuntu@ovn:~/devstack$ sudo ovs-ofctl dump-flows br-int | 19:29 |
| mestery | NXST_FLOW reply (xid=0x4): | 19:29 |
| mestery | cookie=0x0, duration=906.883s, table=0, n_packets=352, n_bytes=40363, idle_age=1, priority=0 actions=NORMAL | 19:29 |
| mestery | ubuntu@ovn:~/devstack$ | 19:29 |
| mestery | HA! | 19:29 |
| mestery | lol | 19:29 |
| russellb | i just recently put the pieces together that i think our pings are full of lies | 19:29 |
| russellb | good news is blp really did make things work | 19:29 |
| russellb | but, issue #1: https://review.openstack.org/179399 | 19:30 |
| russellb | issue #2 is the bug for the patch you acked on ovs-dev, that made ovn-controller exit right away | 19:30 |
| russellb | SO | 19:30 |
| russellb | um | 19:30 |
| russellb | yeah. | 19:30 |
| russellb | ovn-controller never set anything up, so, our VMs are just on ovs in normal mode, haha | 19:31 |
| russellb | OOPS | 19:31 |
| mestery | lol | 19:31 |
| mestery | well ... HAPPY FRIDAY! :) | 19:31 |
| russellb | they totally were able to ping each other though | 19:31 |
| russellb | but don't look behind the curtain | 19:32 |
| russellb | anyway, like i said, the good news is blp made only a couple of minor fixes and it really does work with latest code :) | 19:32 |
| mestery | Cool! Did you pull the latest code into your repo yet? That's where I had my local.conf pointed | 19:33 |
| mestery | Also, we need otherwiseguy to merge your change which I +2'd | 19:33 |
| russellb | ovn repo? | 19:33 |
| russellb | let's see | 19:34 |
| russellb | basically we need the one fix on top of blp's ovn branch | 19:34 |
| * russellb does that | 19:34 | |
| mestery | cool! | 19:34 |
| * otherwiseguy reads history | 19:34 | |
| mestery | otherwiseguy: https://review.openstack.org/179464 | 19:35 |
| russellb | http://github.com/russellb/ovs.git ovn branch has it now | 19:35 |
| russellb | which is just github.com/blp/ovs-reviews.git ovn branch plus my 1 fix | 19:35 |
| otherwiseguy | +2 givified | 19:36 |
| mestery | cool russellb! | 19:36 |
| russellb | and then you need some ovn-networking fixes | 19:36 |
| russellb | all on gerrit | 19:36 |
| russellb | oh the joys of stuff under this rapid of dev, heh | 19:36 |
| mestery | :) | 19:37 |
| russellb | need to run for now, will talk to you all later | 19:38 |
| *** HenryThe8th has quit IRC | 19:50 | |
| *** ajo has joined #openstack-neutron-ovn | 19:50 | |
| *** ajo has quit IRC | 19:52 | |
| *** ajo has joined #openstack-neutron-ovn | 19:52 | |
| *** HenryG has joined #openstack-neutron-ovn | 19:54 | |
| *** HenryG has quit IRC | 19:55 | |
| *** HenryG has joined #openstack-neutron-ovn | 19:56 | |
| openstackgerrit | Merged stackforge/networking-ovn: devstack: Update integration bridge config https://review.openstack.org/179399 | 20:14 |
| openstackgerrit | Merged stackforge/networking-ovn: devstack: Run ovn-controller with sudo. https://review.openstack.org/179464 | 20:15 |
| *** HenryG has quit IRC | 20:16 | |
| openstackgerrit | Kyle Mestery proposed stackforge/networking-ovn: setup.cfg: Remove version https://review.openstack.org/179494 | 20:17 |
| *** HenryG has joined #openstack-neutron-ovn | 20:21 | |
| otherwiseguy | russellb: looks like your global-requirements patch is again out of sync. Crazy. | 20:31 |
| otherwiseguy | russellb: the suggestion for a minimum version may not be horrible. | 20:34 |
| otherwiseguy | mestery: I had to look up what #FF meant. I'm so old. | 20:36 |
| mestery | openstackgerrit: lol | 20:36 |
| mestery | otherwiseguy: lol | 20:36 |
| mestery | thought openstackgerrit, also lol | 20:36 |
| otherwiseguy | I think I *finally* have gotten disk-image-builder to build me a fedora image with a custom cloud-init package based off the latest commit which *looks* like it has ipv6 support so I can finally test it. | 20:40 |
| mestery | woot! | 20:40 |
| otherwiseguy | http://i3.kym-cdn.com/photos/images/facebook/000/234/765/b7e.jpg | 20:41 |
| mestery | rofl | 20:41 |
| otherwiseguy | That dog is better dressed than I ever am. | 20:42 |
| russellb | otherwiseguy: so ... did you see that there's a new version of cirros available (maybe just for testing and not released) that's supposed to have some ipv6 stuff in it? | 20:44 |
| otherwiseguy | russellb: I did not. I gave up on it when I noticed there was no ping6 command. Figures. | 20:44 |
| russellb | otherwiseguy: http://lists.openstack.org/pipermail/openstack-dev/2015-April/062403.html | 20:45 |
| otherwiseguy | I notice that launchpad has the latest release of cloud-init at 0.7.6 but ubuntu has a 0.7.7 package built. | 20:45 |
| russellb | otherwiseguy: i know nothing beyond the existence of that ML post | 20:45 |
| otherwiseguy | One of these ays I'm going to start reading my email. | 20:46 |
| otherwiseguy | and days | 20:46 |
| russellb | it doesn't actually say anything about cloud-init | 20:46 |
| russellb | so *shrug* | 20:46 |
| otherwiseguy | russellb: I guess at least I know what patches need to be modified for the fedora cloud-init packagers to update to the latest version. :p | 20:47 |
| russellb | :) | 20:47 |
| mestery | russellb: Even with your new OVS repo, I still don't see any flows on br-int | 20:56 |
| russellb | mestery: welp. | 20:59 |
| mestery | russellb: It's almost quitting time here, the drugs are making me sleepy. Cold medicine and all :) | 20:59 |
| russellb | mestery: yeah, it has been that kinda week.. | 20:59 |
| russellb | mestery: we'll get things all sorted and more predictable next week :) | 20:59 |
| mestery | ++ | 21:00 |
| russellb | if you feel like looking any more, i'd say 1) is ovn-controller running? and 2) check /usr/local/var/log/openvswitch/ovn-controller.log | 21:00 |
| russellb | for anything interesting | 21:00 |
| russellb | orrrrr we'll figure it out next week | 21:01 |
| russellb | either is fine with me | 21:01 |
| mestery | ovn-controller not running :( | 21:02 |
| mestery | May 1 20:23:20 localhost kernel: [ 5218.260438] ovn-controller[32242]: segfault at 150 ip 000000000040818a sp 00007ffd2fd52150 error 4 in ovn-cont | 21:03 |
| mestery | roller[400000+124000] | 21:03 |
| mestery | That explains it :) | 21:03 |
| russellb | ooooh, fun | 21:03 |
| mestery | yup | 21:03 |
| russellb | i wonder if my patch is broken :-/ | 21:03 |
| russellb | must ... put ... code ... down | 21:04 |
| mestery | lol | 21:04 |
| mestery | Is your patch on your ovn branch from your github? | 21:05 |
| mestery | The last one I see is from blp | 21:05 |
| russellb | it's supposed to be | 21:05 |
| russellb | :-/ | 21:05 |
| russellb | but no | 21:05 |
| russellb | it is not. | 21:05 |
| russellb | ugh | 21:05 |
| mestery | lol | 21:05 |
| mestery | well, that was it! | 21:05 |
| russellb | what the heck | 21:07 |
| russellb | https://github.com/russellb/ovs/commits/ovn | 21:08 |
| russellb | fixed ... | 21:08 |
| russellb | i made a n00b git mistake | 21:09 |
| *** ajo has quit IRC | 21:09 | |
| russellb | the old branch was definitely broken | 21:10 |
| russellb | sorry. | 21:10 |
| russellb | mestery: ^ | 21:10 |
| russellb | ok need to call it a day/week i think | 21:11 |
| russellb | bye! | 21:12 |
| mestery | russellb: later and thanks! | 21:12 |
| *** ajo has joined #openstack-neutron-ovn | 21:15 | |
| *** ajo has quit IRC | 21:18 | |
| *** shettyg has quit IRC | 22:03 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
