*** ramamurthy has quit IRC | 00:02 | |
*** banix has joined #openstack-neutron-ovn | 00:27 | |
*** salv-orl_ has quit IRC | 00:42 | |
*** banix has quit IRC | 00:48 | |
*** chandrav has quit IRC | 00:51 | |
*** ramamurthy has joined #openstack-neutron-ovn | 00:56 | |
*** doonhammer has quit IRC | 01:06 | |
*** banix has joined #openstack-neutron-ovn | 01:07 | |
*** banix has quit IRC | 01:45 | |
*** mickeys has joined #openstack-neutron-ovn | 02:03 | |
*** jckasper has quit IRC | 02:07 | |
*** jckasper has joined #openstack-neutron-ovn | 02:08 | |
*** jckasper has quit IRC | 02:13 | |
*** salv-orlando has joined #openstack-neutron-ovn | 02:19 | |
*** salv-orlando has quit IRC | 02:26 | |
*** rtheis has quit IRC | 02:35 | |
*** salv-orlando has joined #openstack-neutron-ovn | 02:55 | |
*** salv-orlando has quit IRC | 02:59 | |
*** mickeys has quit IRC | 03:00 | |
*** mickeys has joined #openstack-neutron-ovn | 03:01 | |
*** mickeys has quit IRC | 03:05 | |
*** yamamot__ has joined #openstack-neutron-ovn | 03:10 | |
*** salv-orlando has joined #openstack-neutron-ovn | 03:29 | |
*** salv-orlando has quit IRC | 03:36 | |
*** ramamurthy has quit IRC | 03:38 | |
*** yamamot__ has quit IRC | 04:24 | |
*** yamamot__ has joined #openstack-neutron-ovn | 04:28 | |
*** salv-orlando has joined #openstack-neutron-ovn | 04:39 | |
*** salv-orlando has quit IRC | 04:47 | |
*** salv-orlando has joined #openstack-neutron-ovn | 04:49 | |
*** mickeys has joined #openstack-neutron-ovn | 05:02 | |
*** mickeys has quit IRC | 05:05 | |
*** mickeys has joined #openstack-neutron-ovn | 05:06 | |
*** mickeys has quit IRC | 05:10 | |
*** yamamot__ has quit IRC | 05:52 | |
*** banix has joined #openstack-neutron-ovn | 05:58 | |
*** jckasper has joined #openstack-neutron-ovn | 06:12 | |
*** jckasper has quit IRC | 06:17 | |
*** banix has quit IRC | 06:22 | |
*** armax has quit IRC | 06:30 | |
*** yamamot__ has joined #openstack-neutron-ovn | 06:33 | |
*** salv-orlando has quit IRC | 07:51 | |
*** salv-orlando has joined #openstack-neutron-ovn | 07:52 | |
*** mickeys has joined #openstack-neutron-ovn | 07:54 | |
*** mickeys has quit IRC | 07:58 | |
*** arosen12 has quit IRC | 07:59 | |
*** gongysh has joined #openstack-neutron-ovn | 08:02 | |
*** fzdarsky has joined #openstack-neutron-ovn | 08:10 | |
*** openstackgerrit has quit IRC | 08:18 | |
*** openstackgerrit has joined #openstack-neutron-ovn | 08:18 | |
*** pcaruana has joined #openstack-neutron-ovn | 08:30 | |
*** salv-orlando has quit IRC | 09:03 | |
*** roeyc has joined #openstack-neutron-ovn | 09:09 | |
*** gongysh has quit IRC | 09:52 | |
*** yamamot__ has quit IRC | 09:54 | |
*** gongysh has joined #openstack-neutron-ovn | 10:00 | |
*** jckasper has joined #openstack-neutron-ovn | 10:03 | |
*** jckasper has quit IRC | 10:08 | |
openstackgerrit | Babu Shanmugam proposed openstack/networking-ovn: Grenade plugin for testing OVN migration from ML2/OVS https://review.openstack.org/335825 | 10:17 |
---|---|---|
*** salv-orlando has joined #openstack-neutron-ovn | 10:22 | |
*** gongysh has quit IRC | 10:26 | |
*** chandrav has joined #openstack-neutron-ovn | 10:41 | |
*** salv-orl_ has joined #openstack-neutron-ovn | 10:43 | |
*** fzdarsky is now known as fzdarsky|lunch | 10:45 | |
*** salv-orlando has quit IRC | 10:46 | |
*** rtheis has joined #openstack-neutron-ovn | 10:58 | |
*** salv-orlando has joined #openstack-neutron-ovn | 11:04 | |
*** salv-orlando has quit IRC | 11:04 | |
*** salv-orl_ has quit IRC | 11:07 | |
*** roeyc has quit IRC | 11:33 | |
*** ramamurthy has joined #openstack-neutron-ovn | 11:58 | |
*** dkehn_ has joined #openstack-neutron-ovn | 12:22 | |
*** fzdarsky|lunch is now known as fzdarsky | 12:23 | |
*** dkehn__ has quit IRC | 12:25 | |
*** rtheis has quit IRC | 12:29 | |
*** banix has joined #openstack-neutron-ovn | 12:31 | |
*** roeyc has joined #openstack-neutron-ovn | 12:34 | |
*** jckasper has joined #openstack-neutron-ovn | 13:05 | |
*** jckasper has quit IRC | 13:07 | |
*** jckasper has joined #openstack-neutron-ovn | 13:08 | |
*** jckasper has quit IRC | 13:08 | |
*** jckasper has joined #openstack-neutron-ovn | 13:09 | |
*** rtheis has joined #openstack-neutron-ovn | 13:32 | |
*** dkehn has joined #openstack-neutron-ovn | 13:40 | |
*** dkehn_ has quit IRC | 13:41 | |
*** chandrav has quit IRC | 14:00 | |
*** numans has joined #openstack-neutron-ovn | 14:02 | |
*** mlavalle has joined #openstack-neutron-ovn | 14:17 | |
*** lrichard has quit IRC | 14:38 | |
*** lrichard has joined #openstack-neutron-ovn | 14:38 | |
*** salv-orlando has joined #openstack-neutron-ovn | 14:44 | |
*** salv-orl_ has joined #openstack-neutron-ovn | 14:45 | |
*** banix has quit IRC | 14:46 | |
openstackgerrit | Richard Theis proposed openstack/networking-ovn: WIP: Use OVN address set to implement remote security groups https://review.openstack.org/318995 | 14:49 |
*** salv-orlando has quit IRC | 14:49 | |
*** jckasper has quit IRC | 14:53 | |
*** jckasper has joined #openstack-neutron-ovn | 14:54 | |
*** jckasper has quit IRC | 14:55 | |
*** jckasper has joined #openstack-neutron-ovn | 14:56 | |
*** banix has joined #openstack-neutron-ovn | 15:01 | |
*** mickeys has joined #openstack-neutron-ovn | 15:01 | |
*** gongysh has joined #openstack-neutron-ovn | 15:06 | |
*** regXboi has joined #openstack-neutron-ovn | 15:08 | |
regXboi | yo, russellb - you around? | 15:09 |
russellb | I am! | 15:09 |
russellb | \o/ | 15:09 |
regXboi | address sets is crashing on deletes | 15:09 |
*** ramamurthy has quit IRC | 15:09 | |
regXboi | rtheis is seeing tempest failures because of this | 15:10 |
regXboi | it looks like a double destroy | 15:10 |
russellb | ah yes, i heard / saw | 15:10 |
russellb | anything i can do to help? | 15:10 |
regXboi | push the ovs patch when I get it crafted submitted??? :) | 15:10 |
*** doonhammer has joined #openstack-neutron-ovn | 15:10 | |
russellb | ha, sure | 15:10 |
regXboi | I'm still trying to figure out how the second entry is getting into the SSET | 15:11 |
russellb | have you figured out an easy way to reproduce? | 15:11 |
regXboi | stand up devstack and run rally at it | 15:11 |
regXboi | when rally tears down stuff - boom | 15:11 |
regXboi | so I think deletes is the key | 15:11 |
*** ramamurthy has joined #openstack-neutron-ovn | 15:12 | |
*** banix has quit IRC | 15:13 | |
regXboi | I think I'm going to dump the sset list in the interim | 15:16 |
*** banix has joined #openstack-neutron-ovn | 15:16 | |
regXboi | because the logs say it's not pulling the symbol out of sset, but I'm still not 100% sure :) | 15:16 |
openstackgerrit | Richard Theis proposed openstack/networking-ovn: WIP: Use OVN address set to implement remote security groups https://review.openstack.org/318995 | 15:19 |
russellb | ok, hopefully we can narrow it down to a test case we can add to the ovs test suite | 15:19 |
regXboi | that's where I'm trying to get to | 15:19 |
regXboi | because we don't actually test deleting an address set | 15:20 |
regXboi | ok... looking at the logs, the problem is where we try and pull the name out of the SSET when removing the node | 15:25 |
regXboi | it looks like we aren't getting rid of it | 15:25 |
regXboi | one more run to triple check that statement | 15:25 |
*** dkehn__ has joined #openstack-neutron-ovn | 15:25 | |
*** doonhammer has quit IRC | 15:26 | |
*** dkehn has quit IRC | 15:28 | |
*** ramamurthy has quit IRC | 15:35 | |
regXboi | hmm, the sset is getting really screwed up | 15:37 |
russellb | i'll jump in once i finish this expense report ... | 15:40 |
russellb | ... and eat lunch | 15:40 |
russellb | :) | 15:40 |
*** doonhammer has joined #openstack-neutron-ovn | 15:40 | |
regXboi | russellb: question for you as/after you eat lunch - should the local_address_sets have multiple entries for the same name? | 15:43 |
russellb | i honestly don't remember the code at all | 15:44 |
russellb | so you're ahead of me | 15:44 |
regXboi | ok, that's fair | 15:44 |
regXboi | it looks like it's persisted, but not persisted | 15:44 |
*** pcaruana has quit IRC | 15:49 | |
regXboi | well... I should say it's getting corrupted, because looking through for the symbol that leads to the crash, I eventually see it existing in local_address_sets multiple times | 15:49 |
*** gongysh has quit IRC | 15:50 | |
regXboi | oh I see why | 15:50 |
regXboi | when the address set changes, we don't remove the old entry from local_address_sets, we just re-add it | 15:51 |
regXboi | and boom | 15:51 |
*** roeyc1 has joined #openstack-neutron-ovn | 15:51 | |
*** roeyc has quit IRC | 15:51 | |
*** regXboi has quit IRC | 15:55 | |
*** regXboi has joined #openstack-neutron-ovn | 15:55 | |
*** azbiswas has joined #openstack-neutron-ovn | 15:57 | |
regXboi | and that fixes it | 15:59 |
regXboi | russellb: the issue is we need to change the value of the address set, and then make sure that there is only one entry for the name in the address set afterwards | 16:00 |
regXboi | I'm not 100% sure how to write a test to *do* that, but that's the bug | 16:00 |
*** yamamoto has joined #openstack-neutron-ovn | 16:03 | |
*** regXboi has quit IRC | 16:06 | |
*** regXboi has joined #openstack-neutron-ovn | 16:06 | |
*** azbiswas_ has joined #openstack-neutron-ovn | 16:07 | |
*** armax has joined #openstack-neutron-ovn | 16:09 | |
*** azbiswas has quit IRC | 16:10 | |
*** ramamurthy has joined #openstack-neutron-ovn | 16:20 | |
*** dkehn__ is now known as dkehn_ | 16:20 | |
*** azbiswas_ has quit IRC | 16:21 | |
*** azbiswas has joined #openstack-neutron-ovn | 16:24 | |
regXboi | russellb: https://patchwork.ozlabs.org/patch/645967 | 16:25 |
regXboi | that's the one liner | 16:25 |
*** azbiswas has quit IRC | 16:38 | |
*** azbiswas has joined #openstack-neutron-ovn | 16:42 | |
*** regXboi has quit IRC | 16:49 | |
*** regXboi has joined #openstack-neutron-ovn | 16:51 | |
*** azbiswas has quit IRC | 16:54 | |
*** roeyc1 has quit IRC | 17:02 | |
*** azbiswas has joined #openstack-neutron-ovn | 17:04 | |
*** azbiswas_ has joined #openstack-neutron-ovn | 17:12 | |
*** azbiswas has quit IRC | 17:13 | |
*** xiaohhui has quit IRC | 17:20 | |
*** s3wong has joined #openstack-neutron-ovn | 17:31 | |
*** fzdarsky is now known as fzdarsky|afk | 17:32 | |
*** xiaohhui has joined #openstack-neutron-ovn | 17:36 | |
*** chandrav has joined #openstack-neutron-ovn | 17:38 | |
*** salv-orlando has joined #openstack-neutron-ovn | 17:38 | |
*** salv-orl_ has quit IRC | 17:39 | |
regXboi | russellb: for the test case, I think what we need to do is create/update/delete and then make sure the ovn-controller is still running | 17:39 |
russellb | yep, makes sense | 17:40 |
russellb | or create update delete twice | 17:40 |
russellb | with validation throughout | 17:40 |
russellb | or something | 17:40 |
regXboi | well the bug is in update | 17:40 |
regXboi | update leads to two entries | 17:40 |
regXboi | and then delete forces it out | 17:40 |
regXboi | and boom | 17:40 |
regXboi | russellb: what I don't see is how to do the update and destroy? do Ineed to do a find to get the record? | 17:48 |
regXboi | or is that returned by create? | 17:49 |
*** numans has quit IRC | 17:52 | |
*** salv-orlando has quit IRC | 17:55 | |
russellb | OK | 17:58 |
russellb | let's see here. | 17:58 |
*** salv-orlando has joined #openstack-neutron-ovn | 17:58 | |
*** igordcard has quit IRC | 17:59 | |
*** igordcard has joined #openstack-neutron-ovn | 17:59 | |
regXboi | ok I see how to do this, I think | 18:01 |
*** zhouhan has joined #openstack-neutron-ovn | 18:02 | |
regXboi | well, I'm not 100% sure how to check that ovn-controller is still running | 18:02 |
regXboi | other than a hack | 18:02 |
*** yamamoto has quit IRC | 18:03 | |
*** yamamoto has joined #openstack-neutron-ovn | 18:03 | |
*** regXboi has quit IRC | 18:06 | |
*** regXboi has joined #openstack-neutron-ovn | 18:06 | |
regXboi | hmm, all I need is a way to test that the controller is still running | 18:07 |
regXboi | I've got the rest | 18:07 |
russellb | regXboi: maybe run ovs-appctl? | 18:07 |
russellb | with ........ the version command? | 18:08 |
russellb | and make sure the command returns successfully? | 18:08 |
russellb | i assume it returns non-zero if it fails to connect | 18:08 |
regXboi | that should do it | 18:08 |
regXboi | let me try and see | 18:08 |
russellb | ok | 18:08 |
*** yamamoto has quit IRC | 18:09 | |
regXboi | alrighty, lets see if this test does the right thing before the patch | 18:17 |
*** fzdarsky|afk has quit IRC | 18:20 | |
regXboi | hmmm | 18:21 |
regXboi | what starts ovn-controller in the test suite? | 18:21 |
russellb | ovn_attach | 18:23 |
regXboi | ah thanks | 18:24 |
russellb | see tests/ofproto-macros.at | 18:24 |
russellb | or more useful, example uses | 18:24 |
regXboi | I'm looking at the eamples :) | 18:24 |
regXboi | examples even | 18:25 |
flaviof | regXboi: so the crash is that insert in local_address_sets fails as 'duplicate key', right? | 18:27 |
regXboi | flaviof: the issue is that when the address set gets updated, it gets added to the hash a second time | 18:27 |
regXboi | and then when it gets destroyed, we end up with a double destroy in the controller and boom | 18:28 |
flaviof | regXboi: ack. the fix looks right to me (https://patchwork.ozlabs.org/patch/645967/). | 18:28 |
regXboi | oh it's right | 18:28 |
regXboi | I just need the test | 18:28 |
regXboi | which I almost have | 18:28 |
flaviof | ack. | 18:28 |
regXboi | well, that check didn't quite work | 18:29 |
flaviof | regXboi: you are checking if northd is still alive after the update? | 18:30 |
regXboi | ovn-controller | 18:30 |
regXboi | that's where the explosion takes place | 18:30 |
regXboi | ovn-northd is just a simple copy IIRC | 18:31 |
flaviof | that is right. lfow.c is part of ovn-controller | 18:32 |
*** armax_ has joined #openstack-neutron-ovn | 18:32 | |
flaviof | is it crashing and getting re-spun? | 18:32 |
regXboi | not in devstack gate | 18:33 |
regXboi | russellb: got it - updated patch with test coming shortly | 18:33 |
russellb | yay | 18:33 |
*** armax has quit IRC | 18:34 | |
*** armax_ is now known as armax | 18:34 | |
*** azbiswas_ has quit IRC | 18:36 | |
*** azbiswas has joined #openstack-neutron-ovn | 18:38 | |
*** banix has quit IRC | 18:38 | |
regXboi | and it's now at patchworks - https://patchwork.ozlabs.org/patch/646016 | 18:38 |
*** banix has joined #openstack-neutron-ovn | 18:38 | |
*** chandrav has quit IRC | 18:41 | |
openstackgerrit | Merged openstack/networking-ovn: Update OVN reference architecture documentation https://review.openstack.org/311918 | 18:46 |
russellb | sleep 1 :-/ | 18:48 |
* russellb wonders if there's any better way ... | 18:48 | |
lrichard | russellb: I've done some experiments that show many of the sleeps make no difference (hundreds of iterations, wide range of loadavg, running tests in parallel, et.) | 18:52 |
russellb | lrichard: nice | 18:52 |
lrichard | russellb: I'd like to remove some, but first I'd like to see the tests work with the sleeps :) | 18:53 |
russellb | makes sense :) | 18:53 |
russellb | yes, a stable starting point would be nice | 18:53 |
russellb | regXboi: pushed to master | 18:55 |
russellb | rtheis: address sets crash fix now in master | 18:55 |
regXboi | woot ... thx russellb | 18:55 |
rtheis | russellb: thank you | 18:56 |
*** armax_ has joined #openstack-neutron-ovn | 19:01 | |
*** armax has quit IRC | 19:03 | |
*** armax_ is now known as armax | 19:03 | |
*** yamamoto has joined #openstack-neutron-ovn | 19:05 | |
*** chandrav has joined #openstack-neutron-ovn | 19:06 | |
*** yamamoto has quit IRC | 19:10 | |
*** salv-orlando has quit IRC | 19:12 | |
*** woodburn has quit IRC | 19:16 | |
*** regXboi has quit IRC | 19:23 | |
*** doonhammer has quit IRC | 19:29 | |
*** regXboi has joined #openstack-neutron-ovn | 19:31 | |
openstackgerrit | Russell Bryant proposed openstack/networking-ovn: faq: Update HA to reflect ovsdb replication. https://review.openstack.org/339201 | 19:35 |
*** doonhammer has joined #openstack-neutron-ovn | 19:39 | |
*** zhouhan has quit IRC | 19:52 | |
*** mickeys has quit IRC | 20:00 | |
*** yamamoto has joined #openstack-neutron-ovn | 20:07 | |
russellb | any networking-ovn reviews that need priority? | 20:10 |
russellb | i'm sure the answer is yes, but which one(s) heh | 20:10 |
*** yamamoto has quit IRC | 20:12 | |
*** jckasper has quit IRC | 20:12 | |
*** jckasper has joined #openstack-neutron-ovn | 20:13 | |
*** mickeys has joined #openstack-neutron-ovn | 20:15 | |
*** jckasper has quit IRC | 20:18 | |
rtheis | russellb: There are several WIPs. Hopefully, the address set one will be ready soon. https://review.openstack.org/#/c/322488/ is getting close | 20:20 |
rtheis | I haven't got back to looking at https://review.openstack.org/#/c/332434/ | 20:20 |
rtheis | And also https://review.openstack.org/#/c/335825/ | 20:20 |
rtheis | FYI: I'll be out starting tomorrow and through next week on vacation | 20:22 |
russellb | nice, enjoy! | 20:23 |
*** doonhammer has quit IRC | 20:24 | |
*** azbiswa__ has joined #openstack-neutron-ovn | 20:26 | |
*** doonhammer has joined #openstack-neutron-ovn | 20:26 | |
*** azbiswas has quit IRC | 20:27 | |
rtheis | thx | 20:28 |
*** banix has quit IRC | 20:32 | |
*** azbiswa__ has quit IRC | 20:40 | |
*** azbiswas has joined #openstack-neutron-ovn | 20:42 | |
*** salv-orlando has joined #openstack-neutron-ovn | 20:44 | |
*** salv-orlando has quit IRC | 20:46 | |
*** salv-orlando has joined #openstack-neutron-ovn | 20:46 | |
*** banix has joined #openstack-neutron-ovn | 20:48 | |
*** azbiswas has quit IRC | 20:49 | |
*** azbiswas has joined #openstack-neutron-ovn | 20:53 | |
*** azbiswas has quit IRC | 21:05 | |
*** azbiswas has joined #openstack-neutron-ovn | 21:06 | |
openstackgerrit | Richard Theis proposed openstack/networking-ovn: WIP: Use OVN address set to implement remote security groups https://review.openstack.org/318995 | 21:08 |
*** yamamoto has joined #openstack-neutron-ovn | 21:09 | |
russellb | rtheis: fingers crossed for a pass this time? :) | 21:09 |
rtheis | regXboi: did a recheck and it passed | 21:09 |
rtheis | I've updated the patch with some more unit tests | 21:09 |
russellb | nice | 21:10 |
*** azbiswas has quit IRC | 21:14 | |
*** yamamoto has quit IRC | 21:14 | |
*** azbiswas has joined #openstack-neutron-ovn | 21:16 | |
*** doonhammer has quit IRC | 21:19 | |
*** azbiswas has quit IRC | 21:21 | |
openstackgerrit | Ramu Ramamurthy proposed openstack/networking-ovn: WIP - VM metadata access with native-dhcp https://review.openstack.org/315305 | 21:22 |
*** azbiswas has joined #openstack-neutron-ovn | 21:24 | |
*** mamulsow has joined #openstack-neutron-ovn | 21:25 | |
*** doonhammer has joined #openstack-neutron-ovn | 21:30 | |
*** mickeys has quit IRC | 21:38 | |
*** doonhammer has quit IRC | 21:38 | |
*** regXboi has quit IRC | 21:41 | |
*** azbiswas has quit IRC | 21:41 | |
*** banix has quit IRC | 21:42 | |
*** azbiswas has joined #openstack-neutron-ovn | 21:43 | |
*** lrichard has quit IRC | 21:45 | |
*** azbiswas_ has joined #openstack-neutron-ovn | 21:56 | |
*** rtheis has quit IRC | 21:56 | |
*** azbiswas has quit IRC | 21:58 | |
*** mickeys has joined #openstack-neutron-ovn | 22:00 | |
*** banix has joined #openstack-neutron-ovn | 22:04 | |
*** jckasper has joined #openstack-neutron-ovn | 22:06 | |
*** yamamoto has joined #openstack-neutron-ovn | 22:10 | |
*** doonhammer has joined #openstack-neutron-ovn | 22:13 | |
*** yamamoto has quit IRC | 22:16 | |
openstackgerrit | Richard Theis proposed openstack/networking-ovn: WIP: Use OVN address set to implement remote security groups https://review.openstack.org/318995 | 22:16 |
*** salv-orlando has quit IRC | 22:30 | |
*** salv-orlando has joined #openstack-neutron-ovn | 22:34 | |
*** salv-orlando has quit IRC | 22:39 | |
*** ramamurthy has quit IRC | 22:42 | |
*** banix has quit IRC | 22:49 | |
*** azbiswas_ has quit IRC | 22:59 | |
*** azbiswas has joined #openstack-neutron-ovn | 23:00 | |
*** salv-orlando has joined #openstack-neutron-ovn | 23:01 | |
*** azbiswas has quit IRC | 23:04 | |
*** salv-orlando has quit IRC | 23:09 | |
*** yamamoto has joined #openstack-neutron-ovn | 23:12 | |
*** yamamoto has quit IRC | 23:17 | |
*** mickeys has quit IRC | 23:23 | |
*** ramamurthy has joined #openstack-neutron-ovn | 23:25 | |
*** lrichard has joined #openstack-neutron-ovn | 23:32 | |
*** ramamurthy has quit IRC | 23:32 | |
*** azbiswas has joined #openstack-neutron-ovn | 23:33 | |
*** salv-orlando has joined #openstack-neutron-ovn | 23:35 | |
*** azbiswas has quit IRC | 23:45 | |
*** salv-orlando has quit IRC | 23:47 | |
*** azbiswas has joined #openstack-neutron-ovn | 23:49 | |
*** azbiswas has quit IRC | 23:58 | |
*** chandrav has quit IRC | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!