| *** salv-orl_ has quit IRC | 00:06 | |
| *** s3wong has joined #openstack-neutron-ovn | 00:33 | |
| *** numans has joined #openstack-neutron-ovn | 00:36 | |
| *** lrichard has quit IRC | 00:41 | |
| *** mickeys has quit IRC | 00:49 | |
| *** gangil has quit IRC | 00:49 | |
| *** lrichard has joined #openstack-neutron-ovn | 00:53 | |
| *** regXboi has joined #openstack-neutron-ovn | 01:00 | |
| *** doonhammer has quit IRC | 01:04 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 01:08 | |
| *** salv-orlando has quit IRC | 01:15 | |
| *** s3wong has quit IRC | 01:59 | |
| *** gangil has joined #openstack-neutron-ovn | 02:03 | |
| *** regXboi has quit IRC | 02:11 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 02:15 | |
| *** salv-orlando has quit IRC | 02:27 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/networking-ovn: Updated from global requirements https://review.openstack.org/333690 | 03:13 |
|---|---|---|
| *** salv-orlando has joined #openstack-neutron-ovn | 03:29 | |
| *** salv-orlando has quit IRC | 03:37 | |
| *** gongysh has joined #openstack-neutron-ovn | 03:42 | |
| *** yamamot__ has joined #openstack-neutron-ovn | 03:47 | |
| *** regXboi has joined #openstack-neutron-ovn | 03:47 | |
| *** jckasper has joined #openstack-neutron-ovn | 04:09 | |
| *** regXboi has quit IRC | 04:09 | |
| *** regXboi has joined #openstack-neutron-ovn | 04:10 | |
| *** regXboi has quit IRC | 04:24 | |
| *** azbiswas has joined #openstack-neutron-ovn | 04:33 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 04:39 | |
| *** jckasper has quit IRC | 04:49 | |
| *** jckasper has joined #openstack-neutron-ovn | 04:50 | |
| *** salv-orlando has quit IRC | 04:51 | |
| *** yamamot__ has quit IRC | 04:53 | |
| *** jckasper has quit IRC | 04:54 | |
| *** azbiswas has quit IRC | 04:57 | |
| *** azbiswas has joined #openstack-neutron-ovn | 05:05 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 05:14 | |
| *** azbiswas_ has joined #openstack-neutron-ovn | 05:23 | |
| *** azbiswas has quit IRC | 05:25 | |
| *** azbiswas_ has quit IRC | 05:34 | |
| *** azbiswas has joined #openstack-neutron-ovn | 05:37 | |
| *** yamamot__ has joined #openstack-neutron-ovn | 05:46 | |
| *** azbiswas has quit IRC | 06:22 | |
| *** salv-orlando has quit IRC | 06:28 | |
| *** azbiswas has joined #openstack-neutron-ovn | 07:23 | |
| *** azbiswas has quit IRC | 07:28 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 07:36 | |
| *** gangil has quit IRC | 07:39 | |
| *** stenzel has joined #openstack-neutron-ovn | 08:06 | |
| *** mickeys has joined #openstack-neutron-ovn | 08:08 | |
| *** mickeys has quit IRC | 08:24 | |
| *** gongysh has quit IRC | 08:47 | |
| openstackgerrit | li,chen proposed openstack/networking-ovn: Remove 'origin/' in OVN_BRANCH https://review.openstack.org/333826 | 08:56 |
| *** azbiswas has joined #openstack-neutron-ovn | 09:12 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 09:12 | |
| *** azbiswas has quit IRC | 09:17 | |
| *** pcaruana has joined #openstack-neutron-ovn | 10:09 | |
| *** salv-orlando has quit IRC | 10:14 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 10:36 | |
| *** jckasper has joined #openstack-neutron-ovn | 10:52 | |
| *** jckasper has quit IRC | 10:56 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 10:59 | |
| *** salv-orlando has quit IRC | 11:02 | |
| *** rtheis has joined #openstack-neutron-ovn | 11:30 | |
| *** yamamot__ has quit IRC | 12:08 | |
| rtheis | FYI: gate-tempest-dsvm-networking-ovn is failing consistently, I'm looking into it now. | 12:24 |
| *** numans has quit IRC | 12:45 | |
| *** jckasper has joined #openstack-neutron-ovn | 12:46 | |
| *** azbiswas has joined #openstack-neutron-ovn | 12:47 | |
| *** jckasper has quit IRC | 12:48 | |
| *** azbiswas has quit IRC | 12:53 | |
| *** jckasper has joined #openstack-neutron-ovn | 12:55 | |
| *** yamamoto has joined #openstack-neutron-ovn | 12:57 | |
| *** numans has joined #openstack-neutron-ovn | 12:58 | |
| *** regXboi has joined #openstack-neutron-ovn | 13:04 | |
| regXboi | ugh, it looks like the n-ovn gate is ill | 13:04 |
| regXboi | gate-tempest-dsvm-networking-ovn appears to be failing on test_rebuild_server_with_personality | 13:05 |
| rtheis | indeed it is | 13:13 |
| rtheis | regXboi: any ideas on what may be causing this? | 13:15 |
| regXboi | rtheis: no clue | 13:21 |
| rtheis | me either | 13:21 |
| regXboi | rtheis: I can't believe that rearranging comment blocks would trip it suddenly | 13:21 |
| regXboi | rtheis: so I'm wondering if it's something upstream or something that merged and we missed | 13:21 |
| rtheis | I see the failure on another patch set too | 13:21 |
| regXboi | let's pull out logstash | 13:22 |
| rtheis | regXboi: I didn't see anything obvious in neutron commits | 13:22 |
| regXboi | yeah, that's what bothers me | 13:22 |
| regXboi | logstash says it's only showing up in our jobs | 13:24 |
| regXboi | in the past 7 days | 13:24 |
| regXboi | so that says something merged that broke things | 13:24 |
| russellb | seems to be every run against networking-ovn right now | 13:29 |
| regXboi | yep... let's see if it's geographic | 13:30 |
| rtheis | regXboi: how do you check logstash to find this information? | 13:31 |
| regXboi | nope ... it's spread around the clouds | 13:31 |
| regXboi | rtheis: what I did was to search on the following: | 13:31 |
| regXboi | message:"Current status: REBUILD. Current task state: rebuild_spawning" | 13:31 |
| regXboi | and then set the time to 7 days to 5 minutes ago | 13:31 |
| regXboi | and then start looking at the micro-analysis | 13:31 |
| *** jckasper has quit IRC | 13:32 | |
| regXboi | but I don't profess to be a logstash expert | 13:33 |
| *** jckasper has joined #openstack-neutron-ovn | 13:33 | |
| *** regXboi has quit IRC | 13:34 | |
| *** regXboi has joined #openstack-neutron-ovn | 13:34 | |
| *** a_ta has joined #openstack-neutron-ovn | 14:01 | |
| *** brad_behle has joined #openstack-neutron-ovn | 14:06 | |
| *** regXboi has quit IRC | 14:19 | |
| *** stenzel has quit IRC | 14:19 | |
| *** lrichard has quit IRC | 14:26 | |
| *** regXboi has joined #openstack-neutron-ovn | 14:27 | |
| *** lrichard has joined #openstack-neutron-ovn | 14:27 | |
| *** brad_behle has quit IRC | 14:34 | |
| *** brad_behle has joined #openstack-neutron-ovn | 14:35 | |
| rtheis | good news is that I can recreate the problem with devstack environment by launching an instance with a personality file and then rebuilding it. | 14:37 |
| rtheis | bad news is nothing obvious is in the logs | 14:38 |
| *** gongysh has joined #openstack-neutron-ovn | 14:40 | |
| *** numans has quit IRC | 14:48 | |
| mestery | rtheis: Thanks for looking into this! | 14:50 |
| rtheis | mestery: yw, I reverted nova commits to a couple days ago and now it works | 14:54 |
| mestery | rtheis: Well, there you go :) | 14:54 |
| *** doonhammer has joined #openstack-neutron-ovn | 15:10 | |
| rtheis | well now no luck with recreate :( | 15:12 |
| *** azbiswas has joined #openstack-neutron-ovn | 15:15 | |
| *** salv-orl_ has quit IRC | 15:20 | |
| *** doonhammer has quit IRC | 15:26 | |
| regXboi | well, that sorta says nova broke it - do we need to do a bisect now? | 15:30 |
| *** mickeys has joined #openstack-neutron-ovn | 15:33 | |
| *** pcaruana has quit IRC | 15:38 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 15:40 | |
| *** doonhammer has joined #openstack-neutron-ovn | 15:43 | |
| rtheis | this seems to be timing related and nova was a red herring | 15:47 |
| *** irenab has quit IRC | 15:47 | |
| rtheis | and personality files don't appear to make a difference | 15:48 |
| *** thumpba has joined #openstack-neutron-ovn | 15:48 | |
| rtheis | I can recreate by rebuilding an instance before it is "ready" | 15:48 |
| rtheis | If I rebuild before I can log into the instance via the console then it will fail | 15:49 |
| *** irenab has joined #openstack-neutron-ovn | 15:49 | |
| *** regXboi has quit IRC | 15:50 | |
| *** doonhammer has quit IRC | 15:54 | |
| *** chandrav has joined #openstack-neutron-ovn | 15:56 | |
| *** regXboi has joined #openstack-neutron-ovn | 15:56 | |
| * rtheis wonders why ml2 ovs isn't hitting this... maybe time to test that scenario | 15:56 | |
| regXboi | ack | 15:57 |
| *** a_ta has quit IRC | 15:59 | |
| *** a_ta has joined #openstack-neutron-ovn | 16:02 | |
| *** gongysh has quit IRC | 16:07 | |
| * rtheis thinks it is related to neutron provisioning blocks for port activation | 16:08 | |
| *** arosen12 has joined #openstack-neutron-ovn | 16:35 | |
| *** banix has joined #openstack-neutron-ovn | 16:48 | |
| *** gangil has joined #openstack-neutron-ovn | 16:50 | |
| mestery | rtheis: Still thinking it's port activation? | 16:55 |
| rtheis | I do | 16:55 |
| rtheis | but trying to figure out why ovs doesn't have the same problem since it also uses provisioning blocks | 16:56 |
| regXboi | could the order of events be different in the two scenarios? | 16:57 |
| * regXboi just guessing at this point | 16:57 | |
| rtheis | I hacked something that worked with OVN by removing the provisioning block and immediately marking port as active on binding | 16:57 |
| russellb | we can do that temporarily if needed to unblock the gate | 16:58 |
| russellb | though in theory the 'up' state is supposed to give us what we need, right? | 16:58 |
| rtheis | I think so | 16:58 |
| rtheis | I'm studying an ovs environment now | 16:59 |
| *** salv-orl_ has joined #openstack-neutron-ovn | 16:59 | |
| rtheis | regXboi: I think the ordering may be part of it...ovs seems to activate faster and may let nova clear vif binding phase faster | 17:02 |
| *** salv-orlando has quit IRC | 17:02 | |
| regXboi | rtheis: I can see that | 17:02 |
| *** brad_behle has quit IRC | 17:02 | |
| rtheis | I'm going to try delaying ovs port activation to see if I can recreate there | 17:03 |
| regXboi | ack | 17:06 |
| azbiswas | OVN should be able to mark the port as "up/active" within the timeframe of the timeout. | 17:16 |
| rtheis | It does | 17:17 |
| azbiswas | so the problem is that neutron doesn't see it? | 17:17 |
| rtheis | neutron does see it but nova doesn't get the event | 17:19 |
| *** regXboi has quit IRC | 17:30 | |
| *** regXboi has joined #openstack-neutron-ovn | 17:38 | |
| *** s3wong has joined #openstack-neutron-ovn | 17:43 | |
| *** arosen12 has quit IRC | 17:44 | |
| *** thumpba_ has joined #openstack-neutron-ovn | 17:46 | |
| *** thumpba has quit IRC | 17:48 | |
| *** regXboi has quit IRC | 17:54 | |
| *** regXboi has joined #openstack-neutron-ovn | 17:54 | |
| *** yamamoto has quit IRC | 17:57 | |
| *** openstackgerrit has quit IRC | 18:03 | |
| *** openstackgerrit has joined #openstack-neutron-ovn | 18:03 | |
| *** a_ta has quit IRC | 18:15 | |
| *** jckasper has quit IRC | 18:15 | |
| *** jckasper has joined #openstack-neutron-ovn | 18:16 | |
| *** a_ta_ has joined #openstack-neutron-ovn | 18:19 | |
| *** jckasper has quit IRC | 18:20 | |
| *** a_ta_ has quit IRC | 18:23 | |
| *** jckasper has joined #openstack-neutron-ovn | 18:27 | |
| rtheis | I can't recreate with ovs so collected a port provisioning timeline and heading back over to ovn | 18:38 |
| *** openstackstatus has quit IRC | 18:40 | |
| *** openstack has joined #openstack-neutron-ovn | 18:42 | |
| *** openstackstatus has joined #openstack-neutron-ovn | 18:42 | |
| *** ChanServ sets mode: +v openstackstatus | 18:42 | |
| *** regXboi has quit IRC | 18:46 | |
| *** fzdarsky has quit IRC | 18:46 | |
| *** salv-orl_ has quit IRC | 18:49 | |
| *** yamamoto has joined #openstack-neutron-ovn | 18:58 | |
| rtheis | So ovn doesn't report that the port is down and thus no subsequent up ... | 18:59 |
| russellb | is always up? or never changes to up? or? | 19:00 |
| *** yamamoto has quit IRC | 19:00 | |
| rtheis | launch instances will go from down to up | 19:00 |
| *** yamamoto has joined #openstack-neutron-ovn | 19:00 | |
| rtheis | rebuild instance doesn't trigger port going down | 19:00 |
| russellb | ahhh | 19:01 |
| rtheis | thus no up event | 19:01 |
| *** regXboi has joined #openstack-neutron-ovn | 19:01 | |
| mestery | interesting | 19:01 |
| russellb | well... | 19:01 |
| *** fzdarsky has joined #openstack-neutron-ovn | 19:01 | |
| russellb | if rebuild means deleing a port from br-int, and then re-adding it, seeing down then up is what i would expect | 19:02 |
| russellb | unless the system is so overloaded it's recreated before OVN notices and reports all of it happening | 19:02 |
| russellb | maybe changing to down is broken in general ... i can check | 19:02 |
| rtheis | down seems to be broken | 19:03 |
| russellb | agreed | 19:03 |
| russellb | yay ovn bug | 19:03 |
| russellb | you just saved me from working on a document | 19:03 |
| rtheis | I just shut down an instance and the port still reports up | 19:03 |
| rtheis | :) | 19:03 |
| russellb | ovn-controller isn't clearing the chassis column on Port_Binding when a port gets deleted | 19:04 |
| russellb | this would be handled in ovn/controller/binding.c for those following along at home | 19:04 |
| rtheis | recent commit changed this? | 19:04 |
| regXboi | ok, so I found something real - I guess that was worth something :) | 19:05 |
| regXboi | oh | 19:05 |
| regXboi | um yeah | 19:05 |
| regXboi | so I've likely broken it ;) | 19:05 |
| regXboi | mestery: I'll have to take a pass on that performance stuff | 19:05 |
| regXboi | because ovn/controller/binding.c - it *has* changed | 19:05 |
| regXboi | and recently too | 19:05 |
| russellb | yep, probably regXboi's patch | 19:06 |
| regXboi | ok, so this is on my place to go look | 19:06 |
| regXboi | erplate | 19:06 |
| mestery | Nice work regXboi, we let you get a commit into OVS and you break everything | 19:07 |
| mestery | :) | 19:07 |
| regXboi | mestery: it's that anti-midas touch | 19:07 |
| mestery | I think it is | 19:07 |
| regXboi | so, russellb, I may need to ask some questions here as to what triggers the down event | 19:09 |
| * rtheis just glad I don't have to fix neutron port provisioning in networking-ovn again | 19:09 | |
| russellb | Port_Binding chassis column being set to empty | 19:09 |
| russellb | ovn-controller isn't properly handling the ovs port going away | 19:09 |
| russellb | and then updating Port_Binding to clear chassis as a result | 19:09 |
| regXboi | russellb: ok, let me look at the code some more - I thought I caught that correctly | 19:10 |
| russellb | i'll add a test case for what's broken too | 19:12 |
| regXboi | please | 19:12 |
| regXboi | actually, to ovs unit tests? | 19:12 |
| russellb | to ovs, yes | 19:12 |
| regXboi | ack - that will help | 19:12 |
| regXboi | I'll base the fix on that patch | 19:13 |
| regXboi | russellb: you are talking about this piece of code from before the patch, yes? | 19:15 |
| *** salv-orlando has joined #openstack-neutron-ovn | 19:16 | |
| regXboi | if (ctx->ovnsb_idl_txn) { | 19:16 |
| regXboi | VLOG_INFO("Releasing lport %s from this chassis.", | 19:16 |
| regXboi | binding_rec->logical_port); | 19:16 |
| regXboi | sbrec_port_binding_set_chassis(binding_rec, NULL); | 19:16 |
| regXboi | } | 19:16 |
| russellb | yes, that's what would trigger "up" being set to false in Logical_Switch_Port of OVN_Northbound | 19:16 |
| russellb | when ovn-northd sees that change happen | 19:17 |
| regXboi | ah - I see what's wrong - it's in the wrong place | 19:17 |
| regXboi | my bad | 19:17 |
| regXboi | or I should say, it isn't in enough places | 19:18 |
| *** a_ta has joined #openstack-neutron-ovn | 19:20 | |
| *** azbiswas has quit IRC | 19:20 | |
| regXboi | so this is going to be tricky | 19:25 |
| regXboi | the issue is that the list of ports coming out of the iface_ids has changed | 19:25 |
| regXboi | and I wasn't looking to check that | 19:25 |
| *** a_ta has quit IRC | 19:26 | |
| regXboi | russellb: would it be ok to persist the list of local iface ids and use that to handle this case? | 19:26 |
| russellb | i guess so | 19:27 |
| russellb | i'm not deeply familiar with your changes | 19:27 |
| regXboi | let me put together the change and make sure it passes your test and then let's find somebody to help push it in | 19:28 |
| regXboi | russellb: the problem is that this code is looking for changes in the port binding table - it's not correctly handling changes in the local port list | 19:29 |
| *** yamamoto has quit IRC | 19:29 | |
| *** fzdarsky has quit IRC | 19:39 | |
| regXboi | ok russellb, I have a patch to test - do you have tests for it? | 19:48 |
| russellb | regXboi: i do | 19:48 |
| regXboi | has it merged already or is it just in the queue? | 19:48 |
| russellb | local, i was about to post to the list | 19:48 |
| regXboi | ok, post it to the list with the comment that it will fail if merged | 19:49 |
| russellb | one sec and i'll just post ... i have a test case and the reverts necessary to make it pass (not that we need to merge the reverts, just showing the issue) | 19:49 |
| regXboi | ack - hold the reverts in case my idea doesn't work | 19:49 |
| regXboi | or post the reverts with the comment that I'm working the patch to fix | 19:50 |
| russellb | done | 19:52 |
| russellb | posted reverts with comment that you're working on fix | 19:52 |
| russellb | + patch with a test case | 19:52 |
| russellb | you can also test manually by doing something like .... | 19:52 |
| russellb | $ make sandbox SANDBOXFLAGS="--ovn" | 19:52 |
| russellb | $ ovn/env1/setup.sh | 19:52 |
| russellb | $ ovn-nbctl lsp-get-up sw0-port1 | 19:53 |
| russellb | up | 19:53 |
| russellb | $ ovs-vsctl del-port br-int lport1 | 19:53 |
| russellb | $ ovn-nbctl lsp-get-up sw0-port1 | 19:53 |
| russellb | down | 19:53 |
| russellb | ^^^^^ will be "down" when fixed | 19:53 |
| russellb | "up" while still broken | 19:53 |
| regXboi | ok, cool - I'm running compile and unit tests to make sure I have everything else fixed | 19:53 |
| regXboi | er I didn't break anything else | 19:53 |
| regXboi | and then I'll try that and if good, I'll post | 19:54 |
| russellb | k | 19:54 |
| * russellb goes back to doc hacking.... | 19:54 | |
| *** a_ta has joined #openstack-neutron-ovn | 19:55 | |
| regXboi | me goes to clean up mess he made | 19:55 |
| russellb | :) | 19:55 |
| russellb | all good | 19:55 |
| russellb | our CI did its job! | 19:55 |
| mestery | Yay to the CI! | 19:56 |
| regXboi | yeah, but an OVS test would have been better | 19:57 |
| regXboi | so we'll get that in too | 19:57 |
| regXboi | ok, existing unit tests running | 19:59 |
| regXboi | russellb: it passes the above test :) | 20:00 |
| regXboi | now to make sure I didn't break anything else and I'll send it up | 20:01 |
| russellb | passes the new test i added? | 20:01 |
| russellb | or? | 20:01 |
| regXboi | I mean the by hand text you gave | 20:01 |
| regXboi | here | 20:01 |
| russellb | ohhh, got it | 20:01 |
| russellb | cool | 20:01 |
| russellb | that's a good sign | 20:01 |
| regXboi | I'm running it through the rest of the unit tests to make sure I didn't break something else | 20:01 |
| regXboi | and then I'll put it on top of your test and try it again | 20:01 |
| russellb | i double checked that the manual test i gave here does fail without changes | 20:03 |
| russellb | so yeah, you probably got it | 20:03 |
| regXboi | well, I'm being paranoid | 20:03 |
| regXboi | having broken it once :) | 20:04 |
| russellb | :) | 20:04 |
| regXboi | ugh... the Flow IPFIX sanity check test is just sitting here | 20:08 |
| regXboi | so let's take the chance and put your patch in | 20:09 |
| russellb | make check TESTSUITEFLAGS="-k ovn" | 20:14 |
| russellb | that's exhaustive enough :-) | 20:14 |
| regXboi | ack | 20:14 |
| regXboi | the new test is in the e2e space? | 20:15 |
| * regXboi hates raceful tests | 20:16 | |
| regXboi | 2065: ovn -- port state up and down ok | 20:16 |
| russellb | \o/ | 20:16 |
| regXboi | not so fast - had an e2e case fail | 20:16 |
| regXboi | need to make sure its a race case | 20:16 |
| russellb | yeah, the issue is ovn-controller, but the test is the state reflected up through the northbound db | 20:16 |
| russellb | the same way networking-ovn sees the failure | 20:16 |
| regXboi | aw come on... you can't be really failing on me, can you? | 20:17 |
| russellb | i suggest a hammer | 20:18 |
| regXboi | you can, but I think I've broken 3 HVs, 3 LS, 3 lports/LS, 1 LR | 20:19 |
| russellb | sadness | 20:19 |
| *** mickeys has quit IRC | 20:20 | |
| *** jckasper has quit IRC | 20:23 | |
| *** jckasper has joined #openstack-neutron-ovn | 20:23 | |
| lrichard | regXboi: I think "3 HVs, 3 LS, 3 lports/LS, 1 LR" has a history of failing occasionally anyway (racy) | 20:26 |
| regXboi | it does | 20:26 |
| regXboi | so I'm essentially rechecking while I look at code | 20:26 |
| *** jckasper has quit IRC | 20:27 | |
| *** azbiswas has joined #openstack-neutron-ovn | 20:28 | |
| *** yamamoto has joined #openstack-neutron-ovn | 20:30 | |
| regXboi | yeah, this patch broke it | 20:30 |
| regXboi | ok, now why on god's green earth | 20:30 |
| *** jckasper has joined #openstack-neutron-ovn | 20:31 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 20:31 | |
| lrichard | meanwhile, the last 3 travis-ci builds failed with: | 20:33 |
| lrichard | 1051: ofproto-dpif - Flow IPFIX sanity check | 20:33 |
| lrichard | No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself. | 20:33 |
| lrichard | The build has been terminated | 20:33 |
| regXboi | that's where I was failing, so that's not on me :) | 20:34 |
| russellb | lrichard: hm, i think an ipfix related patch just merged? | 20:34 |
| lrichard | russelb: yes, the first of these was an ipfix commit | 20:34 |
| lrichard | trying to reproduce now | 20:34 |
| russellb | yay Friiday! | 20:34 |
| regXboi | no kissinf | 20:34 |
| regXboi | er kidding | 20:35 |
| lrichard | joy! | 20:35 |
| *** jckasper has quit IRC | 20:35 | |
| * regXboi muses it's something in persisting the local ports, but what | 20:35 | |
| lrichard | ipfix is unhappy 1052: ofproto-dpif - Flow IPFIX sanity check - tunnel set FAILED (ofproto-dpif.at:6041) | 20:36 |
| regXboi | HA - IT IS A RACE CONDITION | 20:37 |
| regXboi | 2056: ovn -- 3 HVs, 3 LS, 3 lports/LS, 1 LR ok | 20:37 |
| regXboi | git status | 20:37 |
| regXboi | On branch test | 20:37 |
| regXboi | nothing to commit, working directory clean | 20:37 |
| regXboi | ok sending the patch, russellb | 20:37 |
| russellb | ok | 20:37 |
| regXboi | you are russellb@ovn.org? | 20:38 |
| russellb | russell@ | 20:39 |
| regXboi | ack sent | 20:39 |
| regXboi | and it's landed http://patchwork.ozlabs.org/patch/640417/ | 20:40 |
| mestery | Nice work regXboi regXboi lrichard :) | 20:42 |
| *** yamamoto has quit IRC | 20:42 | |
| mestery | Teamwork #ftw! | 20:42 |
| russellb | regXboi got double credit!! | 20:42 |
| * mestery gives regXboi a gold start | 20:42 | |
| regXboi | mestery: not me twice: russellb and rtheis | 20:42 |
| mestery | *star | 20:42 |
| mestery | Oh yes! | 20:42 |
| * mestery hands rtheis two gold stars | 20:43 | |
| mestery | He was the one who found the regression :) | 20:43 |
| regXboi | but #teamwork_rocks | 20:43 |
| rtheis | yes #teamwork_rocks | 20:43 |
| *** mickeys has joined #openstack-neutron-ovn | 20:43 | |
| regXboi | now can I get an acked by :) | 20:47 |
| *** fzdarsky has quit IRC | 20:48 | |
| * flaviof re-reading buffer | 20:49 | |
| flaviof | regXboi: i see the race too. I normally run the racy test and see it pass: make check TESTSUITEFLAGS="$TESTNUMBER" | 20:50 |
| regXboi | flaviof: yes, but this one took a *LOT* of re-runs - I was into looking for what I broke | 20:51 |
| flaviof | ack | 20:51 |
| regXboi | which made no sense to me | 20:51 |
| * regXboi suspects there may be other races I'll be uncovering .... :-/ | 20:52 | |
| * flaviof never run ofproto*.at .... normally stay on "-k ovn" | 20:52 | |
| regXboi | flaviof: I tend to run everything - just in case | 20:54 |
| russellb | regXboi: i'll let ben review your fix since it isn't a trivial fix and he reviewed the original code | 20:55 |
| russellb | thanks for working on it quickly :) | 20:55 |
| *** regXboi has quit IRC | 20:56 | |
| *** regXboi has joined #openstack-neutron-ovn | 20:58 | |
| regXboi | russellb: sure | 20:58 |
| regXboi | russellb: besides, as the committer that pushed it in, it does fall to him if I hadn't had a patch ready | 20:59 |
| russellb | yep | 20:59 |
| regXboi | but "I broke it, I fix it" is how I roll | 20:59 |
| flaviof | regXboi++ | 21:03 |
| flaviof | regXboi: if you are not breaking anything, they you are not doing anything | 21:03 |
| flaviof | :) | 21:03 |
| regXboi | well, that's always a risk | 21:04 |
| regXboi | and I can say that it did pass unit tests, but that's cold comfort | 21:04 |
| rtheis | russellb: OpenStack mailing list has a python 3 discussion, and I noticed networking-ovn doesn't have a py34 job. Is it time to add one? | 21:05 |
| russellb | yes, we should | 21:05 |
| russellb | there's a version of the ovs python lib on pypi that works with py3 now | 21:05 |
| russellb | so we should be able to | 21:05 |
| rtheis | cool, I'll open a bug to track it | 21:06 |
| russellb | ++ | 21:06 |
| *** yamamoto has joined #openstack-neutron-ovn | 21:11 | |
| russellb | have a nice weekend, everyone | 21:11 |
| rtheis | you too | 21:13 |
| regXboi | well.. he's gone, but the patch to fix things landed at OVS | 21:13 |
| regXboi | rtheis, shall we try a recheck to see what we see? | 21:14 |
| rtheis | yep | 21:14 |
| rtheis | rechecking https://review.openstack.org/#/c/333584/ now | 21:14 |
| regXboi | I'm running a reheck on 330751 | 21:14 |
| *** yamamoto has quit IRC | 21:15 | |
| regXboi | and have them queued up on the status page | 21:15 |
| *** banix has quit IRC | 21:35 | |
| regXboi | meh | 21:35 |
| regXboi | jobs just got re-queued | 21:35 |
| * rtheis runs manual tests now | 21:41 | |
| rtheis | oh yeah...port status is now down when shutting down the instance | 21:41 |
| regXboi | ack | 21:42 |
| regXboi | so hopefully this will pass | 21:42 |
| rtheis | manual recreate scenario using rebuild also passes | 21:44 |
| *** a_ta has quit IRC | 21:44 | |
| rtheis | I think we are good to go | 21:44 |
| rtheis | regXboi: have a good weekend | 21:44 |
| *** regXboi has quit IRC | 21:52 | |
| *** mickeys has quit IRC | 21:54 | |
| *** regXboi has joined #openstack-neutron-ovn | 21:54 | |
| *** mickeys has joined #openstack-neutron-ovn | 22:02 | |
| *** regXboi has quit IRC | 22:06 | |
| *** regXboi has joined #openstack-neutron-ovn | 22:07 | |
| *** regXboi has quit IRC | 22:07 | |
| *** rtheis has quit IRC | 22:14 | |
| *** thumpba_ has quit IRC | 22:24 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:24 | |
| *** thumpba has quit IRC | 22:24 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:25 | |
| *** thumpba has quit IRC | 22:25 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:25 | |
| *** thumpba has quit IRC | 22:26 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:26 | |
| *** thumpba has quit IRC | 22:27 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:27 | |
| *** thumpba has quit IRC | 22:27 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:28 | |
| *** thumpba has quit IRC | 22:28 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:28 | |
| *** thumpba has quit IRC | 22:29 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:29 | |
| *** thumpba has quit IRC | 22:30 | |
| *** thumpba_ has joined #openstack-neutron-ovn | 22:30 | |
| *** thumpba_ has quit IRC | 22:30 | |
| *** thumpba has joined #openstack-neutron-ovn | 22:45 | |
| *** thumpba has quit IRC | 22:45 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 22:58 | |
| *** salv-orlando has quit IRC | 23:01 | |
| *** thumpba has joined #openstack-neutron-ovn | 23:06 | |
| *** thumpba has quit IRC | 23:06 | |
| *** banix has joined #openstack-neutron-ovn | 23:17 | |
| *** mickeys has quit IRC | 23:24 | |
| *** banix has quit IRC | 23:38 | |
| *** banix has joined #openstack-neutron-ovn | 23:39 | |
| *** banix has quit IRC | 23:46 | |
| *** salv-orl_ has quit IRC | 23:47 | |
| *** thumpba has joined #openstack-neutron-ovn | 23:59 | |
| *** thumpba has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!