| *** gdubreui has joined #openstack-neutron | 00:02 | |
| *** networkstatic has quit IRC | 00:05 | |
| *** matsuhashi has joined #openstack-neutron | 01:02 | |
| *** networkstatic has joined #openstack-neutron | 01:43 | |
| *** scotty916 has joined #openstack-neutron | 01:49 | |
| *** scotty916 has quit IRC | 01:52 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Include lswitch id in NSX plugin port mappings https://review.openstack.org/54578 | 01:53 |
|---|---|---|
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Add DB mappings with NSX logical switches https://review.openstack.org/54579 | 01:53 |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Add DB mappings with NSX logical routers https://review.openstack.org/54584 | 01:53 |
| *** scotty916 has joined #openstack-neutron | 01:54 | |
| scotty916 | #openstack | 01:56 |
| *** ljjjustin has joined #openstack-neutron | 02:07 | |
| *** scotty916 has quit IRC | 02:12 | |
| *** scotty916 has joined #openstack-neutron | 02:13 | |
| *** scotty916 has quit IRC | 02:22 | |
| *** scotty916 has joined #openstack-neutron | 02:24 | |
| *** scotty916 has quit IRC | 02:27 | |
| *** scotty916 has joined #openstack-neutron | 02:28 | |
| *** Alssi has joined #openstack-neutron | 02:35 | |
| openstackgerrit | Yohei Matsuhashi proposed a change to openstack/python-neutronclient: Enable to select specific network service_type https://review.openstack.org/54534 | 02:36 |
| *** scotty916 has quit IRC | 02:37 | |
| *** ljjjustin has quit IRC | 02:49 | |
| *** ljjjustin has joined #openstack-neutron | 03:01 | |
| *** coolsvap has joined #openstack-neutron | 03:06 | |
| *** matsuhashi has quit IRC | 03:19 | |
| *** matsuhashi has joined #openstack-neutron | 03:19 | |
| *** matsuhashi has quit IRC | 03:24 | |
| *** dims has quit IRC | 03:25 | |
| *** matsuhashi has joined #openstack-neutron | 03:31 | |
| *** gongysh has joined #openstack-neutron | 03:56 | |
| *** matsuhashi has quit IRC | 03:57 | |
| *** matsuhashi has joined #openstack-neutron | 03:57 | |
| openstackgerrit | A change was merged to openstack/neutron: ExtraRoute: fix _get_extra_routes_by_router_id() https://review.openstack.org/55532 | 04:01 |
| *** matsuhashi has quit IRC | 04:02 | |
| *** ian_ott has joined #openstack-neutron | 04:12 | |
| *** jecarey has joined #openstack-neutron | 04:17 | |
| *** ljjjustin has quit IRC | 04:40 | |
| *** chandankumar has joined #openstack-neutron | 04:45 | |
| *** jecarey has quit IRC | 04:47 | |
| *** scotty916 has joined #openstack-neutron | 04:55 | |
| *** matsuhashi has joined #openstack-neutron | 05:01 | |
| *** scotty916 has quit IRC | 05:12 | |
| *** marun has joined #openstack-neutron | 05:17 | |
| *** scotty916 has joined #openstack-neutron | 05:24 | |
| *** scotty916 has quit IRC | 05:31 | |
| *** wenjianhn has joined #openstack-neutron | 05:45 | |
| *** matsuhashi has quit IRC | 05:56 | |
| *** matsuhashi has joined #openstack-neutron | 05:59 | |
| *** scotty916 has joined #openstack-neutron | 05:59 | |
| *** scotty916 has quit IRC | 06:03 | |
| *** ktbenton_ has quit IRC | 06:05 | |
| *** ktbenton_ has joined #openstack-neutron | 06:07 | |
| *** ljjjustin has joined #openstack-neutron | 06:10 | |
| openstackgerrit | Zhongyue Luo proposed a change to openstack/neutron: Use DictOpt for mapping options https://review.openstack.org/45990 | 06:29 |
| openstackgerrit | Zhongyue Luo proposed a change to openstack/neutron: Use DictOpt for mapping options https://review.openstack.org/45990 | 06:30 |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Imported Translations from Transifex https://review.openstack.org/54680 | 06:35 |
| *** amritanshu_RnD has joined #openstack-neutron | 06:47 | |
| *** matsuhashi has quit IRC | 06:56 | |
| *** matsuhashi has joined #openstack-neutron | 06:57 | |
| *** gdubreui has quit IRC | 06:57 | |
| *** matsuhashi has quit IRC | 06:57 | |
| *** matsuhashi has joined #openstack-neutron | 06:57 | |
| *** bvandenh_ has quit IRC | 07:02 | |
| *** matsuhashi has quit IRC | 07:08 | |
| *** matsuhashi has joined #openstack-neutron | 07:09 | |
| *** matsuhas_ has joined #openstack-neutron | 07:12 | |
| *** matsuhashi has quit IRC | 07:12 | |
| openstackgerrit | Joe Mills proposed a change to openstack/neutron: Midonet to support port association at floating IP creation https://review.openstack.org/55792 | 07:15 |
| openstackgerrit | li,chen proposed a change to openstack/neutron: Remove the warning for Scheduling Network https://review.openstack.org/55793 | 07:20 |
| openstackgerrit | li,chen proposed a change to openstack/neutron: Remove the warning for Scheduling Network https://review.openstack.org/55793 | 07:26 |
| openstackgerrit | A change was merged to openstack/neutron: Don't shadow str https://review.openstack.org/55740 | 07:26 |
| *** matsuhas_ has quit IRC | 07:32 | |
| *** matsuhashi has joined #openstack-neutron | 07:32 | |
| *** jlibosva has joined #openstack-neutron | 07:36 | |
| *** LiJiansheng has joined #openstack-neutron | 08:22 | |
| *** amuller has joined #openstack-neutron | 08:23 | |
| *** lijunwu has joined #openstack-neutron | 08:28 | |
| lijunwu | ping LiJiansheng | 08:28 |
| *** matsuhashi has quit IRC | 08:29 | |
| *** matsuhashi has joined #openstack-neutron | 08:30 | |
| *** marun has quit IRC | 08:31 | |
| *** marun has joined #openstack-neutron | 08:32 | |
| *** matsuhashi has quit IRC | 08:34 | |
| *** matsuhashi has joined #openstack-neutron | 08:37 | |
| *** jpich has joined #openstack-neutron | 08:41 | |
| openstackgerrit | berlin proposed a change to openstack/neutron: Fix adding router_id return attribute when CRU firewall and vip obj https://review.openstack.org/48568 | 08:52 |
| *** hua has quit IRC | 09:06 | |
| *** rossella_s has joined #openstack-neutron | 09:11 | |
| *** afazekas has joined #openstack-neutron | 09:19 | |
| *** networkstatic is now known as networkstatic_zZ | 09:26 | |
| openstackgerrit | Monty Taylor proposed a change to openstack/neutron: Replace stubout with fixtures https://review.openstack.org/55805 | 09:32 |
| openstackgerrit | Arata Notsu proposed a change to openstack/neutron: Do not run "ovs-ofctl add-flow" with an invalid in_port https://review.openstack.org/53636 | 09:44 |
| *** lijunwu has quit IRC | 09:46 | |
| *** gdubreui has joined #openstack-neutron | 09:59 | |
| *** gdubreui has quit IRC | 10:05 | |
| *** gdubreui has joined #openstack-neutron | 10:05 | |
| *** ramishra has joined #openstack-neutron | 10:06 | |
| *** gongysh has quit IRC | 10:11 | |
| *** rossella_s has quit IRC | 10:37 | |
| *** rossella_s has joined #openstack-neutron | 10:37 | |
| *** ljjjustin has quit IRC | 10:40 | |
| *** ppenjoy has joined #openstack-neutron | 10:47 | |
| *** ppenjoy has left #openstack-neutron | 10:47 | |
| *** LiJiansheng has quit IRC | 10:57 | |
| *** matsuhashi has quit IRC | 11:14 | |
| *** matsuhashi has joined #openstack-neutron | 11:14 | |
| *** gdubreui has quit IRC | 11:15 | |
| *** matsuhashi has quit IRC | 11:15 | |
| *** matsuhas_ has joined #openstack-neutron | 11:15 | |
| *** LiJiansheng has joined #openstack-neutron | 11:19 | |
| *** matsuhas_ has quit IRC | 11:25 | |
| *** matsuhashi has joined #openstack-neutron | 11:26 | |
| openstackgerrit | Arata Notsu proposed a change to openstack/neutron: Do not run "ovs-ofctl add-flow" with an invalid in_port https://review.openstack.org/53636 | 11:26 |
| *** pcm_ has joined #openstack-neutron | 11:27 | |
| *** matsuhashi has quit IRC | 11:30 | |
| *** matsuhashi has joined #openstack-neutron | 11:31 | |
| openstackgerrit | Arata Notsu proposed a change to openstack/neutron: Remove confusing comment and code for LBaaS https://review.openstack.org/55822 | 11:44 |
| *** amritanshu_RnD has quit IRC | 11:50 | |
| *** amritanshu_RnD has joined #openstack-neutron | 11:51 | |
| *** coolsvap has quit IRC | 11:57 | |
| *** yamahata has joined #openstack-neutron | 12:07 | |
| openstackgerrit | Assaf Muller proposed a change to openstack/python-neutronclient: [fwaas] Can now create disabled firewall rules https://review.openstack.org/55826 | 12:13 |
| *** ramishra has quit IRC | 12:24 | |
| openstackgerrit | Akihiro Motoki proposed a change to openstack/neutron: Replace mox in unit tests with mock https://review.openstack.org/54044 | 12:31 |
| openstackgerrit | Yohei Matsuhashi proposed a change to openstack/python-neutronclient: Enable to select specific network service type https://review.openstack.org/54534 | 12:34 |
| *** amrit_ has joined #openstack-neutron | 12:47 | |
| *** LiJiansheng has quit IRC | 12:54 | |
| EmilienM | salv-orlando: ping | 12:57 |
| salv-orlando | hi EmilienM - how are you? | 12:57 |
| EmilienM | salv-orlando: fine, thx! you ? | 12:57 |
| salv-orlando | Fine as anybody could be fine on a monday... | 12:57 |
| EmilienM | :) | 12:57 |
| EmilienM | salv-orlando: i was about starting some work on tempest, what about negative tests for test_routers ? | 12:58 |
| EmilienM | want to start something simple first | 12:58 |
| salv-orlando | Yeah, that would be good. We're also lacking coverage on tests for floating ups, that would be a next step | 12:59 |
| EmilienM | salv-orlando: ok. Let me create blueprints and start work on it | 12:59 |
| salv-orlando | Ask somebody in openstack-qa if you need to create blueprints or bugs for this | 13:00 |
| EmilienM | salv-orlando: ack | 13:00 |
| salv-orlando | and if you can tag everything as 'neutron', that would make easier for us to find them in tempest | 13:00 |
| EmilienM | make sense | 13:00 |
| *** matsuhashi has quit IRC | 13:01 | |
| *** matsuhashi has joined #openstack-neutron | 13:02 | |
| *** matsuhashi has quit IRC | 13:04 | |
| *** matsuhashi has joined #openstack-neutron | 13:04 | |
| openstackgerrit | A change was merged to openstack/neutron: Simplify using external routers and metadata https://review.openstack.org/50292 | 13:06 |
| EmilienM | salv-orlando: <mkoderer> EmilienM: but negative test are currently under discussion in tempest | 13:06 |
| EmilienM | salv-orlando: <mkoderer> EmilienM: so I wouldn't spent so much effort in it | 13:07 |
| EmilienM | uh? | 13:07 |
| *** ppenjoy has joined #openstack-neutron | 13:07 | |
| salv-orlando | oh well, tempest is another project where people have plenty of opinions, just like neutron ;) | 13:07 |
| EmilienM | salv-orlando: sure | 13:08 |
| salv-orlando | I do not recall this specific discussion, must be relatively new | 13:08 |
| EmilienM | salv-orlando: i don't mind, let's improve test coverage anyway | 13:08 |
| sdague | salv-orlando: yeh, this is new, we got flooded with negative tests during RC, so we were trying to cut back and come up with a generative approach instead | 13:08 |
| sdague | however, in the neutron case I think we can make an exception | 13:09 |
| *** openstackgerrit has quit IRC | 13:09 | |
| salv-orlando | sdague: I think there are so little negative tests for neutron that some can help a lot while we get to a point where they can be generated | 13:09 |
| sdague | yep, agreed | 13:10 |
| *** openstackgerrit has joined #openstack-neutron | 13:10 | |
| sdague | I'm going to suggest that | 13:10 |
| salv-orlando | ok thanks a lot | 13:10 |
| *** x86brandon has quit IRC | 13:17 | |
| *** ppenjoy has left #openstack-neutron | 13:22 | |
| *** carl_baldwin has joined #openstack-neutron | 13:57 | |
| *** dims has joined #openstack-neutron | 13:59 | |
| rossella_s | salv-orlando EmilienM I am looking into tempest too. I could take care of the tests for floating ips, what do you think? | 14:00 |
| openstackgerrit | Evgeny Fedoruk proposed a change to openstack/neutron: Adding more tests and fixes driver as a result https://review.openstack.org/54155 | 14:00 |
| EmilienM | rossella_s: i started with that finally | 14:01 |
| EmilienM | rossella_s: a first shoot : https://review.openstack.org/55846 | 14:01 |
| EmilienM | but it's a WIP, i'll amend with more tests | 14:01 |
| rossella_s | EmilienM: got it | 14:02 |
| EmilienM | rossella_s: maybe you could go ahead with routers stuffs | 14:03 |
| rossella_s | EmilienM: ok, I'll have a look at that then. I have to get started, it will take some time :) | 14:03 |
| EmilienM | rossella_s: a discussion is going to happen on thursday (qa meeting) about negative testing in neutron, we should follow that | 14:05 |
| rossella_s | EmilienM: thanks for the hint | 14:06 |
| salv-orlando | rossella_s, EmilienM: I reckon neutron will get an exception on negative tests, given sdague's words. However for floating IPs I have a suspect that even the 'positive' tests do not provide enough coverage, so that's something that Rossella could tackle. | 14:07 |
| *** julim has joined #openstack-neutron | 14:08 | |
| *** x86brandon has joined #openstack-neutron | 14:08 | |
| EmilienM | salv-orlando: ok, let's try to break the api then :) | 14:09 |
| rossella_s | salv-orlando: thanks, I will take it then | 14:09 |
| *** matsuhashi has quit IRC | 14:13 | |
| *** aymenfrikha has joined #openstack-neutron | 14:21 | |
| *** openstackgerrit has quit IRC | 14:22 | |
| *** openstackgerrit has joined #openstack-neutron | 14:23 | |
| *** matsuhashi has joined #openstack-neutron | 14:25 | |
| *** larsks has joined #openstack-neutron | 14:30 | |
| *** mestery has quit IRC | 14:31 | |
| *** mestery has joined #openstack-neutron | 14:31 | |
| *** alexpilotti has joined #openstack-neutron | 14:41 | |
| *** gongysh has joined #openstack-neutron | 14:47 | |
| *** amritanshu_RnD has quit IRC | 14:58 | |
| *** amrit_ has quit IRC | 14:58 | |
| *** armax has joined #openstack-neutron | 15:02 | |
| *** networkstatic_zZ has quit IRC | 15:02 | |
| *** networkstatic has joined #openstack-neutron | 15:03 | |
| *** carl_baldwin has quit IRC | 15:06 | |
| *** gongysh has quit IRC | 15:08 | |
| *** jecarey has joined #openstack-neutron | 15:09 | |
| *** nplanel has joined #openstack-neutron | 15:10 | |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Updated from global requirements https://review.openstack.org/54425 | 15:13 |
| *** amuller has quit IRC | 15:14 | |
| openstackgerrit | Amir Sadoughi proposed a change to openstack/neutron: Added check on plugin.supported_extension_aliases https://review.openstack.org/47432 | 15:19 |
| *** matsuhashi has quit IRC | 15:23 | |
| *** matsuhashi has joined #openstack-neutron | 15:23 | |
| *** matsuhashi has quit IRC | 15:28 | |
| *** jlibosva has quit IRC | 15:37 | |
| *** coolsvap has joined #openstack-neutron | 15:41 | |
| *** aymenfrikha has quit IRC | 15:44 | |
| *** carl_baldwin has joined #openstack-neutron | 15:47 | |
| *** ppenjoy has joined #openstack-neutron | 15:59 | |
| *** ppenjoy has left #openstack-neutron | 16:00 | |
| *** aymenfrikha has joined #openstack-neutron | 16:02 | |
| *** opilotte has joined #openstack-neutron | 16:03 | |
| *** afazekas has quit IRC | 16:15 | |
| *** aymenfrikha has quit IRC | 16:23 | |
| *** djoreilly has joined #openstack-neutron | 16:25 | |
| *** HenryG has joined #openstack-neutron | 16:25 | |
| doude | mestery: ping | 16:29 |
| *** aymenfrikha has joined #openstack-neutron | 16:30 | |
| *** mlavalle has joined #openstack-neutron | 16:41 | |
| *** zackf has joined #openstack-neutron | 16:42 | |
| openstackgerrit | A change was merged to openstack/neutron: Remove confusing comment and code for LBaaS https://review.openstack.org/55822 | 16:54 |
| *** manishg has quit IRC | 16:58 | |
| *** annegentle has joined #openstack-neutron | 17:01 | |
| *** marun has quit IRC | 17:07 | |
| *** aymenfrikha has quit IRC | 17:11 | |
| *** carl_baldwin has quit IRC | 17:33 | |
| *** jpich has quit IRC | 17:34 | |
| *** jog0 has joined #openstack-neutron | 17:37 | |
| mestery | doude: pong | 17:42 |
| *** rossella_s has quit IRC | 17:44 | |
| jog0 | https://bugs.launchpad.net/neutron/+bug/1250168 | 17:45 |
| jog0 | the bug above is a regression that is blocking us from gating on the neutron-large-ops test | 17:48 |
| *** carl_baldwin has joined #openstack-neutron | 17:50 | |
| *** manishg has joined #openstack-neutron | 18:13 | |
| *** dsockwell has quit IRC | 18:17 | |
| *** mlavalle has quit IRC | 18:18 | |
| *** briancline has quit IRC | 18:23 | |
| *** Shixiong has joined #openstack-neutron | 18:27 | |
| *** carl_baldwin has quit IRC | 18:32 | |
| openstackgerrit | Joe Gordon proposed a change to openstack/neutron: Cleanup HACKING.rst https://review.openstack.org/55885 | 18:34 |
| *** jlibosva has joined #openstack-neutron | 18:35 | |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Updated from global requirements https://review.openstack.org/54425 | 18:48 |
| *** mlavalle has joined #openstack-neutron | 18:55 | |
| *** Shixiong has quit IRC | 19:08 | |
| *** dsockwell has joined #openstack-neutron | 19:23 | |
| openstackgerrit | Joe Gordon proposed a change to openstack/python-neutronclient: Make HACKING.rst DRYer https://review.openstack.org/55904 | 19:29 |
| *** briancline has joined #openstack-neutron | 19:39 | |
| *** chandankumar has quit IRC | 19:46 | |
| *** otherwiseguy has quit IRC | 19:53 | |
| *** alexpilotti has quit IRC | 20:03 | |
| *** djoreilly has quit IRC | 20:11 | |
| *** carl_baldwin has joined #openstack-neutron | 20:30 | |
| *** davlaps has joined #openstack-neutron | 20:33 | |
| *** davlaps is now known as help | 20:34 | |
| *** help is now known as devlaps | 20:34 | |
| *** devlaps has quit IRC | 20:36 | |
| *** davlaps has joined #openstack-neutron | 20:37 | |
| *** davlaps has quit IRC | 20:40 | |
| *** carl_baldwin has quit IRC | 20:42 | |
| *** carl_baldwin has joined #openstack-neutron | 20:45 | |
| openstackgerrit | Anita Kuno proposed a change to openstack/neutron: Revert "sync log from oslo" https://review.openstack.org/55913 | 20:52 |
| anteaya | EmilienM salv-orlando thanks for addressing tempest, let me know if you encounter any obstacles so I can help | 21:00 |
| salv-orlando | thanks anteaya | 21:03 |
| anteaya | and thank you, rossella_s | 21:03 |
| anteaya | hopefully the reversion of the sync log from oslo patch can get large-ops passing again | 21:03 |
| anteaya | then we can take another shot at that functionality | 21:04 |
| *** x86brandon has quit IRC | 21:06 | |
| salv-orlando | anteaya: do you have a quick gerrit link to this revert patch? | 21:07 |
| anteaya | https://review.openstack.org/#/c/55913/1 | 21:08 |
| anteaya | I have no idea why or how a change to logging would have large ops failing | 21:09 |
| anteaya | but large-ops started failing October 31st: http://graphite.openstack.org/graphlot/?from=-2weeks&until=-1week&target=stats.zuul.pipeline.check.job.gate-tempest-devstack-vm-neutron-large-ops.SUCCESS&target=stats.zuul.pipeline.check.job.gate-tempest-devstack-vm-neutron-large-ops.FAILURE | 21:09 |
| salv-orlando | anteaya: indeed - this is why I asked to see the revert patch! | 21:09 |
| anteaya | and there were only 2 patches merged to neutron that day, this patch and one that was a comment to a file | 21:09 |
| anteaya | great | 21:10 |
| anteaya | I am interested in hearing your thoughts | 21:10 |
| anteaya | I'm wrong, I didn't scroll down far enough | 21:12 |
| anteaya | the other patch might be a candiate as well: http://git.openstack.org/cgit/openstack/neutron/commit/?id=c1577c5954ce4103f81708b4a92569e7ab34af8d | 21:12 |
| anteaya | I can offer a patch to revert this one as well | 21:13 |
| anteaya | not as well, but instead | 21:14 |
| *** julim has quit IRC | 21:14 | |
| salv-orlando | anteaya: I now see I reviewed the original commit for the log sync - I don't see how that could trigger a non-deterministic failure if not for some weird interactions deriving from replacing cstringIO with six.moves | 21:14 |
| salv-orlando | the multi-worker patch is definitely looking more dodgy. | 21:15 |
| salv-orlando | I kept it under stress test for a few days before approving, but I might have missed something - very likely. | 21:15 |
| anteaya | okay, will submit a patch for that and let's see what comes back from the tests | 21:15 |
| anteaya | possibly | 21:15 |
| anteaya | let's try the revert and go from there | 21:16 |
| salv-orlando | sounds reasonable. I know several folks want to use the multi-worker feature, but if we prove that breaks large ops we need to revert and repush when it's safe. | 21:16 |
| *** julim has joined #openstack-neutron | 21:16 | |
| jog0 | anteaya: its not the oslo patch | 21:17 |
| anteaya | jog0: okay | 21:17 |
| jog0 | if you look at the first jenkkins comment https://review.openstack.org/#/c/54666/ | 21:17 |
| jog0 | large-ops passed | 21:17 |
| jog0 | also https://review.openstack.org/#/c/37131/ passed to | 21:20 |
| jog0 | but we only have one datapoint on that :/ | 21:20 |
| salv-orlando | jog0: If I had to choose a culprit based on the nature of the change only, I would pick 37131 | 21:21 |
| jog0 | salv-orlando: agreed | 21:21 |
| salv-orlando | jog0: however, to counter this very same argument, the change introduced by 37131 is not enabled in the gate. This just to create even more confusion, if possible. | 21:22 |
| jog0 | Merge "Avoid dhcp agent race condition on subnet and network delete" | 21:23 |
| jog0 | Fix query error on dhcp release port for postgresql | 21:23 |
| jog0 | are two more candidates | 21:23 |
| jog0 | anteaya: ^ | 21:23 |
| openstackgerrit | Anita Kuno proposed a change to openstack/neutron: Revert "Creates multiple worker processes for API server" https://review.openstack.org/55917 | 21:23 |
| jog0 | (commit message titles from git log) | 21:24 |
| jog0 | salv-orlando: from the commit msgs those both looks possible on the surface | 21:24 |
| salv-orlando | jog0 | 21:25 |
| salv-orlando | : | 21:25 |
| salv-orlando | sorry… I think those two change also introduced other failures in the gate. Some of them have already taken care of, some others are under review. Not sure if they will fix large-ops as well | 21:26 |
| * salv-orlando looking for relevant patches under review | 21:26 | |
| salv-orlando | https://review.openstack.org/#/c/54850/ | 21:26 |
| anteaya | so far 54850 has only had Armando Jenkins and smokestack working on it | 21:28 |
| anteaya | who else might be able to review that one? | 21:28 |
| anteaya | it doesn't always have to be you salv-orlando | 21:29 |
| jog0 | salv-orlando: can we just revert the original two patches | 21:29 |
| salv-orlando | jog0: not sure whether it's a good idea to just revert anything that could have possibly caused the error. Especially since I don't know if anybody (at least I haven't) has thoroughly looked at the logs in order to nail down the issue. | 21:30 |
| salv-orlando | anteaya: I promised Armando a review, but we still need another core. I am sure markmcclain will step in too. | 21:31 |
| jog0 | salv-orlando: true, I just figured we can leaverge openstack-infra to see which one is the troublemaker in this case | 21:31 |
| salv-orlando | I will vote for the "revert everything" approach if we're unable to do a conclusive post-mortem. And even in that case, I would prefer to revert candidate patches one at a time. | 21:32 |
| anteaya | salv-orlando: great thanks | 21:32 |
| jog0 | I am not saying merge the reverts just propose em | 21:32 |
| salv-orlando | jog0: we can leverage openstack-infra to the extent that mtaylor and team do not bark at us for overusing resources! | 21:32 |
| mordred | salv-orlando: resources we have | 21:32 |
| anteaya | so let's leverage away | 21:33 |
| jog0 | mordred: \o/ | 21:33 |
| salv-orlando | mordred: gah I cannot use that excuse anymore | 21:33 |
| anteaya | no excuses left | 21:33 |
| anteaya | we must produce stable results | 21:33 |
| jog0 | anteaya: well we have a stable result, the test does fail consistently ;) | 21:33 |
| anteaya | well there is that | 21:34 |
| anteaya | goal: stability on Monday, achieved | 21:34 |
| *** sgran has quit IRC | 21:34 | |
| jog0 | anteaya: I almost forgot if yuo go to https://review.openstack.org/#/c/54752/ | 21:34 |
| jog0 | you can just click on the revert change button | 21:34 |
| jog0 | and that creates a new patch | 21:34 |
| anteaya | look at that revert change button, how sweet | 21:35 |
| salv-orlando | jog0: I would not revert 54752 | 21:35 |
| anteaya | that will save me a few minutes | 21:35 |
| anteaya | I would like to see what 55917 comes back with first | 21:35 |
| salv-orlando | if you revert 54572 postgres jobs will fail most of the times. | 21:35 |
| anteaya | creating a revert patch harms nothing | 21:36 |
| anteaya | you don't have to merge it | 21:36 |
| anteaya | but we can get the logs on the reversion | 21:36 |
| jog0 | either way, just wanted to point out that button | 21:36 |
| anteaya | button acknowledged | 21:36 |
| anteaya | my gratitude | 21:36 |
| salv-orlando | no worries. I will take a look at devstack patches merged in the same timeframe. Changes in default neutron configuration might have uncovered some bug which was always in the source code. | 21:37 |
| jog0 | salv-orlando: ohh good idea | 21:37 |
| *** otherwiseguy has joined #openstack-neutron | 21:39 | |
| *** otherwiseguy has quit IRC | 21:40 | |
| *** otherwiseguy has joined #openstack-neutron | 21:40 | |
| anteaya | I am heading out for a walk, back in a while | 21:42 |
| *** networkstatic has quit IRC | 22:02 | |
| *** carl_baldwin has quit IRC | 22:12 | |
| *** jlibosva has quit IRC | 22:12 | |
| *** pcm_ has quit IRC | 22:16 | |
| openstackgerrit | A change was merged to openstack/neutron: DHCP agent scheduler support for BigSwitch plugin https://review.openstack.org/54485 | 22:19 |
| *** yamahata has quit IRC | 22:37 | |
| *** yamahata has joined #openstack-neutron | 22:39 | |
| *** carl_baldwin has joined #openstack-neutron | 22:41 | |
| anteaya | back | 22:42 |
| *** otherwiseguy has quit IRC | 22:42 | |
| anteaya | so this failed to get large-ops passing: https://review.openstack.org/#/c/55917/ | 22:46 |
| anteaya | jog0: it failed on a timeout issue waiting for a uuid (image) to finish building and return an active status | 22:47 |
| anteaya | is that the consistent finding for the large-ops failures? | 22:47 |
| salv-orlando | that's always the pattern anteaya. There also no nova-side or devstack-side change related in any way. | 22:49 |
| salv-orlando | I am scavenging the logs - basically I am looking for a situation in which allocate_for_instance in the nova driver does not complete, possibly because of a deadlock in neutron | 22:50 |
| jog0 | anteaya: that is consistent | 22:50 |
| *** yamahata has quit IRC | 22:55 | |
| jog0 | while we are digging into neutron bugs: | 22:56 |
| *** mlavalle has quit IRC | 22:56 | |
| jog0 | https://bugs.launchpad.net/neutron/+bug/1235486 | 22:57 |
| jog0 | https://bugs.launchpad.net/neutron/+bug/1239637 | 22:57 |
| jog0 | are two of the most common gate failing jobs | 22:57 |
| anteaya | okay, thanks | 22:57 |
| anteaya | more splunking to do | 22:58 |
| anteaya | hey jog0 you may have missed this, but I am cobbling together a code sprint for Neutron Tempest in the 2nd week of January in Montreal | 22:58 |
| anteaya | would be awesome if you could attend | 22:59 |
| anteaya | 3 days sometime that week, I need to source venue and production help | 22:59 |
| anteaya | okay both bugs are assigned and report status in progress | 23:00 |
| anteaya | salv-orlando: are armando and robert kukura in channel do you know? | 23:01 |
| jog0 | I will be in Australia I think http://linux.conf.au/ | 23:02 |
| salv-orlando | armando is armax - but I think he's in a meeting now. Bob is rkukura, he's on EST so perhaps he's not in office now | 23:02 |
| anteaya | so will I, jog0, it takes place the week after Australia | 23:02 |
| salv-orlando | jog0: anteaya scheduled the thing keeping linuxcon in mind. This is not a good excuse as well :) | 23:02 |
| armax | I'm here | 23:02 |
| anteaya | okay I am EST time too, it is 6pm | 23:02 |
| anteaya | hello armax | 23:02 |
| anteaya | we were talking a look at https://bugs.launchpad.net/neutron/+bug/1239637 | 23:03 |
| anteaya | how is that going? | 23:03 |
| anteaya | *taking | 23:03 |
| jog0 | anteaya: both bugs say they are in progress but I don't see any outstanding patches to fix em | 23:03 |
| anteaya | salv-orlando: now you are singing my song, no excuses | 23:04 |
| jog0 | anteaya: ahh your one step ahead of me, count me in in that case | 23:04 |
| anteaya | grand thanks | 23:04 |
| armax | yeah | 23:04 |
| armax | launchpad bug reports are kinda messed up | 23:04 |
| armax | the issues are covered by these two patches: | 23:04 |
| armax | https://review.openstack.org/#/c/46563/ | 23:04 |
| armax | https://review.openstack.org/#/c/54850/ | 23:04 |
| armax | https://review.openstack.org/#/c/55000/ | 23:05 |
| anteaya | we were talking about 54850 earlier | 23:05 |
| armax | k | 23:05 |
| armax | wanna fill me in? | 23:05 |
| anteaya | salv-orlando said he would review and get markmclain to as well | 23:06 |
| anteaya | sure | 23:06 |
| anteaya | as you know, neutron tests are failing | 23:06 |
| anteaya | they are affecting other projects | 23:06 |
| anteaya | as of the summit, I am helping to address that situation | 23:06 |
| anteaya | I am on the infra team | 23:06 |
| anteaya | so I am currently doing some evaluation | 23:06 |
| anteaya | as well as meeting people | 23:07 |
| anteaya | my technical knowledge is not as deep as some peoples | 23:07 |
| anteaya | but I am a good conversation starter | 23:07 |
| armax | ok | 23:07 |
| armax | if you look at https://review.openstack.org/#/c/55000/ | 23:07 |
| anteaya | does that help to give a sense? | 23:07 |
| armax | this is stacked up on top of fix for 54850 | 23:07 |
| armax | the gate jobs have been pretty stable | 23:08 |
| armax | and the other failures that I have seen are related to bug #1232971 | 23:09 |
| anteaya | when you say the gate jobs have been pretty stable, what do you mean? | 23:09 |
| anteaya | and what are you basing that assessment on? | 23:09 |
| salv-orlando | armax: they're explicitly looking at largeops | 23:09 |
| salv-orlando | it has be voting soon | 23:09 |
| *** carl_baldwin has quit IRC | 23:09 | |
| armax | I haven't looked at the largeops at all | 23:09 |
| armax | at the time I looked at it it was non-voting | 23:09 |
| salv-orlando | yeah. So that's the deal. | 23:09 |
| armax | before adding fuel to the fire | 23:10 |
| armax | I'd rather see the check-tempest-devstack-vm-neutron, check-tempest-devstack-vm-neutron-pg, check-tempest-devstack-vm-neutron-isolated and check-tempest-devstack-vm-neutron-pg-isolated | 23:10 |
| armax | stable first | 23:10 |
| jog0 | armax: large ops passed two weeks ago | 23:10 |
| armax | ic | 23:11 |
| jog0 | and was stable | 23:11 |
| jog0 | it does 'nova boot --num-instances=150' | 23:11 |
| jog0 | rather the logical equivalent and thats it | 23:11 |
| anteaya | armax: I agree, they all need to be stable | 23:12 |
| armax | I'd need to look into the large-ops one to understand what's going on | 23:12 |
| anteaya | if you want to advocate for those tests being stable I will support you in that | 23:12 |
| anteaya | and find others to work on large-ops | 23:12 |
| armax | what I noticed when I worked on https://review.openstack.org/#/c/55000/ and https://review.openstack.org/#/c/54850/ | 23:12 |
| armax | is that large-ops failed consistently | 23:12 |
| anteaya | yes | 23:13 |
| armax | while the isolated ones and the pg ones were being successful | 23:13 |
| anteaya | has since October 31st | 23:13 |
| armax | so I figured there was some other failure mode going on | 23:13 |
| armax | I can have a look into that | 23:13 |
| armax | but not until later today | 23:13 |
| armax | I need to dash now | 23:13 |
| jog0 | armax: see above for what largeops does | 23:13 |
| anteaya | that would be great if you are willing | 23:13 |
| anteaya | and thanks | 23:13 |
| armax | that said | 23:13 |
| armax | getting fixes for 55000 and 54850 in | 23:14 |
| armax | would be great | 23:14 |
| armax | or at least get some feedback | 23:14 |
| armax | btw large-ops is still non-voting right? | 23:15 |
| anteaya | well salv-orlando just approved 55000 | 23:15 |
| armax | yeah | 23:15 |
| anteaya | so we should see something soon | 23:15 |
| jog0 | armax: we are trying to get it to be voting | 23:15 |
| armax | but that depends on 54850 | 23:15 |
| armax | gotcha | 23:15 |
| anteaya | and yes large-ops is currently non-voting | 23:15 |
| anteaya | ah okay so 54850 first, then reverify on 55000 | 23:15 |
| armax | yeah | 23:16 |
| anteaya | jog0: do you feel able to review https://review.openstack.org/#/c/54850/ | 23:16 |
| anteaya | can't hurt | 23:16 |
| armax | taking into account that there is still bug #1232971 lurking around | 23:16 |
| anteaya | I'm reading the review comments now | 23:17 |
| anteaya | armax: we have to do the best we can in this direction | 23:17 |
| armax | so even though two nasty ones will be addressed reviews won't be succeeding at 100% rate at least until that one gets addressed | 23:17 |
| anteaya | we will fall and then stand up again | 23:17 |
| anteaya | fair enough | 23:17 |
| anteaya | let's start by increasing the success percentage from what it is | 23:18 |
| armax | anyhow, I'll look into the large-ops asap | 23:18 |
| anteaya | thank you, | 23:18 |
| anteaya | I am grateful | 23:18 |
| armax | is there any bug report already being filed or more information I can rely on? | 23:18 |
| armax | sure thing | 23:18 |
| anteaya | also hope you can attend the sprint in Montreal in the 2nd week of January | 23:18 |
| armax | not likely ;) | 23:18 |
| armax | unless there a irc channel associated with it | 23:18 |
| anteaya | armax: you know what I know at present, but if I learn more I will update the appropriate bug report or file a new one | 23:18 |
| jog0 | anteaya: unfortunately I don't think I am. I am not very familer with the code base | 23:19 |
| anteaya | no, we have to do in person | 23:19 |
| anteaya | jog0: understood | 23:19 |
| anteaya | I am going to give it a whirl for syntax and grammar just to get in the groove | 23:19 |
| anteaya | not much of a contribution but better than standing here pointing fingers | 23:19 |
| anteaya | armax: where are you based? | 23:20 |
| anteaya | armax: here is the bug jog0 filed on large-ops: https://bugs.launchpad.net/neutron/+bug/1250168 | 23:21 |
| openstackgerrit | Ivar Lazzaro proposed a change to openstack/neutron: Embrane LBaaS Driver https://review.openstack.org/54236 | 23:28 |
| armax | I am based in Palo Alto, CA | 23:33 |
| armax | anteya: thanks for report 1250168 I'll look into it | 23:34 |
| anteaya | armax: thanks. The bug is assigned to me because I attempted to address it by reverting two different patches | 23:37 |
| anteaya | I don't think I have found the correct patches/approach though | 23:38 |
| anteaya | if you have a better fix, the bug is all yours | 23:38 |
| *** armax has left #openstack-neutron | 23:38 | |
| *** otherwiseguy has joined #openstack-neutron | 23:54 | |
| salv-orlando | armax: fit the large ops job uses the fake driver. I did not know that, but it makes sense as it launches 150 vms | 23:59 |
| salv-orlando | fit -> fyi | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!