| *** AlexeyAbashkin has joined #openstack-powervm | 00:24 | |
| *** AlexeyAbashkin has quit IRC | 00:28 | |
| *** svenkat has joined #openstack-powervm | 00:49 | |
| *** svenkat has quit IRC | 01:12 | |
| *** svenkat has joined #openstack-powervm | 01:27 | |
| *** esberglu has quit IRC | 02:04 | |
| *** esberglu has joined #openstack-powervm | 02:04 | |
| *** esberglu has quit IRC | 02:09 | |
| *** esberglu has joined #openstack-powervm | 02:31 | |
| *** esberglu has quit IRC | 02:39 | |
| *** esberglu has joined #openstack-powervm | 02:40 | |
| *** esberglu has quit IRC | 02:44 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 03:24 | |
| *** AlexeyAbashkin has quit IRC | 03:29 | |
| *** apearson has joined #openstack-powervm | 03:49 | |
| *** esberglu has joined #openstack-powervm | 03:51 | |
| *** esberglu has quit IRC | 03:56 | |
| *** svenkat has quit IRC | 04:12 | |
| *** esberglu has joined #openstack-powervm | 04:48 | |
| *** esberglu has quit IRC | 04:52 | |
| *** k0da has joined #openstack-powervm | 05:04 | |
| *** apearson has quit IRC | 05:35 | |
| *** mdrabe has quit IRC | 06:11 | |
| *** mdrabe has joined #openstack-powervm | 06:13 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 07:53 | |
| *** k0da has quit IRC | 09:11 | |
| *** k0da has joined #openstack-powervm | 09:24 | |
| *** k0da has quit IRC | 10:51 | |
| *** svenkat has joined #openstack-powervm | 13:25 | |
| *** apearson has joined #openstack-powervm | 13:34 | |
| *** esberglu has joined #openstack-powervm | 13:48 | |
| esberglu | #startmeeting powervm_driver_meeting | 14:01 |
|---|---|---|
| openstack | Meeting started Tue Feb 20 14:01:25 2018 UTC and is due to finish in 60 minutes. The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| *** openstack changes topic to " (Meeting topic: powervm_driver_meeting)" | 14:01 | |
| openstack | The meeting name has been set to 'powervm_driver_meeting' | 14:01 |
| esberglu | edmondsw: efried: You guys around? Sorry didn't realize the recurring meeting notices were done | 14:03 |
| efried | oh, yeah, I forgot because no calendar entry. | 14:03 |
| efried | But I'm here. | 14:03 |
| esberglu | #topic In-tree Driver | 14:04 |
| *** openstack changes topic to "In-tree Driver (Meeting topic: powervm_driver_meeting)" | 14:04 | |
| esberglu | The bp looked good to me, I think edmondsw is gonna add the interface hotplug stuff still? | 14:04 |
| esberglu | Only other open IT item for me is snapshot | 14:06 |
| esberglu | Hoping we can get vSCSI, snapshot and localdisk in pretty early in the cycle | 14:06 |
| esberglu | Because the migration stuff is gonna require a lot more work, both coding and testing | 14:07 |
| esberglu | And getting CI up to speed | 14:07 |
| efried | yuh | 14:09 |
| esberglu | #topic Out-of-tree Driver | 14:09 |
| *** openstack changes topic to "Out-of-tree Driver (Meeting topic: powervm_driver_meeting)" | 14:09 | |
| efried | edmondsw and I learned things about MagicMock and one of our test cases yesterday. | 14:09 |
| efried | about the former: it doesn't blow up when you try to iterate on it (Mock does). And about the latter: we weren't testing what we thought we were testing. Which happens a lot. | 14:10 |
| esberglu | efried: You mean we have a bunch of tests not doing what we thought? | 14:12 |
| efried | Probably. But at least this one for sure. | 14:12 |
| efried | Interested in the recap? | 14:13 |
| esberglu | efried: Yeah | 14:13 |
| efried | This was for driver capabilities. We've been setting has_imagecache dynamically, because whether the driver supports it depends on which disk driver we're using. | 14:14 |
| efried | Localdisk supports it, the others don't. | 14:14 |
| efried | So we had a test that said assertFalse(driver.capabilities['has_imagecache']) | 14:14 |
| efried | which seems reasonable enough | 14:15 |
| efried | The dynamic setting used to be code in the driver that did something like: | 14:15 |
| efried | self.capabilities.update(disk_drv.capabilities) | 14:15 |
| efried | i.e. "take the stuff in the disk_drv.capabilities dict and overwrite/add whatever's in self.capabilities" | 14:16 |
| efried | edmondsw wasn't happy with the "add" part, because it meant we were pulling capability keys into our compute driver that aren't part of the ComputeDriver interface. | 14:16 |
| esberglu | yep | 14:17 |
| efried | So to get rid of that, he changed it to be explicit about just pulling in has_imagecache: | 14:17 |
| efried | self.capabilities['has_imagecache'] = disk_drv.capabilities['has_imagecache'] | 14:17 |
| efried | Whereupon the above assertion started to fail. | 14:17 |
| efried | And when he changed it to assertTrue, it passed. | 14:17 |
| efried | Which should not happen. | 14:17 |
| efried | If we were testing this properly, both of those modes of assignment should have been equivalent in terms of the has_imagecache key. So it should have been the same before & after. Which in fact should have been a signal that the code change was valid. | 14:19 |
| esberglu | But the code change is valid, so we realized it was a problem with our test | 14:20 |
| efried | Well, in broad terms, yeah. But we couldn't suss what the problem could possibly be. | 14:20 |
| efried | The first thing I suspected was that one or both of those .capabilities was a Mock. But that didn't make sense, because you get an exception if you try to [key access] or .update a Mock. | 14:21 |
| efried | So either code path should have raised an exception. | 14:21 |
| efried | But it turns out that the disk_drv.capabilities dict was a *Magic*Mock | 14:22 |
| efried | And MagicMock, in contrast with Mock, has default impls for magic methods. Like __getitem__ (returns a mock) and __iter__ (yields as if an empty list). | 14:23 |
| efried | So when we were doing .update(), we were updating the driver's capabilities - where has_imagecache is defaulted to False - from an empty list of disk driver capabilities. Which left it False. | 14:24 |
| efried | But when we did the assignment, we were assigning a mock to the value. And a mock evaluates to True. | 14:24 |
| esberglu | Ah okay I see | 14:25 |
| esberglu | I'm not seeing where in the code the capabilities are a MagicMock vs Mock | 14:26 |
| efried | Yeah, that was tricky to track down. The driver is a real driver, which we were doing on purpose so we could get "real" values in the capabilities dict | 14:28 |
| efried | BUT | 14:28 |
| efried | the setUp for this test class is installing a fixture that actually mocks the disk driver constructor | 14:29 |
| efried | So we're getting a real compute driver, but when it goes to create the disk driver, that mock patch takes over and we get a MagicMock. | 14:29 |
| esberglu | Okay I follow | 14:30 |
| efried | What's next? | 14:32 |
| esberglu | efried: How do we fix it? | 14:32 |
| efried | Don't let the disk driver get mocked. | 14:32 |
| efried | I suggested moving the test case up to the first class in the suite, which doesn't do any of the fixture stuff. | 14:33 |
| efried | And also adding test cases where we use different disk drivers to make sure it's True only for LocalStorage. | 14:33 |
| efried | And also adding assertions to ensure that the extraneous keys aren't being created. | 14:34 |
| esberglu | Sounds good | 14:34 |
| esberglu | #topic Device Passthrough | 14:35 |
| *** openstack changes topic to "Device Passthrough (Meeting topic: powervm_driver_meeting)" | 14:35 | |
| efried | IOW, what I'm sure edmondsw thought was going to be an ace across the net is rapidly metastasizing | 14:35 |
| esberglu | Yep sounds that way | 14:35 |
| efried | No news from me on dev passthrough. We're frantically writing & reviewing specs ahead of the PTG. | 14:35 |
| efried | I abandoned that resource class affinity spec. | 14:36 |
| efried | And put up this new one (unrelated) last night: https://review.openstack.org/#/c/546009/ | 14:36 |
| esberglu | #topic PowerVM CI | 14:38 |
| *** openstack changes topic to "PowerVM CI (Meeting topic: powervm_driver_meeting)" | 14:38 | |
| efried | I believe I'm still waiting on prompts from edmondsw et al for next steps on dev passthrough. | 14:38 |
| efried | sorry, I'm a step behind you all day. | 14:38 |
| esberglu | np | 14:38 |
| esberglu | So the big thing for CI for queens is getting multinode jobs so we can do migration testing | 14:39 |
| esberglu | I haven't looked into that at all really in the past, so I need to do some research there | 14:39 |
| efried | Yeah. Big fun times. | 14:39 |
| efried | I can tell you this: I will be of absolutely no help. | 14:39 |
| efried | I have managed to go my entire career without ever migrating a partition | 14:40 |
| esberglu | Huh, would have bet against that :) | 14:40 |
| esberglu | The other thing for CI I wanted to bring up is that stale vopt issue we were seeing | 14:41 |
| esberglu | Since fixing up the post stack cleaner, that has gone WAY down. However it did hit a handful of times in the last week | 14:41 |
| esberglu | So we either need to re-explore that vopt naming change or try to figure out where the vopts are coming from now | 14:42 |
| efried | interesting. | 14:42 |
| efried | Did you manually clean up (or check) any stale vopts that were lying around from before we fixed the cleaner? | 14:42 |
| esberglu | efried: Yes | 14:43 |
| efried | So we're somehow still ending up with stale ones. | 14:43 |
| *** k0da has joined #openstack-powervm | 14:44 | |
| esberglu | Yep, looks that way | 14:44 |
| efried | I wonder if it could be timing issues with multiple tests (possibly even from multiple patches) running at once. | 14:44 |
| efried | Like, the offending vopt isn't actually stale; it's just in the process of being created/mapped or unmapped/deleted when the other test hits this code path. | 14:45 |
| efried | ...and it's still the naming issue. | 14:45 |
| esberglu | efried: If that were the case I don't think the post stack cleaner update would have cut down the occurrences so much | 14:45 |
| efried | No, I'm saying both issues that we talked about were problems. The post-stack cleaner took care of most of them, but not all. | 14:46 |
| efried | Unless... is it possible the cleaner fix didn't get deployed to whatever server hit the problem? | 14:46 |
| esberglu | efried: Nope, the latest of that gets pulled every run | 14:46 |
| efried | How often is this happening? Did we just see this once in a little blip and never again? | 14:47 |
| esberglu | 7 times since I left on wednesday | 14:49 |
| esberglu | Where it was a handful a day before the cleaner fix | 14:49 |
| esberglu | Either way, needs further debugging. I will look into it and let you know if I get stuck | 14:51 |
| efried | Spread out across multiple nodes? | 14:51 |
| esberglu | efried: Yep | 14:51 |
| efried | I was thinking we could reinstate the naming fix and see if it clears up entirely. | 14:51 |
| esberglu | efried: And if it doesn't, that points to a possible timing issue | 14:52 |
| esberglu | I'm on board with that | 14:52 |
| efried | ight. will look for that review. I need to un-abandon the renaming patch, eh? | 14:53 |
| * efried looks... | 14:53 | |
| efried | Done: https://review.openstack.org/#/c/540453/ | 14:53 |
| esberglu | efried: Cool I'll put up the change to patch it in right after this | 14:54 |
| esberglu | efried: That's all for me today, anything else? | 14:55 |
| efried | don't think so. | 14:55 |
| esberglu | #endmeeting | 14:55 |
| *** openstack changes topic to "This channel is for PowerVM-related development and discussion. For general OpenStack support, please use #openstack." | 14:55 | |
| openstack | Meeting ended Tue Feb 20 14:55:49 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:55 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-02-20-14.01.html | 14:55 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-02-20-14.01.txt | 14:55 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-02-20-14.01.log.html | 14:55 |
| efried | Thanks esberglu | 14:56 |
| esberglu | I'll add the meeting invites again. It would be nice if you got notified when your recurring meetings end | 14:56 |
| efried | Mm. Patent. | 14:57 |
| esberglu | efried: 6349 | 15:18 |
| efried | esberglu: +2 | 15:19 |
| *** tjakobs has joined #openstack-powervm | 15:23 | |
| *** apearson has quit IRC | 16:01 | |
| *** apearson has joined #openstack-powervm | 16:06 | |
| *** AlexeyAbashkin has quit IRC | 16:26 | |
| *** k0da has quit IRC | 17:32 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 17:57 | |
| *** AlexeyAbashkin has quit IRC | 18:02 | |
| *** openstackgerrit has quit IRC | 18:03 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 18:13 | |
| *** efried has quit IRC | 18:15 | |
| *** efried has joined #openstack-powervm | 18:15 | |
| *** AlexeyAbashkin has quit IRC | 18:17 | |
| *** k0da has joined #openstack-powervm | 19:10 | |
| *** k0da has quit IRC | 19:40 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 20:23 | |
| *** AlexeyAbashkin has quit IRC | 20:27 | |
| *** k0da has joined #openstack-powervm | 20:59 | |
| *** k0da has quit IRC | 21:04 | |
| *** k0da has joined #openstack-powervm | 21:16 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 21:23 | |
| *** AlexeyAbashkin has quit IRC | 21:27 | |
| *** svenkat has quit IRC | 21:28 | |
| *** apearson has quit IRC | 22:06 | |
| *** apearson has joined #openstack-powervm | 22:06 | |
| *** apearson has quit IRC | 22:06 | |
| *** tjakobs has quit IRC | 22:38 | |
| *** tjakobs has joined #openstack-powervm | 22:41 | |
| edmondsw | efried we definitely found a rabbit hole with this test issue. | 22:44 |
| edmondsw | I spent some more time on it last night and early this morning, and the driver fixture is causing issues even in that Init class that doesn't have any fixtures. | 22:44 |
| edmondsw | I wrote new tests that pass when run individually, but fail (because of the fixture) if I run the whole test suite | 22:45 |
| edmondsw | so something isn't cleaning up properly? I don't know much about how fixtures work, so I'll have to dig into that | 22:45 |
| edmondsw | after I get back on Thursday | 22:45 |
| edmondsw | I'll go ahead and throw up what I've done in case you want to see it, but I know it's not working | 22:46 |
| *** openstackgerrit has joined #openstack-powervm | 22:49 | |
| openstackgerrit | Matthew Edmonds proposed openstack/nova-powervm master: stop setting extra capabilities https://review.openstack.org/545984 | 22:49 |
| efried | super weird. That is definitely not supposed to happen. | 23:02 |
| *** openstackgerrit has quit IRC | 23:04 | |
| *** tjakobs has quit IRC | 23:07 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 23:23 | |
| *** svenkat has joined #openstack-powervm | 23:25 | |
| *** AlexeyAbashkin has quit IRC | 23:27 | |
| *** svenkat has quit IRC | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!