Wednesday, 2016-06-29

« Tuesday, 2016-06-28 Index Thursday, 2016-06-30 »
*** tlian has quit IRC00:03
*** tlian has joined #openstack-powervm00:11
*** thorst has joined #openstack-powervm00:13
*** jwcroppe has quit IRC00:14
*** jwcroppe has joined #openstack-powervm00:14
*** tlian2 has joined #openstack-powervm00:15
*** tlian has quit IRC00:15
*** jwcroppe has quit IRC00:18
*** edmondsw has quit IRC00:19
*** thorst has quit IRC00:21
*** thorst has joined #openstack-powervm00:59
*** openstack has joined #openstack-powervm01:24
*** ManojK has quit IRC01:43
*** thorst has quit IRC01:43
*** thorst has joined #openstack-powervm01:44
*** tlian has joined #openstack-powervm01:45
*** tlian2 has quit IRC01:48
*** thorst has quit IRC01:52
*** arnoldje has joined #openstack-powervm02:43
*** thorst has joined #openstack-powervm02:50
*** svenkat has joined #openstack-powervm02:51
*** svenkat has quit IRC02:57
*** thorst has quit IRC02:57
*** jwcroppe has joined #openstack-powervm03:12
*** Ashana has joined #openstack-powervm03:21
*** Ashana has quit IRC03:26
*** Ashana has joined #openstack-powervm03:27
*** kotra03 has quit IRC03:31
*** Ashana has quit IRC03:32
*** Ashana has joined #openstack-powervm03:33
*** Ashana has quit IRC03:38
*** Ashana has joined #openstack-powervm03:39
*** jwcroppe has quit IRC03:41
*** Ashana has quit IRC03:43
*** Ashana has joined #openstack-powervm03:45
*** tjakobs has joined #openstack-powervm03:47
*** Ashana has quit IRC03:49
*** Ashana has joined #openstack-powervm03:51
*** Ashana has quit IRC03:55
*** thorst has joined #openstack-powervm03:55
*** tjakobs has quit IRC03:56
*** Ashana has joined #openstack-powervm04:01
*** thorst has quit IRC04:03
*** Ashana has quit IRC04:05
*** Ashana has joined #openstack-powervm04:07
*** Ashana has quit IRC04:11
*** tlian has quit IRC04:12
*** Ashana has joined #openstack-powervm04:13
*** Ashana has quit IRC04:17
*** Ashana has joined #openstack-powervm04:18
*** Ashana has quit IRC04:23
*** Ashana has joined #openstack-powervm04:24
*** Ashana has quit IRC04:29
*** Ashana has joined #openstack-powervm04:30
*** jwcroppe has joined #openstack-powervm04:30
*** Ashana has quit IRC04:35
*** Ashana has joined #openstack-powervm04:36
*** Ashana has quit IRC04:41
*** Ashana has joined #openstack-powervm04:42
*** Ashana has quit IRC04:46
*** Ashana has joined #openstack-powervm04:48
*** Ashana has quit IRC04:52
*** Ashana has joined #openstack-powervm04:54
*** kotra03 has joined #openstack-powervm04:55
*** Ashana has quit IRC04:58
*** thorst has joined #openstack-powervm05:01
*** Ashana has joined #openstack-powervm05:07
*** jwcroppe has quit IRC05:07
*** thorst has quit IRC05:08
*** Ashana has quit IRC05:11
*** Ashana has joined #openstack-powervm05:19
*** arnoldje has quit IRC05:19
*** Ashana has quit IRC05:23
*** Ashana has joined #openstack-powervm05:24
*** jwcroppe has joined #openstack-powervm05:27
*** Ashana has quit IRC05:29
*** Ashana has joined #openstack-powervm05:36
*** Ashana has quit IRC05:41
*** Ashana has joined #openstack-powervm05:48
*** Ashana has quit IRC05:53
*** Ashana has joined #openstack-powervm05:54
*** Ashana has quit IRC05:58
*** Ashana has joined #openstack-powervm06:02
*** thorst has joined #openstack-powervm06:05
*** Ashana has quit IRC06:06
*** thorst has quit IRC06:12
*** Ashana has joined #openstack-powervm06:13
*** Ashana has quit IRC06:18
*** Ashana has joined #openstack-powervm06:19
*** Ashana has quit IRC06:24
*** Ashana has joined #openstack-powervm06:25
*** Ashana has quit IRC06:29
*** Ashana has joined #openstack-powervm06:31
*** Ashana has quit IRC06:36
*** Ashana has joined #openstack-powervm06:43
*** Ashana has quit IRC06:47
*** Ashana has joined #openstack-powervm06:54
*** Ashana has quit IRC06:59
*** Ashana has joined #openstack-powervm07:08
*** thorst has joined #openstack-powervm07:10
*** Ashana has quit IRC07:12
*** Ashana has joined #openstack-powervm07:14
*** thorst has quit IRC07:17
*** Ashana has quit IRC07:18
*** Ashana has joined #openstack-powervm07:20
*** jwcroppe has quit IRC07:23
*** Ashana has quit IRC07:24
*** k0da has joined #openstack-powervm07:25
*** Ashana has joined #openstack-powervm07:32
*** Ashana has quit IRC07:36
*** Ashana has joined #openstack-powervm07:43
*** Ashana has quit IRC07:47
*** Ashana has joined #openstack-powervm07:49
*** Ashana has quit IRC07:53
*** Ashana has joined #openstack-powervm08:03
*** Ashana has quit IRC08:08
*** Ashana has joined #openstack-powervm08:08
*** jwcroppe has joined #openstack-powervm08:12
*** Ashana has quit IRC08:13
*** thorst has joined #openstack-powervm08:15
*** erlarese_ has joined #openstack-powervm08:20
*** Ashana has joined #openstack-powervm08:20
*** erlarese has quit IRC08:23
*** erlarese_ is now known as erlarese08:23
*** thorst has quit IRC08:23
*** Ashana has quit IRC08:25
*** Ashana has joined #openstack-powervm08:26
*** jwcroppe has quit IRC08:28
*** Ashana has quit IRC08:30
*** Ashana has joined #openstack-powervm08:32
*** Ashana has quit IRC08:36
*** Ashana has joined #openstack-powervm08:38
*** Ashana has quit IRC08:43
*** Ashana has joined #openstack-powervm08:44
*** Ashana has quit IRC08:48
*** Ashana has joined #openstack-powervm08:55
*** Ashana has quit IRC09:00
*** Ashana has joined #openstack-powervm09:09
*** Ashana has quit IRC09:14
*** Ashana has joined #openstack-powervm09:15
*** Ashana has quit IRC09:19
*** thorst has joined #openstack-powervm09:20
*** Ashana has joined #openstack-powervm09:21
*** Ashana has quit IRC09:25
*** Ashana has joined #openstack-powervm09:26
*** thorst has quit IRC09:28
*** Ashana has quit IRC09:31
*** Ashana has joined #openstack-powervm09:32
*** Ashana has quit IRC09:36
*** Ashana has joined #openstack-powervm09:38
*** Ashana has quit IRC09:43
*** Ashana has joined #openstack-powervm09:50
*** Ashana has quit IRC09:54
*** Ashana has joined #openstack-powervm09:56
*** Ashana has quit IRC10:00
*** Ashana has joined #openstack-powervm10:09
*** Ashana has quit IRC10:14
*** Ashana has joined #openstack-powervm10:15
*** Ashana has quit IRC10:19
*** thorst has joined #openstack-powervm10:25
*** thorst has quit IRC10:33
*** smatzek has joined #openstack-powervm11:05
*** svenkat has joined #openstack-powervm11:13
*** thorst has joined #openstack-powervm11:42
*** k0da has quit IRC11:49
*** k0da has joined #openstack-powervm11:54
*** jwcroppe has joined #openstack-powervm11:58
*** seroyer has joined #openstack-powervm11:58
*** jwcroppe has quit IRC12:05
*** k0da has quit IRC12:33
*** k0da has joined #openstack-powervm12:38
*** Ashana has joined #openstack-powervm12:39
*** ManojK has joined #openstack-powervm12:45
*** edmondsw has joined #openstack-powervm12:48
*** ManojK has quit IRC12:55
*** esberglu has joined #openstack-powervm13:00
*** tlian has joined #openstack-powervm13:01
*** seroyer has quit IRC13:03
*** mdrabe has joined #openstack-powervm13:08
*** tblakeslee has joined #openstack-powervm13:08
*** ManojK has joined #openstack-powervm13:10
*** k0da has quit IRC13:11
*** k0da has joined #openstack-powervm13:13
*** svenkat_ has joined #openstack-powervm13:22
*** svenkat has quit IRC13:23
*** svenkat_ is now known as svenkat13:23
*** lmtaylor1 has joined #openstack-powervm13:30
*** mdrabe has quit IRC13:34
*** mdrabe has joined #openstack-powervm13:35
thorstadreznec: The drop in the DVR communication seems to be between the qr and the tap device.13:35
*** ManojK has quit IRC13:38
*** ManojK has joined #openstack-powervm13:40
*** efried has quit IRC13:43
*** jwcroppe has joined #openstack-powervm14:04
*** apearson has joined #openstack-powervm14:06
*** jwcroppe has quit IRC14:09
*** mdrabe has quit IRC14:12
*** tblakeslee has quit IRC14:13
*** mdrabe has joined #openstack-powervm14:13
*** tblakeslee has joined #openstack-powervm14:15
*** Ashana has quit IRC14:20
*** Ashana has joined #openstack-powervm14:27
*** Ashana has quit IRC14:31
*** mdrabe has quit IRC14:32
*** mdrabe has joined #openstack-powervm14:32
*** Ashana has joined #openstack-powervm14:32
*** mdrabe has quit IRC14:36
*** mdrabe has joined #openstack-powervm14:36
*** Ashana has quit IRC14:37
*** mdrabe_ has joined #openstack-powervm14:39
*** arnoldje has joined #openstack-powervm14:40
*** mdrabe has quit IRC14:42
*** Ashana has joined #openstack-powervm14:44
*** kotra03 has quit IRC14:44
*** Ashana has quit IRC14:49
*** Ashana has joined #openstack-powervm14:50
*** Ashana has quit IRC14:54
*** Ashana has joined #openstack-powervm15:11
*** catintheroof has joined #openstack-powervm15:16
*** efried has joined #openstack-powervm15:17
*** miltonm has joined #openstack-powervm15:29
*** ManojK has quit IRC15:38
*** ManojK has joined #openstack-powervm15:40
*** esberglu has quit IRC15:42
*** k0da has quit IRC15:43
*** esberglu has joined #openstack-powervm15:44
*** esberglu has quit IRC15:51
*** esberglu_ has joined #openstack-powervm15:51
openstackgerritMerged openstack/nova-powervm: Remove vios module - use pypowervm.tasks.partition  https://review.openstack.org/33347415:56
*** esberglu_ has quit IRC15:58
*** esberglu has joined #openstack-powervm16:05
*** svenkat has quit IRC16:15
thorstesberglu: I think that since we're waiting for clbush to get us the SSP enhancements...we should do the memory upgrades over this next week16:16
thorstjust knock it out.  I can help you identify the physical locations of the boxes, but we would need to know how much memory is in which box so far16:16
thorstso that we can bump them all to 25616:16
*** kotra03 has joined #openstack-powervm16:31
*** mdrabe_ has quit IRC16:31
*** mdrabe has joined #openstack-powervm16:33
thorstefried: any changes you want me to review?  I'm frustrated with DVR and need to take a break...17:11
efriedthorst, yes, 3460 please17:11
thorston it17:11
efriedthorst, I feel really good about the 7..10 changes.  I think it greatly improves consumability/readability.17:12
thorstthis change set as a whole is....a big one.17:12
thorstdefinitely the introduction of new 'magic' logic17:13
thorstdo we really need a min/max redundancy level?17:13
thorstshould that just be 'these are how many I need'?17:13
efriedthorst, I felt it was pretty valid to support both.  It's a really easy way to allow the consumer to say things like, "I don't *neeeeed* redundancy, but I'll take it if it's possible" (the default); or "I must have at least dual redundancy, but four-way is better" (min=2, max=4).17:17
efriedImportant point being: you can set it up so we don't *have* to fail if we can't get to the exact number you specify.17:17
thorstmeh...I feel like that's led to some weirdness come full lifecycle17:17
thorstI've seen a similar concept used elsewhere17:17
thorstand initial deploy will start with min redundancy17:17
thorstLive Migrate to a host that could support more redundancy17:18
thorstbut of course, we're still at min redundancy17:18
thorstunless we're going to change the backing NICs for addl redundancy as part of live migrate?17:18
efriedYou'll have a similar ilk of problem if you just use however many ports are configured.17:18
efriedSource system has two, destination has three (or one!)17:19
thorstyeah...that's why I generally just kept it simple and said "I need two"17:19
thorstso you have to have that...17:19
efriedI don't really have a problem with the idea of rerunning this algorithm on the destination of a migrate.17:19
efriedAnd you end up with whatever you end up with.17:19
efriedI wouldn't think we want to fail a migration if the port configs are asymmetrical.17:19
efriedAs long as they're telling us the same physnet is available.17:19
thorstI mean...we need to fail if there are not enough ports17:20
efriedEnough ports for what?17:20
thorstthe question is, does a live migrate bump us up to more ports17:20
thorstenough for min_redundancy17:20
efriedWhich is defined by what?17:20
efriedI wasn't actually planning to provide a config option to allow them to specify redundancy levels.17:20
efriedWere you?17:20
thorstefried: I assumed it was a nova.conf option for the PowerVM driver.17:20
efriedI'm saying we could do without it.17:21
thorstwell, or we could go back to that idea where redundancy == number of VIOSes you have17:21
thorst1 or 2....17:21
thorstbut meh17:21
thorstwe could do without17:21
efriedE.g. default policy to: min_redundancy=2, max_redundancy=<number of ports in the whitelist for the physnet>17:21
efried(min_redundancy capped at 1 if only 1 port)17:22
thorstI assume we always strive for max?17:22
efriedThe algorithm always strives for max_redundancy, yes.17:22
efriedRight now it's not set up to default max_redundancy to len(pports).17:23
thorstso then you definitely don't want it to be all the ports17:23
thorstI'd honestly say a max should generally be 2 but some operators may want 317:23
thorst417:23
thorstbeyond that, we're probably over-engineering17:23
efriedSo, assuming we leave this method with its current spec, we have the nova-powervm driver override as:17:24
efriedmin_redundancy=min(2, len(pports))17:24
efriedmax_redundancy=min(4, len(pports))17:24
efriedAnd provide no config option17:24
efriedWe can add configurability later if deemed necessary.  That would be pretty simple to do.17:25
efriedBut I feel like all of that conversation has to do with the community driver, not the pypowervm method, which I feel should be flexible enough to facilitate such future direction.17:26
thorstmin_redundancy=min(1, min(2, len(pports)))17:26
thorstI'd say min redundancy should be 1, MAYBE 2.17:26
*** svenkat has joined #openstack-powervm17:26
thorstefried: fair enough17:27
efriedI definitely *don't* feel like we can preemptively dictate a static/hardcoded min or max at the pypowervm level.17:27
efriedor even an algorithmic one.17:27
thorstwe could17:27
thorstwe shouldn't17:27
thorst(mr. precision)17:27
thorstsorry sorry17:28
mdrabethorst, efried can I change the subject? It's about https://bugs.launchpad.net/nova-powervm/+bug/159679717:29
openstackLaunchpad bug 1596797 in nova-powervm "After failure of evacuate operation the VM on the target host is not cleaned up" [Undecided,New] - Assigned to Tyler Blakeslee (tblakes)17:29
thorstmdrabe: rip it17:29
mdrabeSo should the Create task for the LPAR have revert implemented?17:29
thorstmdrabe: def17:29
mdrabeI believe the original concern was that the manager failure path actually issues a _shutdown_instance17:30
thorstand it'd be shutting down something that doesn't exist?17:30
mdrabeso if a normal spawn fails, then the Create's revert would delete, then the manager's fail path would delete, and probs hit a 40417:30
mdrabeye17:30
thorstso the real question is...should shutdown instance fail if the instance is not there?17:30
thorstesberglu is actually adding something similar to the dlt_lpar code...from something we saw in our CI (two delete requests to same lpar at once)17:31
mdrabeWell, I think the Manas was on to something in that rebuild_instance in the manager actually needs to do the instance clean up17:31
mdrabethat Manas*17:31
efriedReverts (all of them) should never cause flow failure.  In general, they ought to catch and ignore any exceptions.17:31
mdrabeThat's another topic actually, some reverts don't do that17:32
mdrabeBut in this case the revert isn't implemented, if it was it'd probably cause a problem17:32
efriedThough I'm on board with the idea of (logging a warning and) ignoring shutdown failure on a nonexistent LPAR.17:32
thorstefried: the issue is that at our level, we really don't know that its a revert.  Unless we're talking 'taskflow revert' versus overall 'nova rebuild revert'17:32
efriedAren't we?17:32
mdrabethorst yes that's what we're talking17:33
*** svenkat has quit IRC17:33
mdrabebut it's about figuring which should be responsible for reverting17:33
thorstmdrabe: I guess I'm in support of a driver change to get the create/shutdown to work properly.17:33
thorstand maybe a nova change where we handle it higher up17:33
mdrabeI think the nova-powervm bits are fine actually17:34
thorstwell, I don't.  General spawn would leave LPARs around17:34
mdrabeAnd the bug is actually novas17:34
mdrabeBut it wouldn't because the manager'll clean em up17:34
thorstooo17:34
thorsthmmm17:34
thorstwell, so question number 1 is - can this be replicated in libvirt?17:35
mdrabebut rebuild_instance in the manager, used for evacuation, doesn't do instance cleanup as part of the fail path17:35
*** svenkat has joined #openstack-powervm17:36
thorstyeah.  I agree we can fix there (and probably should) - as long as we could demonstrate this scenario w/ libvirt17:36
mdrabeI betting on it cuz just looking at the code it's clear that no driver.destroy is invoked17:36
thorstthat's the number 1 question when any powervm driver bug comes up17:36
thorstthat needs core nova changes17:36
thorstyeah, I agree - should be easy to show17:36
mdrabeOn the libvirt side I'm guessing it'd correspond to the domain sticking around17:36
thorstmdrabe: if we can show that, I think we're good17:37
mdrabebut yea okay, check out libvirt and engage with nova peeps if reproducible there17:37
*** ManojK has quit IRC17:40
*** ManojK has joined #openstack-powervm17:40
*** tblakeslee has quit IRC17:42
*** tblakeslee has joined #openstack-powervm18:36
*** seroyer has joined #openstack-powervm18:40
openstackgerritLauren Taylor proposed openstack/nova-powervm: Some VM deploys fail from expired keystone token  https://review.openstack.org/33248518:43
*** ManojK has quit IRC18:54
*** esberglu has quit IRC18:59
openstackgerritLauren Taylor proposed openstack/nova-powervm: Some VM deploys fail from expired keystone token  https://review.openstack.org/33248519:00
openstackgerritLauren Taylor proposed openstack/nova-powervm: Some VM deploys fail from expired keystone token  https://review.openstack.org/33248519:14
*** tlian has quit IRC19:14
*** ManojK has joined #openstack-powervm19:15
*** tlian has joined #openstack-powervm19:21
*** kriskend has joined #openstack-powervm19:42
*** catintheroof has quit IRC19:43
*** jwcroppe has joined #openstack-powervm20:11
*** Ashana has quit IRC20:24
*** esberglu has joined #openstack-powervm20:29
*** Ashana has joined #openstack-powervm20:31
*** Ashana has quit IRC20:35
*** Ashana has joined #openstack-powervm20:37
*** Ashana has quit IRC20:41
*** Ashana has joined #openstack-powervm20:42
*** Ashana has quit IRC20:47
*** Ashana has joined #openstack-powervm20:48
*** Ashana has quit IRC20:52
*** Ashana has joined #openstack-powervm20:54
*** Ashana has quit IRC20:59
*** svenkat has quit IRC21:00
*** Ashana has joined #openstack-powervm21:00
*** Ashana has quit IRC21:04
*** Ashana has joined #openstack-powervm21:08
*** Ashana has quit IRC21:12
*** ManojK has quit IRC21:13
*** Ashana has joined #openstack-powervm21:14
*** ManojK has joined #openstack-powervm21:15
*** esberglu has quit IRC21:16
*** Ashana has quit IRC21:18
*** esberglu has joined #openstack-powervm21:18
*** Ashana has joined #openstack-powervm21:20
*** thorst has quit IRC21:21
*** k0da has joined #openstack-powervm21:22
*** efried1 has joined #openstack-powervm21:23
*** Ashana has quit IRC21:24
*** efried has quit IRC21:25
*** efried has joined #openstack-powervm21:25
*** Ashana has joined #openstack-powervm21:26
*** smatzek has quit IRC21:27
*** efried1 has quit IRC21:27
*** esberglu has quit IRC21:28
*** Ashana has quit IRC21:30
*** kriskend has quit IRC21:32
*** Ashana has joined #openstack-powervm21:37
*** Ashana has quit IRC21:42
*** Ashana has joined #openstack-powervm21:43
*** lmtaylor1 has left #openstack-powervm21:44
*** thorst has joined #openstack-powervm21:46
*** Ashana has quit IRC21:47
*** ManojK has quit IRC21:48
*** Ashana has joined #openstack-powervm21:49
*** seroyer has quit IRC21:49
*** thorst has quit IRC21:50
*** Ashana has quit IRC21:53
*** Ashana has joined #openstack-powervm21:55
*** Ashana has quit IRC21:59
*** Ashana has joined #openstack-powervm22:01
*** Ashana has quit IRC22:05
*** thorst has joined #openstack-powervm22:07
*** Ashana has joined #openstack-powervm22:09
*** arnoldje has quit IRC22:09
*** mdrabe has quit IRC22:11
*** Ashana has quit IRC22:13
*** Ashana has joined #openstack-powervm22:15
*** Ashana has quit IRC22:19
*** Ashana has joined #openstack-powervm22:20
*** Ashana has quit IRC22:24
*** Ashana has joined #openstack-powervm22:26
*** Ashana has quit IRC22:31
*** Ashana has joined #openstack-powervm22:32
*** seroyer has joined #openstack-powervm22:34
*** Ashana has quit IRC22:36
*** Ashana has joined #openstack-powervm22:38
*** tblakeslee has quit IRC22:38
*** efried has quit IRC22:38
*** Ashana has quit IRC22:42
*** seroyer has quit IRC22:43
*** Ashana has joined #openstack-powervm22:44
*** k0da has quit IRC22:45
*** edmondsw has quit IRC22:46
*** Ashana has quit IRC22:48
*** Ashana has joined #openstack-powervm22:50
*** Ashana has quit IRC22:54
*** Ashana has joined #openstack-powervm22:56
*** thorst has quit IRC22:58
*** thorst has joined #openstack-powervm22:59
*** Ashana has quit IRC23:00
*** Ashana has joined #openstack-powervm23:01
*** Ashana has quit IRC23:06
*** thorst has quit IRC23:07
*** Ashana has joined #openstack-powervm23:09
*** Ashana has quit IRC23:13
*** Ashana has joined #openstack-powervm23:15
*** Ashana has quit IRC23:19
*** Ashana has joined #openstack-powervm23:21
*** Ashana has quit IRC23:25
*** apearson has quit IRC23:45
« Tuesday, 2016-06-28 Index Thursday, 2016-06-30 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!