| *** alex_xu has quit IRC | 00:10 | |
| *** slaweq has joined #openstack-nova | 00:16 | |
| *** frank_wang has quit IRC | 00:19 | |
| *** takashin has joined #openstack-nova | 00:23 | |
| *** slaweq has quit IRC | 00:24 | |
| *** ricolin has joined #openstack-nova | 00:56 | |
| openstackgerrit | Merged openstack/nova stable/stein: Ignore hw_vif_type for direct, direct-physical vNIC types https://review.opendev.org/667354 | 01:01 |
|---|---|---|
| *** tetsuro has joined #openstack-nova | 01:02 | |
| *** imacdonn has quit IRC | 01:11 | |
| *** imacdonn has joined #openstack-nova | 01:12 | |
| *** slaweq has joined #openstack-nova | 01:14 | |
| *** spatel has joined #openstack-nova | 01:16 | |
| *** _alastor_ has joined #openstack-nova | 01:16 | |
| *** spatel has quit IRC | 01:20 | |
| *** slaweq has quit IRC | 01:25 | |
| *** adriant has joined #openstack-nova | 01:49 | |
| *** austinsun has joined #openstack-nova | 01:57 | |
| *** ricolin_ has joined #openstack-nova | 02:08 | |
| *** ricolin has quit IRC | 02:11 | |
| *** slaweq has joined #openstack-nova | 02:15 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Make evacuation respects anti-affinity rule https://review.opendev.org/649963 | 02:18 |
| *** ricolin__ has joined #openstack-nova | 02:18 | |
| *** ricolin_ has quit IRC | 02:21 | |
| *** spatel has joined #openstack-nova | 02:22 | |
| *** slaweq has quit IRC | 02:24 | |
| *** alex_xu has joined #openstack-nova | 02:26 | |
| *** whoami-rajat has joined #openstack-nova | 02:26 | |
| *** spatel has quit IRC | 02:27 | |
| *** tetsuro has quit IRC | 03:03 | |
| *** tetsuro has joined #openstack-nova | 03:04 | |
| *** lewe has joined #openstack-nova | 03:12 | |
| lewe | Hi, Maybe someone can help me, I am trying to cold migrate an instance but it does not seems to work right. The instance was newly created on the destination hosts rather than copying the original disk from the instance original host. Thank you. | 03:13 |
| *** slaweq has joined #openstack-nova | 03:14 | |
| *** tetsuro has quit IRC | 03:20 | |
| *** slaweq has quit IRC | 03:25 | |
| openstackgerrit | wangwei1 proposed openstack/nova master: fix spelling error in nova/api/validation/__init__.py https://review.opendev.org/669244 | 03:36 |
| *** BjoernT has joined #openstack-nova | 03:37 | |
| *** BjoernT_ has joined #openstack-nova | 03:40 | |
| *** BjoernT has quit IRC | 03:42 | |
| *** tetsuro has joined #openstack-nova | 03:55 | |
| *** slaweq has joined #openstack-nova | 04:16 | |
| *** rcernin has quit IRC | 04:17 | |
| *** lewe has quit IRC | 04:20 | |
| *** _alastor_ has quit IRC | 04:24 | |
| *** slaweq has quit IRC | 04:24 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Fix a broken link https://review.opendev.org/669248 | 04:26 |
| *** abhishekk has joined #openstack-nova | 04:36 | |
| openstackgerrit | Seyeong Kim proposed openstack/nova stable/queens: Share snapshot image membership with instance owner https://review.opendev.org/661667 | 04:45 |
| *** udesale has joined #openstack-nova | 04:49 | |
| *** BjoernT_ has quit IRC | 04:53 | |
| *** tkajinam has quit IRC | 04:54 | |
| *** tkajinam has joined #openstack-nova | 04:55 | |
| *** tetsuro has quit IRC | 04:57 | |
| *** tetsuro has joined #openstack-nova | 05:04 | |
| *** rcernin has joined #openstack-nova | 05:10 | |
| *** slaweq has joined #openstack-nova | 05:16 | |
| *** slaweq has quit IRC | 05:24 | |
| *** tkajinam_ has joined #openstack-nova | 05:25 | |
| *** tkajinam has quit IRC | 05:27 | |
| *** Luzi has joined #openstack-nova | 05:27 | |
| *** ricolin_ has joined #openstack-nova | 05:32 | |
| *** ricolin__ has quit IRC | 05:34 | |
| *** rcernin has quit IRC | 05:41 | |
| *** ratailor has joined #openstack-nova | 05:42 | |
| *** rcernin has joined #openstack-nova | 05:58 | |
| openstackgerrit | wangwei1 proposed openstack/nova stable/stein: fix spelling error in nova/api/validation/__init__.py https://review.opendev.org/669251 | 06:00 |
| *** shilpasd has quit IRC | 06:00 | |
| *** luksky11 has joined #openstack-nova | 06:06 | |
| *** slaweq has joined #openstack-nova | 06:12 | |
| *** slaweq has quit IRC | 06:17 | |
| *** slaweq has joined #openstack-nova | 06:27 | |
| *** ivve has joined #openstack-nova | 06:31 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova-specs master: Fix the cross_az_attach value in AZ-unshelved spec https://review.opendev.org/669261 | 06:32 |
| *** slaweq has quit IRC | 06:35 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration https://review.opendev.org/640021 | 06:39 |
| *** slaweq has joined #openstack-nova | 06:46 | |
| *** maciejjozefczyk has joined #openstack-nova | 06:54 | |
| *** boxiang has joined #openstack-nova | 06:59 | |
| *** jaosorior has joined #openstack-nova | 07:02 | |
| *** ivve has quit IRC | 07:06 | |
| *** rpittau|afk is now known as rpittau | 07:06 | |
| *** xek has joined #openstack-nova | 07:06 | |
| *** ricolin__ has joined #openstack-nova | 07:10 | |
| *** ricolin_ has quit IRC | 07:13 | |
| *** damien_r has joined #openstack-nova | 07:14 | |
| *** hoonetorg has quit IRC | 07:16 | |
| openstackgerrit | Yongli He proposed openstack/nova master: Clean up orphan instances virt driver https://review.opendev.org/648912 | 07:18 |
| openstackgerrit | Yongli He proposed openstack/nova master: clean up orphan instances https://review.opendev.org/627765 | 07:18 |
| *** tesseract has joined #openstack-nova | 07:21 | |
| *** ccamacho has joined #openstack-nova | 07:23 | |
| openstackgerrit | Yongli He proposed openstack/nova master: clean up orphan instances https://review.opendev.org/627765 | 07:24 |
| *** helenafm has joined #openstack-nova | 07:27 | |
| *** tssurya has joined #openstack-nova | 07:27 | |
| *** hoonetorg has joined #openstack-nova | 07:29 | |
| *** ricolin__ is now known as ricolin | 07:44 | |
| *** rcernin has quit IRC | 07:45 | |
| *** ttsiouts has joined #openstack-nova | 07:55 | |
| *** ralonsoh has joined #openstack-nova | 08:00 | |
| *** rcernin has joined #openstack-nova | 08:02 | |
| *** ivve has joined #openstack-nova | 08:06 | |
| *** ttsiouts has quit IRC | 08:06 | |
| *** ttsiouts has joined #openstack-nova | 08:07 | |
| *** xek has quit IRC | 08:10 | |
| *** ttsiouts has quit IRC | 08:11 | |
| *** tkajinam_ has quit IRC | 08:11 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova-specs master: Fix indentation in the "Secure Boot for KVM ... guests" spec https://review.opendev.org/669284 | 08:12 |
| kashyap | stephenfin: johnthetubaguy: A trivial cosmetic fix --^ (rST syntax related) | 08:13 |
| *** lpetrut has joined #openstack-nova | 08:23 | |
| *** lpetrut has quit IRC | 08:24 | |
| *** lpetrut has joined #openstack-nova | 08:25 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Get rid of args to RBDDriver.__init__() https://review.opendev.org/668564 | 08:25 |
| *** ttsiouts has joined #openstack-nova | 08:26 | |
| *** tetsuro has quit IRC | 08:29 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove native LUKS compat code https://review.opendev.org/669121 | 08:35 |
| *** tetsuro has joined #openstack-nova | 08:38 | |
| *** tetsuro has quit IRC | 08:40 | |
| *** rcernin has quit IRC | 08:49 | |
| *** ociuhandu has joined #openstack-nova | 09:04 | |
| *** takashin has left #openstack-nova | 09:05 | |
| *** rcernin has joined #openstack-nova | 09:05 | |
| *** jistr has quit IRC | 09:07 | |
| *** ricolin has quit IRC | 09:08 | |
| *** belmoreira has joined #openstack-nova | 09:12 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Fix spelling error in 'nova/api/validation/__init__.py' https://review.opendev.org/669244 | 09:12 |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/stein: Fix spelling error in 'nova/api/validation/__init__.py' https://review.opendev.org/669251 | 09:14 |
| *** jistr has joined #openstack-nova | 09:14 | |
| kashyap | stephenfin: Hope I answered your question there: https://review.opendev.org/#/c/669129/ | 09:15 |
| kashyap | (Not an entirely satisfactory one; but it's the truth :-)) | 09:16 |
| kashyap | New rev upcoming. | 09:16 |
| *** rcernin has quit IRC | 09:16 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_PERF_VERSION https://review.opendev.org/669128 | 09:17 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove support for Intel CMT `perf` events https://review.opendev.org/669129 | 09:17 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: vif: Remove MIN_LIBVIRT_MACVTAP_PASSTHROUGH_VLAN https://review.opendev.org/669176 | 09:17 |
| openstackgerrit | Merged openstack/nova-specs master: Fix indentation in the "Secure Boot for KVM ... guests" spec https://review.opendev.org/669284 | 09:25 |
| *** abhishekk is now known as abhishekk|interv | 09:34 | |
| *** jaosorior has quit IRC | 09:36 | |
| *** belmoreira has quit IRC | 09:52 | |
| *** stakeda has quit IRC | 09:57 | |
| *** ttsiouts has quit IRC | 10:08 | |
| *** ttsiouts has joined #openstack-nova | 10:08 | |
| *** abhishekk|interv is now known as abhishekk | 10:10 | |
| *** ttsiouts has quit IRC | 10:13 | |
| *** abhishekk has quit IRC | 10:25 | |
| *** brault has joined #openstack-nova | 10:32 | |
| *** luksky11 has quit IRC | 10:32 | |
| *** xek has joined #openstack-nova | 10:36 | |
| *** brault has quit IRC | 10:37 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Fix live migration break group policy simultaneously https://review.opendev.org/651969 | 10:38 |
| bauzas | gibi: remember me, do we document in the API docs something like https://review.opendev.org/#/c/630723/4/nova/api/openstack/compute/migrate_server.py ? | 10:40 |
| bauzas | not the API guide I mean | 10:40 |
| bauzas | I can read https://docs.openstack.org/neutron/latest/admin/config-qos-min-bw.html tho | 10:41 |
| bauzas | and https://developer.openstack.org/api-guide/compute/port_with_resource_request.html document what's missing | 10:41 |
| bauzas | but I wonder if within the API docs, we said that too | 10:42 |
| bauzas | I mean api-ref | 10:42 |
| bauzas | lemme check actually | 10:42 |
| bauzas | nope, we don't (eg. https://developer.openstack.org/api-ref/compute/?expanded=resize-server-resize-action-detail#resize-server-resize-action ) | 10:44 |
| bauzas | anyway, not a big deal, it's just for internal docs | 10:44 |
| bauzas | gibi: FWIW, ping me if you have any changes about ^ btw. | 10:44 |
| *** boxiang has quit IRC | 10:50 | |
| *** maciejjozefczyk has quit IRC | 11:00 | |
| *** maciejjozefczyk has joined #openstack-nova | 11:01 | |
| *** luksky11 has joined #openstack-nova | 11:02 | |
| mdbooth | >>> x.config_drive = False | 11:04 |
| mdbooth | >>> x.config_drive | 11:04 |
| mdbooth | u'False' | 11:04 |
| mdbooth | >>> x.config_drive = True | 11:04 |
| mdbooth | Don't you just love python? | 11:04 |
| mdbooth | And that field has the wrong type, of course | 11:05 |
| mdbooth | >>> x.config_drive = False | 11:06 |
| mdbooth | >>> bool(x.config_drive) | 11:06 |
| mdbooth | True | 11:06 |
| *** whoami-rajat has quit IRC | 11:06 | |
| openstackgerrit | Merged openstack/nova master: api-ref: Fix a broken link https://review.opendev.org/669248 | 11:06 |
| *** trident has quit IRC | 11:10 | |
| *** belmoreira has joined #openstack-nova | 11:13 | |
| *** trident has joined #openstack-nova | 11:14 | |
| *** brault has joined #openstack-nova | 11:14 | |
| *** cdent has joined #openstack-nova | 11:16 | |
| *** mrch_ has joined #openstack-nova | 11:25 | |
| *** ratailor has quit IRC | 11:28 | |
| *** tbachman has quit IRC | 11:37 | |
| *** brault has quit IRC | 11:46 | |
| *** brault has joined #openstack-nova | 11:47 | |
| stephenfin | kashyap: Replied to https://review.opendev.org/#/c/669129/ | 11:47 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: vif: Remove MIN_LIBVIRT_MACVTAP_PASSTHROUGH_VLAN https://review.opendev.org/669176 | 11:48 |
| *** ttsiouts has joined #openstack-nova | 11:49 | |
| gibi | bauzas: we have the limitation separately, not in the api-ref. I think it was intentional as it is expected to be a temporary limitation | 11:51 |
| gibi | bauzas: the actual implementation of server move operation still needs my time. it is started up here https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/support-move-ops-with-qos-ports | 11:51 |
| gibi | bauzas: but I was busy with https://review.opendev.org/#/c/637955/ | 11:52 |
| *** yaawang has quit IRC | 12:00 | |
| *** yaawang has joined #openstack-nova | 12:02 | |
| *** openstackgerrit has quit IRC | 12:04 | |
| *** tbachman has joined #openstack-nova | 12:05 | |
| *** whoami-rajat has joined #openstack-nova | 12:19 | |
| *** belmoreira has quit IRC | 12:21 | |
| *** xek has quit IRC | 12:26 | |
| *** xek_ has joined #openstack-nova | 12:26 | |
| kashyap | stephenfin: Will respond (saddled with a couple off meetings); thanks for the thoughtful reply. | 12:29 |
| *** psachin has joined #openstack-nova | 12:33 | |
| *** belmoreira has joined #openstack-nova | 12:34 | |
| *** openstackgerrit has joined #openstack-nova | 12:41 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Get rid of args to RBDDriver.__init__() https://review.opendev.org/668564 | 12:41 |
| *** eharney has joined #openstack-nova | 12:47 | |
| bauzas | gibi: ack ok | 12:47 |
| *** udesale has quit IRC | 12:51 | |
| *** udesale has joined #openstack-nova | 12:51 | |
| *** tesseract has quit IRC | 12:52 | |
| *** shilpasd has joined #openstack-nova | 12:52 | |
| *** belmoreira has quit IRC | 12:53 | |
| *** belmorei_ has joined #openstack-nova | 12:53 | |
| *** tesseract has joined #openstack-nova | 12:54 | |
| *** cdent has quit IRC | 13:03 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Error out interrupted builds https://review.opendev.org/666857 | 13:05 |
| *** jangutter has quit IRC | 13:13 | |
| *** jangutter has joined #openstack-nova | 13:14 | |
| *** mrch_ has quit IRC | 13:24 | |
| *** tbachman has quit IRC | 13:30 | |
| *** tbachman has joined #openstack-nova | 13:33 | |
| *** cdent has joined #openstack-nova | 13:33 | |
| *** mriedem has joined #openstack-nova | 13:36 | |
| *** trident has quit IRC | 13:41 | |
| *** Luzi has quit IRC | 13:50 | |
| *** tbachman has quit IRC | 13:53 | |
| *** ratailor has joined #openstack-nova | 14:00 | |
| *** tbachman has joined #openstack-nova | 14:03 | |
| *** belmorei_ has quit IRC | 14:09 | |
| *** cdent has quit IRC | 14:09 | |
| *** _alastor_ has joined #openstack-nova | 14:12 | |
| mriedem | lyarwood: have you actually hit a recreate for this? https://review.opendev.org/#/q/I15a7c13edf78884ec223fd531a78a341106b41b8 | 14:12 |
| lyarwood | mriedem: did you mean reason? | 14:15 |
| mriedem | no, | 14:15 |
| mriedem | when that was reported there was no recreate in the bug for it, | 14:15 |
| mriedem | and i was unable to recreate it in a functional test | 14:15 |
| mriedem | so i'm hesitant to backport that without knowing how to recreate it | 14:15 |
| lyarwood | mriedem: ah reproducer | 14:15 |
| mriedem | yeah | 14:16 |
| lyarwood | mriedem: no, that's fair it just looked like a trivial thing to change | 14:16 |
| mriedem | ok i'd rather not backport it unless we have someone else reporting the problem | 14:16 |
| lyarwood | mriedem: ack that's fine, I'll take a look next week at reproducing, if I can't I'll close | 14:16 |
| mriedem | btw it occurs to me now we maybe shouldn't have merged this https://review.opendev.org/#/c/663887/ | 14:18 |
| mriedem | b/c i can't run fast8/pep8 on my bionic vm (py27 and py36) on rocky changes now | 14:19 |
| mriedem | stephenfin: ^ | 14:19 |
| stephenfin | Because bionic doesn't support py3.5? | 14:21 |
| stephenfin | It's a tough one. Without it, I can't run the same on my Fedora 30 machine for the reasons in the commit message | 14:22 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Add functional recreate test for regression bug 1825537 https://review.opendev.org/669361 | 14:23 |
| openstack | bug 1825537 in OpenStack Compute (nova) rocky "finish_resize failures incorrectly revert allocations" [Medium,Confirmed] https://launchpad.net/bugs/1825537 | 14:23 |
| mriedem | rocky doesn't support py37 at all though right? | 14:23 |
| mriedem | rocky tox.ini at least has py36 targets | 14:23 |
| sean-k-mooney | stephenfin: it use py36 | 14:23 |
| mriedem | iow, if you're doing rocky stuff and only have 3.7, you're going to need another vm | 14:23 |
| mriedem | bionic has 2.7 and 3.6 | 14:24 |
| stephenfin | It doesn't, no, but the use of python3 means "give me any version of Python 3.x" | 14:24 |
| sean-k-mooney | stephenfin: we dont support py3.5 anymore | 14:24 |
| sean-k-mooney | its not part of the train PTI | 14:24 |
| stephenfin | We did in rocky though | 14:24 |
| stephenfin | That's a stable-only patch | 14:24 |
| sean-k-mooney | rocky proably yes | 14:24 |
| stephenfin | mriedem: Could we use basepython = python3.6 instead, assuming that's available on both Bionic and Xenial (or whatever 16.04 is?) | 14:25 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Revert "tox: Use basepython of python3.5" https://review.opendev.org/669362 | 14:25 |
| mriedem | stephenfin: in rocky i don't think so, | 14:25 |
| mriedem | because rocky supported (in the gate and the tox default env list) 27 and 35 | 14:26 |
| mriedem | but has a 36 target | 14:26 |
| mriedem | there is no official 37 test targeting in rocky though | 14:26 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova-specs master: Boot a VM with an unaddressed port https://review.opendev.org/641670 | 14:27 |
| mriedem | so like i said, i think if you're doing rocky changes and only have 37, you'll just need another dev vm for working on those older branches (which is what i have to do for ocata changes at this point) | 14:27 |
| mriedem | i have a xenial vm just for doing ocata stuff now | 14:27 |
| mriedem | xenial less for the python thing but more b/c of native packages like postgresql | 14:27 |
| *** zul has quit IRC | 14:29 | |
| *** ttsiouts has quit IRC | 14:30 | |
| *** ttsiouts has joined #openstack-nova | 14:31 | |
| *** ttsiouts has quit IRC | 14:36 | |
| *** ttsiouts has joined #openstack-nova | 14:37 | |
| *** belmoreira has joined #openstack-nova | 14:58 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove nova-consoleauth https://review.opendev.org/652969 | 15:04 |
| stephenfin | lyarwood: The stable/stein variant of this has merged now. Could you tackle this one? https://review.opendev.org/#/c/667355/ | 15:07 |
| *** tesseract-RH has joined #openstack-nova | 15:08 | |
| *** tbachman has quit IRC | 15:08 | |
| *** xek_ has quit IRC | 15:09 | |
| *** tesseract has quit IRC | 15:10 | |
| *** rpittau is now known as rpittau|afk | 15:12 | |
| *** cdent has joined #openstack-nova | 15:13 | |
| *** tbachman has joined #openstack-nova | 15:14 | |
| dansmith | sean-k-mooney: mriedem: Artom made several changes to the revert resize patch yesterday but didn't respond to comments and also rebased it so there's a lot of change.. did I miss any discussion about that? | 15:20 |
| dansmith | looks like he removed the try..except around the signature change which was my deal, but otherwise I either need to eye-diff it or pull it down | 15:21 |
| sean-k-mooney | dansmith: he change it to the db lookup | 15:21 |
| *** lpetrut has quit IRC | 15:21 | |
| sean-k-mooney | right | 15:21 |
| sean-k-mooney | i havent looked at it this mornig | 15:22 |
| dansmith | alright | 15:22 |
| *** damien_r has quit IRC | 15:22 | |
| mriedem | i was going to have to eye-diff anyway | 15:23 |
| *** gyee has joined #openstack-nova | 15:28 | |
| *** tssurya has quit IRC | 15:32 | |
| sean-k-mooney | artom is on pto while he is flying home to canada today but if there are small things like mising test exctra ill be around for an hour or two to fix them | 15:34 |
| sean-k-mooney | i think artom is back on monday but if not ill continue looking at this until he is back. | 15:34 |
| *** belmoreira has quit IRC | 15:36 | |
| *** luksky11 has quit IRC | 15:38 | |
| *** helenafm has quit IRC | 15:42 | |
| *** shilpasd has quit IRC | 15:46 | |
| *** maciejjozefczyk has quit IRC | 15:48 | |
| *** tbachman has quit IRC | 15:51 | |
| *** tbachman has joined #openstack-nova | 15:54 | |
| dansmith | sean-k-mooney: I don't see the "will wait for bind time events" string in the logs | 15:55 |
| dansmith | on the hybrid plug test in experimental | 15:55 |
| dansmith | maybe that test is single-host which means we're not running any of this new stuff anyway? | 15:55 |
| sean-k-mooney | it is | 15:56 |
| *** ratailor_ has joined #openstack-nova | 15:56 | |
| *** ratailor_ has quit IRC | 15:56 | |
| sean-k-mooney | it is testign the resize to same host case | 15:56 |
| dansmith | ah yeah I see the same host debug message instead | 15:56 |
| sean-k-mooney | https://review.opendev.org/#/c/664442/14 is testing resize to different host | 15:56 |
| *** ratailor has quit IRC | 15:56 | |
| dansmith | ah, right forgot that was on top and was thinking we'd get testing from mriedem's patch, I see | 15:57 |
| sean-k-mooney | ya this is the message form the dnm change http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/screen-n-cpu.txt.gz#_Jul_04_19_49_57_371761 | 15:58 |
| *** mdbooth_ has joined #openstack-nova | 15:58 | |
| dansmith | yeah, but lower we receive and discard that event: http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/screen-n-cpu.txt.gz#_Jul_04_19_49_58_003384 | 15:59 |
| dansmith | see the "no event matching" above that | 15:59 |
| sean-k-mooney | ya i think this might be the souce node i need to check the other node | 15:59 |
| sean-k-mooney | and confim this is the right vm | 15:59 |
| dansmith | we shouldn't emit the "will wait..." message on the wrong node right? | 15:59 |
| sean-k-mooney | we should not e might get thos events on both nodes | 16:00 |
| dansmith | receive, but not waiting | 16:00 |
| dansmith | actually, that's the unplug event | 16:00 |
| dansmith | we get the plug event shortly after that: | 16:00 |
| dansmith | http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/screen-n-cpu.txt.gz#_Jul_04_19_50_00_983164 | 16:01 |
| *** mdbooth has quit IRC | 16:02 | |
| sean-k-mooney | i dont kno if we ever actully wait for unplugged events do we | 16:02 |
| *** mdbooth_ has quit IRC | 16:02 | |
| sean-k-mooney | it looks like libvirt died oh that vm breilfy http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/screen-n-cpu.txt.gz#_Jul_04_19_59_12_862599 | 16:08 |
| sean-k-mooney | which is unrelated but annoying | 16:08 |
| sean-k-mooney | actully that is part of the testing i think | 16:08 |
| sean-k-mooney | i think we are killing it so we can do evacuate testing | 16:09 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: WIP: Drop source node allocations if finish_resize fails https://review.opendev.org/669393 | 16:15 |
| mriedem | the nova-live-migration job, renamed to nova-multinode in 664442, yes does evacuate testing and kills libvirtd on the controller node | 16:17 |
| sean-k-mooney | in any case we start waiting for the event here http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/subnode-2/screen-n-cpu.txt.gz#_Jul_04_19_49_48_754192 and we recive it here http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/subnode-2/screen-n-cpu.txt.gz#_Jul_04_19_49_51_905792 | 16:19 |
| sean-k-mooney | there is a vif unplugged and plugged event recrved just before we wait that im not sure where they came form | 16:20 |
| sean-k-mooney | http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/subnode-2/screen-n-cpu.txt.gz#_Jul_04_19_49_40_441961 and http://logs.openstack.org/42/664442/14/check/nova-multinode/7c532c4/logs/subnode-2/screen-n-cpu.txt.gz#_Jul_04_19_49_45_258629 | 16:20 |
| *** ttsiouts has quit IRC | 16:21 | |
| sean-k-mooney | but it does look like we do wait as expected | 16:21 |
| *** ttsiouts has joined #openstack-nova | 16:21 | |
| *** jangutter has quit IRC | 16:22 | |
| *** ttsiouts has quit IRC | 16:26 | |
| *** udesale has quit IRC | 16:28 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Drop source node allocations if finish_resize fails https://review.opendev.org/669393 | 16:31 |
| mriedem | lyarwood: dansmith: ^ this backport got a bit hairy because before stein we had to deal with non-migration-based allocations, | 16:31 |
| mriedem | so i've documentd the difference in the commit message and code, and added a functional test for the non-migration-based allocations legacy wrinkle | 16:31 |
| mriedem | just fyi so it doesn't get fast-approved as a stable core | 16:31 |
| *** cdent has quit IRC | 16:32 | |
| dansmith | ugh | 16:35 |
| *** slaweq has quit IRC | 16:52 | |
| *** tesseract-RH has quit IRC | 17:01 | |
| *** factor has joined #openstack-nova | 17:03 | |
| *** whoami-rajat has quit IRC | 17:06 | |
| *** factor has quit IRC | 17:08 | |
| *** luksky11 has joined #openstack-nova | 17:14 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: [WIP] Allow to spawn VMs with port without IP address https://review.opendev.org/669411 | 17:18 |
| redkrieg | I'm having an interesting issue with a large flavor not being deployable on a compute host but two flavors of half the size working fine. the host has just enough disk and ram for the flavor, plus disk_allocation_ratio and ram_allocation_ratio are > 1. Nova scheduler logs with debug enabled don't even show the host in question during filtering, so I think it's being removed earlier somehow. I | 17:22 |
| redkrieg | see accurate information in the "Update host state from compute node" ComputeNode output just minutes before launching an instance. Can anyone suggest a reason it wouldn't even be considered for filtering? | 17:22 |
| *** ralonsoh has quit IRC | 17:23 | |
| redkrieg | I've narrowed it down to a ram issue by dropping my flavor's ram requirement to 30000 from 32768. the logs show that the hypervisor is reporting 32141 but even 32000 doesn't work. Is there some hard limit I don't know about that bypasses the allocation ratio settings? | 17:37 |
| *** psachin has quit IRC | 17:42 | |
| *** factor has joined #openstack-nova | 17:48 | |
| *** ociuhandu_ has joined #openstack-nova | 17:54 | |
| *** slaweq has joined #openstack-nova | 17:55 | |
| *** ociuhandu_ has quit IRC | 17:55 | |
| *** ociuhandu has quit IRC | 17:57 | |
| openstackgerrit | Corey Bryant proposed openstack/nova master: Add Python 3 Train unit tests https://review.opendev.org/669437 | 17:58 |
| openstackgerrit | Corey Bryant proposed openstack/os-vif master: Add Python 3 Train unit tests https://review.opendev.org/669438 | 18:00 |
| openstackgerrit | Corey Bryant proposed openstack/python-novaclient master: Add Python 3 Train unit tests https://review.opendev.org/669439 | 18:01 |
| *** slaweq has quit IRC | 18:05 | |
| mriedem | redkrieg: which release? placement could be filtering it out - enable debug logging in the placement logs and you should see the allocation candidate filtering messages (on newer releases), there is the reserved_host_ram_mb config option on the compute that will be reporting reserved MEMORY_MB to placement | 18:08 |
| mriedem | which is part of the overall capacity calculation | 18:08 |
| Nick_A | mriedem - rocky, phys_ram=32141MB used_ram=0MB, reserved_host_memory_mb = 0 in nova.conf | 18:12 |
| Nick_A | we found https://docs.openstack.org/arch-design/design-compute/design-compute-overcommit.html which explains why 32768 doesn't work, but we're not seeing why 32000 is also failing | 18:12 |
| *** ociuhandu has joined #openstack-nova | 18:13 | |
| *** ociuhandu has quit IRC | 18:17 | |
| mriedem | are you using the RamFilter or AggregateRamFilter? | 18:17 |
| *** factor has quit IRC | 18:20 | |
| redkrieg | yes, we do have the AggregateRamFilter active. I don't see this host being filtered by it though. | 18:21 |
| openstackgerrit | Merged openstack/nova master: libvirt: Remove MIN_{QEMU,LIBVIRT}_LUKS_VERSION https://review.opendev.org/668924 | 18:25 |
| openstackgerrit | Merged openstack/nova-specs master: Fix the cross_az_attach value in AZ-unshelved spec https://review.opendev.org/669261 | 18:31 |
| mriedem | the reason i ask is because those filters return a "limit" from the scheduler which gets passed down to the compute which is part of the resource claim, | 18:34 |
| *** factor has joined #openstack-nova | 18:34 | |
| mriedem | and if the scheduler is passing down a limit for something that you don't expect, that might be the cause | 18:35 |
| mriedem | https://docs.openstack.org/nova/latest/admin/configuration/schedulers.html#allocation-ratios | 18:35 |
| mriedem | iow, what you think and what placement is using might be different allocation ratios | 18:37 |
| mriedem | hopefully that doc gives some things to look at | 18:37 |
| *** factor has quit IRC | 18:40 | |
| *** factor has joined #openstack-nova | 18:40 | |
| *** factor has quit IRC | 18:47 | |
| Nick_A | It actually looks like we're dealing with some sort of qemu/kernel bug... It successfully attempts a claim on compute if we set the flavor RAM less than the physical RAM, but libvirt errors at certain amounts with "qemu-system-x86_64: cannot set up guest memory 'pc.ram': Cannot allocate memory." | 18:48 |
| *** tbachman has quit IRC | 18:56 | |
| dansmith | Nick_A: using anything that requires pinned memory? cpu pinning, huge pages, etc? | 19:15 |
| dansmith | also, even if you have 32g of memory, you can't necessarily expect to land a 30g instance there if you have no swap and have a large enough footprint of other stuff | 19:16 |
| *** factor has joined #openstack-nova | 19:16 | |
| dansmith | but, if you're getting to libvirt, then you're out of the realm of it being a nova thing | 19:17 |
| Nick_A | we are doing cpu pinning on here - yeah, I think the only nova issue was the hard limit based on physical ram | 19:17 |
| Nick_A | I am curious why that's the case if you happen to know | 19:17 |
| Nick_A | we have non-openstack centos libvirt nodes that are happy to create KVMs with over 32GB on the same hardware | 19:18 |
| dansmith | do you also have a numa topology specified? | 19:18 |
| dansmith | there's no 32g limit specifically of course | 19:18 |
| Nick_A | "Regardless of the overcommit ratio, an instance can not be placed on any physical node with fewer raw (pre-overcommit) resources than the instance flavor requires." Am I misunderstanding that? | 19:19 |
| *** tbachman has joined #openstack-nova | 19:19 | |
| Nick_A | re: numa - we're using openstack-ansible defaults, not sure | 19:19 |
| Nick_A | I got that from https://docs.openstack.org/arch-design/design-compute/design-compute-overcommit.html | 19:21 |
| *** slaweq has joined #openstack-nova | 19:21 | |
| dansmith | no, I'm just saying there's no specific 32g limit. nova's overcommit is for landing multiple instances on a node, however ill-advised | 19:21 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old TODO about forced_host policy check https://review.opendev.org/669474 | 19:21 |
| dansmith | a single instance larger than physical memory is silly, so yeah, no support for it | 19:22 |
| Nick_A | right, but there is a limit based on the physical RAM preventing a single instance from being allocated above that amount. I'm curious why it allows multiple instances to do so but not a single one. | 19:22 |
| dansmith | it allows it because it was designed that way for earlier cloud providers that did it for cost scale, but it's never a good idea for performance, obviously, as you're just using host disk for the overage | 19:23 |
| dansmith | a single instance that is larger than the host doesn't make sense, you might as well run a smaller guest and let it do its own swapping | 19:24 |
| dansmith | anyway, if you're doing cpu pinning you might be running up against something that requires page pinning which is getting you to the "cannot allocate memory" thing | 19:25 |
| dansmith | I doubt it's a kernel/qemu bug | 19:25 |
| dansmith | best bet would be to ask around monday morning when more of the folks that are familiar with that stuff are around | 19:25 |
| Nick_A | thank you | 19:25 |
| dansmith | sean, sfinucan, etc | 19:25 |
| *** wwriverrat has quit IRC | 19:28 | |
| openstackgerrit | Corey Bryant proposed openstack/os-resource-classes master: Add Python 3 Train unit tests https://review.opendev.org/669479 | 19:43 |
| openstackgerrit | Corey Bryant proposed openstack/os-traits master: Add Python 3 Train unit tests https://review.opendev.org/669480 | 19:44 |
| mriedem | bauzas: how is your audit command coming? it's been a couple of weeks and this came up in the ML today http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007534.html | 20:15 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove Newton-era min compute checks for server create with device tags https://review.opendev.org/669523 | 20:35 |
| mriedem | dansmith: is https://review.opendev.org/#/c/667177/ just waiting for my eye-ball diff and if we're good, fix the thing you noted and then we can move forward? | 20:37 |
| dansmith | mriedem: yeah | 20:37 |
| *** rdopiera has quit IRC | 21:13 | |
| mriedem | ok comments in there now | 21:16 |
| mriedem | also - https://review.opendev.org/#/c/668631/ - it's a good thing we didn't do the interface change in the same patch... | 21:17 |
| *** maciejjozefczyk has joined #openstack-nova | 21:17 | |
| *** ivve has quit IRC | 21:25 | |
| *** maciejjozefczyk has quit IRC | 21:25 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Init HostState.failed_builds https://review.opendev.org/669538 | 21:31 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove Rocky-era min compute trusted certs compat check https://review.opendev.org/669539 | 21:37 |
| *** tbachman has quit IRC | 22:01 | |
| sean-k-mooney | Nick_A: im not sure if this is the issue but i do not belive we allow vms to oversubsribe resouces against itself. | 22:02 |
| sean-k-mooney | in placment the max_unit for MEMORY_MB should be set to the total capasity of the host | 22:02 |
| sean-k-mooney | so we will not allow vms that request more ram then the host has to be schduled to a node | 22:03 |
| sean-k-mooney | the same is true from ram and disk | 22:03 |
| sean-k-mooney | i havent read the full scrollback but that is the first limitation that came to mind | 22:03 |
| sean-k-mooney | reading back | 22:05 |
| sean-k-mooney | we reserve 512MB of ram for the host by default | 22:05 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add negative test for cross-cell finish_resize failing https://review.opendev.org/643451 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Refresh instance in MigrationTask.execute Exception handler https://review.opendev.org/669012 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add negative test for prep_snapshot_based_resize_at_source failing https://review.opendev.org/669013 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source https://review.opendev.org/637058 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask https://review.opendev.org/637070 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add confirm_snapshot_based_resize conductor RPC method https://review.opendev.org/637075 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize from the API https://review.opendev.org/637316 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add revert_snapshot_based_resize_at_dest compute method https://review.opendev.org/637630 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deal with cross-cell resize in _remove_deleted_instances_allocations https://review.opendev.org/639453 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add finish_revert_snapshot_based_resize_at_source compute method https://review.opendev.org/637647 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask https://review.opendev.org/638046 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.opendev.org/638047 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.opendev.org/638048 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.opendev.org/638268 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add archive_deleted_rows wrinkle to cross-cell functional test https://review.opendev.org/651650 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.opendev.org/614353 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional test for anti-affinity cross-cell migration https://review.opendev.org/661859 | 22:08 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Support cross-cell moves in external_instance_event https://review.opendev.org/658478 | 22:08 |
| Nick_A | sean-k-mooney thanks | 22:10 |
| sean-k-mooney | i would have to think about it some more but im guessing you are hiting one of the limit we have to make sure a single vm does not exceed the total amount of one of the resouce we have on the host | 22:11 |
| *** jistr has quit IRC | 22:24 | |
| *** jistr has joined #openstack-nova | 22:24 | |
| Nick_A | yeah that's clearly why 32768 wasn't working. We were just surprised it had a limit of that sort when the RAM allocated to multiple smaller instances could still cross that threshold. | 22:30 |
| *** luksky11 has quit IRC | 22:33 | |
| sean-k-mooney | i think the qemu error is proably a similar similar safte check | 22:36 |
| *** dklyle has quit IRC | 22:36 | |
| sean-k-mooney | the os and qemu both need some ram to work properly. so there is some overhad per host that should not be allocated to a guest unless you have enough swap space | 22:37 |
| sean-k-mooney | but even then i generally do not recommend ram over commit at all | 22:37 |
| sean-k-mooney | it causes more issues the in sovles as the kernel OOM killer will reap the vms first if it need to allocate memory and it can swap the vm memory to make room | 22:38 |
| *** tbachman has joined #openstack-nova | 22:39 | |
| *** tbachman_ has joined #openstack-nova | 22:40 | |
| *** tbachman has quit IRC | 22:43 | |
| *** tbachman_ is now known as tbachman | 22:43 | |
| *** mriedem has quit IRC | 22:47 | |
| openstackgerrit | Merged openstack/nova master: libvirt: Remove MIN_LIBVIRT_PERF_VERSION https://review.opendev.org/669128 | 22:54 |
| openstackgerrit | Merged openstack/nova master: libvirt: vif: Remove MIN_LIBVIRT_MACVTAP_PASSTHROUGH_VLAN https://review.opendev.org/669176 | 23:00 |
| *** sapd1 has quit IRC | 23:05 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!