| *** ccamacho has quit IRC | 00:01 | |
| *** tetsuro has joined #openstack-nova | 00:09 | |
| *** tetsuro_ has joined #openstack-nova | 00:10 | |
| *** tetsuro has quit IRC | 00:13 | |
| *** avolkov has quit IRC | 00:35 | |
| *** gyee has quit IRC | 00:36 | |
| *** k_mouza has joined #openstack-nova | 00:51 | |
| *** k_mouza has quit IRC | 00:56 | |
| *** k_mouza has joined #openstack-nova | 00:58 | |
| *** k_mouza has quit IRC | 01:04 | |
| melwitt | mnaser: I implemented a re-architecting of quotas in pike, which began counting quota usage on-the-fly by counting resources (this only does a read). prior to pike, we did do the select-for-update locking thing | 01:09 |
|---|---|---|
| *** huaqiang has quit IRC | 01:18 | |
| *** k_mouza has joined #openstack-nova | 01:27 | |
| *** k_mouza has quit IRC | 01:31 | |
| *** mriedem has left #openstack-nova | 02:01 | |
| *** spatel has joined #openstack-nova | 02:10 | |
| *** Yumeng has joined #openstack-nova | 02:18 | |
| *** sapd__x has joined #openstack-nova | 02:20 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/nova master: Support inject-nmi action in watchdog https://review.opendev.org/741072 | 02:26 |
| *** tetsuro_ has quit IRC | 02:40 | |
| *** spatel has quit IRC | 02:49 | |
| *** rcernin has quit IRC | 02:56 | |
| *** mkrai has joined #openstack-nova | 02:57 | |
| *** rcernin has joined #openstack-nova | 02:57 | |
| *** ociuhandu has joined #openstack-nova | 03:01 | |
| *** ociuhandu has quit IRC | 03:06 | |
| *** brinzhang0 has joined #openstack-nova | 03:15 | |
| openstackgerrit | Merged openstack/nova master: doc: Correct typo https://review.opendev.org/740670 | 03:18 |
| *** brinzhang_ has quit IRC | 03:18 | |
| *** brinzhang has joined #openstack-nova | 03:19 | |
| *** spatel has joined #openstack-nova | 03:22 | |
| *** spatel has quit IRC | 03:22 | |
| *** brinzhang0 has quit IRC | 03:22 | |
| *** spatel has joined #openstack-nova | 03:23 | |
| *** spatel has quit IRC | 03:23 | |
| *** spatel has joined #openstack-nova | 03:30 | |
| *** spatel has quit IRC | 03:30 | |
| openstackgerrit | Keigo Noha proposed openstack/nova master: Change default num_retries for glance to 3 https://review.opendev.org/740389 | 03:32 |
| *** redrobot has quit IRC | 03:36 | |
| *** psachin has joined #openstack-nova | 03:39 | |
| *** brinzhang_ has joined #openstack-nova | 03:41 | |
| *** brinzhang has quit IRC | 03:44 | |
| *** k_mouza has joined #openstack-nova | 03:44 | |
| *** k_mouza has quit IRC | 03:48 | |
| *** brinzhang0 has joined #openstack-nova | 04:03 | |
| *** brinzhang_ has quit IRC | 04:05 | |
| *** yaawang has quit IRC | 04:10 | |
| *** markvoelker has joined #openstack-nova | 04:11 | |
| *** yaawang has joined #openstack-nova | 04:11 | |
| *** sapd__x has quit IRC | 04:12 | |
| *** markvoelker has quit IRC | 04:15 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-nova | 04:33 | |
| *** ratailor has joined #openstack-nova | 04:47 | |
| *** vishalmanchanda has joined #openstack-nova | 04:50 | |
| *** sapd__x has joined #openstack-nova | 04:58 | |
| *** markvoelker has joined #openstack-nova | 05:01 | |
| *** markvoelker has quit IRC | 05:05 | |
| *** links has joined #openstack-nova | 05:11 | |
| *** k_mouza has joined #openstack-nova | 05:13 | |
| *** brinzhang_ has joined #openstack-nova | 05:16 | |
| *** yaawang has quit IRC | 05:16 | |
| *** yaawang has joined #openstack-nova | 05:16 | |
| *** k_mouza has quit IRC | 05:18 | |
| *** brinzhang0 has quit IRC | 05:18 | |
| *** brinzhang0 has joined #openstack-nova | 05:21 | |
| *** udesale has joined #openstack-nova | 05:22 | |
| *** brinzhang_ has quit IRC | 05:24 | |
| *** ratailor has quit IRC | 05:27 | |
| *** ratailor has joined #openstack-nova | 05:27 | |
| *** bbowen has quit IRC | 05:32 | |
| *** bbowen has joined #openstack-nova | 05:34 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/nova master: Support inject-nmi action in watchdog https://review.opendev.org/741072 | 05:38 |
| *** brinzhang_ has joined #openstack-nova | 05:40 | |
| *** brinzhang0 has quit IRC | 05:43 | |
| *** udesale has quit IRC | 06:07 | |
| *** bbowen has quit IRC | 06:17 | |
| *** udesale has joined #openstack-nova | 06:20 | |
| openstackgerrit | Jie Li proposed openstack/nova-specs master: add volume backed server rebuild spec https://review.opendev.org/739349 | 06:22 |
| *** yaawang has quit IRC | 06:26 | |
| *** yaawang has joined #openstack-nova | 06:27 | |
| *** bbowen has joined #openstack-nova | 06:29 | |
| *** dklyle has quit IRC | 06:35 | |
| *** ralonsoh has joined #openstack-nova | 06:48 | |
| *** takamatsu has quit IRC | 06:59 | |
| *** damien_r has joined #openstack-nova | 07:00 | |
| *** tesseract has joined #openstack-nova | 07:01 | |
| *** damien_r has quit IRC | 07:04 | |
| *** rcernin has quit IRC | 07:07 | |
| *** k_mouza has joined #openstack-nova | 07:09 | |
| *** jsuchome has joined #openstack-nova | 07:11 | |
| *** mkrai has quit IRC | 07:13 | |
| *** damien_r has joined #openstack-nova | 07:13 | |
| *** k_mouza has quit IRC | 07:13 | |
| *** ociuhandu has joined #openstack-nova | 07:17 | |
| *** happyhemant has joined #openstack-nova | 07:17 | |
| *** rcernin has joined #openstack-nova | 07:17 | |
| *** k_mouza has joined #openstack-nova | 07:18 | |
| *** k_mouza has quit IRC | 07:22 | |
| *** damien_r has quit IRC | 07:23 | |
| *** damien_r has joined #openstack-nova | 07:24 | |
| *** ociuhandu_ has joined #openstack-nova | 07:28 | |
| *** ociuhandu has quit IRC | 07:31 | |
| *** nightmare_unreal has joined #openstack-nova | 07:35 | |
| *** dtantsur|afk is now known as dtantsur | 07:45 | |
| *** tosky has joined #openstack-nova | 07:46 | |
| *** udesale has quit IRC | 07:47 | |
| *** udesale has joined #openstack-nova | 07:47 | |
| *** elod_off is now known as elod | 07:51 | |
| *** k_mouza has joined #openstack-nova | 07:54 | |
| *** maciejjozefczyk has joined #openstack-nova | 07:56 | |
| *** k_mouza has quit IRC | 07:59 | |
| *** k_mouza has joined #openstack-nova | 08:03 | |
| *** markvoelker has joined #openstack-nova | 08:12 | |
| *** avolkov has joined #openstack-nova | 08:12 | |
| *** markvoelker has quit IRC | 08:17 | |
| *** xek has joined #openstack-nova | 08:23 | |
| *** yaawang has quit IRC | 08:24 | |
| *** rcernin has quit IRC | 08:27 | |
| *** yaawang has joined #openstack-nova | 08:31 | |
| *** tetsuro has joined #openstack-nova | 08:31 | |
| *** ociuhandu_ has quit IRC | 08:33 | |
| *** ociuhandu has joined #openstack-nova | 08:33 | |
| *** tetsuro has quit IRC | 08:35 | |
| *** derekh has joined #openstack-nova | 08:36 | |
| *** bhagyashris is now known as bhagyashris|lunc | 08:43 | |
| *** mkrai has joined #openstack-nova | 08:45 | |
| *** ccamacho has joined #openstack-nova | 08:47 | |
| *** rcernin has joined #openstack-nova | 08:48 | |
| openstackgerrit | Huaqiang Wang proposed openstack/nova master: hardware: Enable 'hw:cpu_dedicated_mask' for creating a mixed instance https://review.opendev.org/714706 | 08:50 |
| *** mkrai has quit IRC | 09:03 | |
| *** mkrai has joined #openstack-nova | 09:03 | |
| *** alex_xu has joined #openstack-nova | 09:13 | |
| *** rcernin has quit IRC | 09:17 | |
| *** martinkennelly has joined #openstack-nova | 09:17 | |
| *** mkrai has quit IRC | 09:58 | |
| *** mkrai has joined #openstack-nova | 10:05 | |
| *** happyhemant has quit IRC | 10:07 | |
| *** k_mouza has quit IRC | 10:07 | |
| *** k_mouza has joined #openstack-nova | 10:10 | |
| *** dpaclt has joined #openstack-nova | 10:13 | |
| dpaclt | Hi All I am unable to launch new vms .. http://paste.openstack.org/show/795937/ ..Anyone please suggest | 10:13 |
| lyarwood | dpaclt: this isn't a support channel but grep for the req-4ce3ba68-34f8-453c-bb2a-9be9ef9adcbd id across the api/scheduler/conductor logs and you should see why all of your hosts are unable to schedule the instance | 10:15 |
| lyarwood | elod: https://review.opendev.org/#/c/732717/ (and the two under it) is finally ready btw if you have time today | 10:18 |
| dpaclt | Thanks lyarwood | 10:19 |
| *** damien_r has quit IRC | 10:19 | |
| *** Yumeng has quit IRC | 10:24 | |
| *** yaawang has quit IRC | 10:24 | |
| *** yaawang has joined #openstack-nova | 10:25 | |
| *** dpaclt has left #openstack-nova | 10:27 | |
| *** jraju__ has joined #openstack-nova | 10:29 | |
| *** links has quit IRC | 10:30 | |
| *** mvkr has quit IRC | 10:34 | |
| *** bhagyashris|lunc is now known as bhagyashris | 10:37 | |
| elod | lyarwood: sure, added to my TODOs :) | 10:44 |
| lyarwood | thanks! | 10:45 |
| openstackgerrit | Merged openstack/nova master: objects: Introduce 'pcpuset' field for InstanceNUMACell https://review.opendev.org/714658 | 10:49 |
| *** martinkennelly has quit IRC | 10:51 | |
| *** tosky has quit IRC | 11:05 | |
| *** martinkennelly has joined #openstack-nova | 11:07 | |
| *** tosky has joined #openstack-nova | 11:10 | |
| *** martinkennelly has quit IRC | 11:18 | |
| *** sapd__x has quit IRC | 11:22 | |
| *** ccamacho has quit IRC | 11:29 | |
| *** rcernin has joined #openstack-nova | 11:31 | |
| *** udesale_ has joined #openstack-nova | 11:31 | |
| *** udesale has quit IRC | 11:34 | |
| *** rcernin has quit IRC | 11:47 | |
| openstackgerrit | Elod Illes proposed openstack/nova stable/queens: Add functional test for bug 1550919 https://review.opendev.org/732715 | 12:01 |
| openstack | bug 1550919 in OpenStack Compute (nova) queens "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Lee Yarwood (lyarwood) | 12:01 |
| openstackgerrit | Elod Illes proposed openstack/nova stable/queens: libvirt: Fix misleading debug msg "Instance is running" https://review.opendev.org/732716 | 12:01 |
| *** mkrai has quit IRC | 12:03 | |
| *** damien_r has joined #openstack-nova | 12:04 | |
| *** mgariepy has joined #openstack-nova | 12:09 | |
| openstackgerrit | Elod Illes proposed openstack/nova stable/queens: libvirt: Don't delete disks on shared storage during evacuate https://review.opendev.org/732717 | 12:10 |
| jsuchome | gibi: hi, could you take another look at https://review.opendev.org/#/c/738738/ ? | 12:16 |
| *** tkajinam has quit IRC | 12:24 | |
| *** devfaz has joined #openstack-nova | 12:26 | |
| *** martinkennelly has joined #openstack-nova | 12:26 | |
| devfaz | Hi, is this a bug or known. If I try to disable an nova-compute service of a system terminated just seconds ago I get a trace with "An error occurred while updating the COMPUTE_STATUS_DISABLED trait on compute node resource providers managed by host XX. The trait will be synchronized automatically by the compute service when the update_available_resource periodic task runs" and rabbitmq-msgid timeout. | 12:27 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.PY2 and six.PY3 https://review.opendev.org/726512 | 12:27 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.add_metaclass https://review.opendev.org/726565 | 12:27 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.reraise https://review.opendev.org/726898 | 12:28 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.moves https://review.opendev.org/727224 | 12:28 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757 | 12:29 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777 | 12:29 |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.binary_type/integer_types/string_types https://review.opendev.org/728094 | 12:29 |
| *** vapjes has joined #openstack-nova | 12:35 | |
| *** ratailor has quit IRC | 12:38 | |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (1/2) https://review.opendev.org/728109 | 12:40 |
| *** martinkennelly has quit IRC | 12:53 | |
| *** ccamacho has joined #openstack-nova | 12:55 | |
| *** ccamacho has quit IRC | 12:56 | |
| *** ccamacho has joined #openstack-nova | 12:57 | |
| *** ccamacho has quit IRC | 12:58 | |
| openstackgerrit | Takashi Natsume proposed openstack/nova master: Remove six.text_type (2/2) https://review.opendev.org/728117 | 13:00 |
| lyarwood | dansmith: https://bugs.launchpad.net/nova/+bug/1858667 - I'm hitting this odd locking issue in some func tests, this smells like some behaviour has changed in py38 tbh but I'm having a hard time pinning it down. Would you be able to take a look? | 13:02 |
| openstack | Launchpad bug 1858667 in OpenStack Compute (nova) "TestMultiCellMigrate.test_poll_unconfirmed_resizes_with_(no_)upcall race failing with greenlet error: Error auto-confirming resize: Reader <greenlet.greenlet object at 0x7f55c19dd048> to writer privilege escalation not allowed. Will retry later." [Undecided,Confirmed] | 13:02 |
| *** brinzhang_ has quit IRC | 13:17 | |
| lyarwood | melwitt: ^ also if you have any time, I'm not even sure why we are wrapping target_cell if I'm honest | 13:21 |
| lyarwood | CELL_CACHE is the only global thing I can see? | 13:21 |
| dansmith | lyarwood: the entire DB state is global, which is what the lock is protecting | 13:30 |
| dansmith | lyarwood: your python REPL examples aren't representative I think because the lock grabbing is serial in the code, yet you're nesting them (I think, without indentation) | 13:31 |
| *** artom has joined #openstack-nova | 13:31 | |
| dansmith | I'm not sure what the references to fasteners has to do with this, as we're using regular python.threading locks, IIRC | 13:32 |
| dansmith | if I had to guess, I'd say maybe we're recursing on the lock somewhere, i.e. targeting the cell and then calling something that targets the cell again? | 13:35 |
| dansmith | ah lockutils uses fasteners, I see | 13:35 |
| lyarwood | dansmith: yeah sorry back | 13:35 |
| lyarwood | dansmith: and yeah I did use indentation, launchpad-- just removed it for some reason | 13:35 |
| lyarwood | dansmith: but yes my examples are bogus given what is actually being called | 13:36 |
| dansmith | ack | 13:36 |
| lyarwood | dansmith: I just hit the error and got carried away :) | 13:36 |
| dansmith | is that in a periodic? poll unconfirmed resizes? | 13:37 |
| dansmith | I may be making this up, but I think melwitt might have added something much later than this to make the periodics run targeted to the right cell, so maybe that's where the conflict is here... this is now getting lucky and running a periodic when we previously didn't need to change the target or something | 13:38 |
| lyarwood | yeah | 13:38 |
| *** Guest14648 has joined #openstack-nova | 13:43 | |
| *** Guest14648 is now known as redrobot | 13:45 | |
| *** ftarasenko has joined #openstack-nova | 13:46 | |
| masterpe | I try to live migrate a instance, but I get the message in the controller: "Unable to find record for source node compute40.domain.tld on compute48" | 13:48 |
| masterpe | but compute40 is incorrect | 13:48 |
| masterpe | What is wrong? | 13:49 |
| sean-k-mooney | that looks like maybe the instace.host is wrong | 13:56 |
| sean-k-mooney | which can happen if you have previous failed migrations in post live migrate | 13:56 |
| sean-k-mooney | but im not sure | 13:57 |
| sean-k-mooney | masterpe: i would check that the vm is actully on the host nova thinks its on | 13:58 |
| *** jangutter has quit IRC | 13:59 | |
| *** jangutter has joined #openstack-nova | 13:59 | |
| *** jangutter has quit IRC | 13:59 | |
| *** Liang__ has joined #openstack-nova | 13:59 | |
| *** Liang__ is now known as LiangFang | 13:59 | |
| *** jangutter has joined #openstack-nova | 14:00 | |
| masterpe | If I do a openstack server show instanceID: | 14:05 |
| masterpe | OS-EXT-SRV-ATTR:host | compute48 | 14:05 |
| masterpe | OS-EXT-SRV-ATTR:hypervisor_hostname | compute40.domain.tld | 14:05 |
| openstackgerrit | Huaqiang Wang proposed openstack/nova master: compute: bump nova-compute version and check in API https://review.opendev.org/740933 | 14:08 |
| openstackgerrit | Huaqiang Wang proposed openstack/nova master: hardware: create 'mixed' instance for realtime and shared CPUs https://review.opendev.org/728485 | 14:08 |
| *** huaqiang has joined #openstack-nova | 14:09 | |
| masterpe | sean-k-mooney: How to correct this? I have try to do a "update instances set node = 'compute48.domain.tld' where uuid = 'e848a42b-a7b2-4885-a290-34da7beb98ad';" | 14:10 |
| masterpe | But that did not solve it | 14:11 |
| masterpe | is is active on compute48 | 14:16 |
| sean-k-mooney | you should not change that | 14:17 |
| sean-k-mooney | actully that is strang | 14:17 |
| sean-k-mooney | what hypervior are you using | 14:18 |
| masterpe | qemu with kvm | 14:19 |
| sean-k-mooney | ok in that cas esthe host and hypervior host should start the same | 14:23 |
| sean-k-mooney | did you rename the host in the nova config? | 14:24 |
| sean-k-mooney | or chage its hostname at any point | 14:24 |
| masterpe | the hostname is not nova.conf | 14:24 |
| *** mriedem has joined #openstack-nova | 14:25 | |
| masterpe | no | 14:25 |
| sean-k-mooney | ok so you dont have host=compute48 https://github.com/openstack/nova/blob/master/nova/conf/netconf.py#L52-L56 | 14:26 |
| sean-k-mooney | what is the actul host name currently | 14:26 |
| sean-k-mooney | is it compute40 of compute 48 | 14:26 |
| *** efried has quit IRC | 14:31 | |
| *** mlavalle has joined #openstack-nova | 14:32 | |
| masterpe | command: hostnamectl status, gives Static hostname: compute48 | 14:33 |
| masterpe | and /etc/hosts has 127.0.1.1 compute48.domain.tld compute48 & 10.8.1.48 compute48.domain.tld compute48 | 14:35 |
| *** efried has joined #openstack-nova | 14:36 | |
| *** dklyle has joined #openstack-nova | 14:36 | |
| masterpe | nova-compute --version is on 19.1.1 | 14:38 |
| *** efried has quit IRC | 14:40 | |
| *** dave-mccowan has quit IRC | 14:43 | |
| masterpe | When I create a new instance on that compute all is fine | 14:46 |
| *** dave-mccowan has joined #openstack-nova | 14:47 | |
| lyarwood | dansmith: as you thought https://review.opendev.org/#/c/699291/7/nova/compute/api.py@3702 is the issue in that change, the failing tests are calling target_cell when running the _poll_unconfirmed_resizes periodic https://github.com/openstack/nova/blob/1fa6799e4171d7c93f4d6330f2437891733d7d27/nova/tests/functional/test_cross_cell_migrate.py#L988-L991 resulting in the reader to writer error. | 14:50 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: Handle Neutron errors in _post_live_migration() https://review.opendev.org/729763 | 14:51 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: Add regression test for bug 1879787 https://review.opendev.org/741230 | 14:51 |
| openstack | bug 1879787 in OpenStack Compute (nova) "post_live_migration does not handle Neutron errors" [Medium,In progress] https://launchpad.net/bugs/1879787 - Assigned to Artom Lifshitz (notartom) | 14:51 |
| *** efried has joined #openstack-nova | 14:51 | |
| dansmith | lyarwood: ah yep | 14:51 |
| lyarwood | dansmith: is there anyway of checking if a context is already correctly targetted at a cell? | 14:57 |
| lyarwood | actually nvm _get_source_compute_service doesn't have any other callers I can just mock it | 14:57 |
| dansmith | yes, but it would be better to arrange for that not to happen I think, because if that gets called from a compute node it won't be able to target because it won't have api/connection | 14:58 |
| *** mkrai has joined #openstack-nova | 15:09 | |
| *** LiangFang has quit IRC | 15:10 | |
| *** jraju__ has quit IRC | 15:12 | |
| *** bbowen has quit IRC | 15:15 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Ensure source compute is up when confirming a resize https://review.opendev.org/699291 | 15:16 |
| *** bbowen has joined #openstack-nova | 15:18 | |
| dansmith | lyarwood: you were only seeing the failure in your change there? | 15:20 |
| lyarwood | dansmith: I am, however Matt had already created this bug a while ago. | 15:22 |
| * lyarwood checks what he actually saw it against | 15:22 | |
| lyarwood | dansmith: ah it's the same change | 15:22 |
| dansmith | yeah, but presumably that bug was just for the periodic not working, right? | 15:22 |
| lyarwood | dansmith: so we can close this out | 15:22 |
| lyarwood | dansmith: it's the same test failure | 15:23 |
| dansmith | I'm foncused | 15:24 |
| dansmith | you filed a bug for the target recursion | 15:24 |
| melwitt | re: the earlier mention, I didn't add anything to target cell in periodics. I did briefly work on trying to get periodics to re-use a same RequestContext every time (so that request-id stays consistent in logs) but never finished it | 15:24 |
| melwitt | so that might be what you were thinking of | 15:25 |
| dansmith | melwitt: oh I thought you had done something specific to the targeting, although maybe that re-used context would have made them pre-targeted to the right cell? | 15:25 |
| dansmith | lyarwood: did you see that target recursion anywhere other than on matt's change to the periodic? | 15:26 |
| lyarwood | dansmith: mriedem created https://bugs.launchpad.net/nova/+bug/1858667 a while ago that I referenced in my first ping, I missed that it only documents this failure in this unmerged change. | 15:26 |
| openstack | Launchpad bug 1858667 in OpenStack Compute (nova) "TestMultiCellMigrate.test_poll_unconfirmed_resizes_with_(no_)upcall race failing with greenlet error: Error auto-confirming resize: Reader <greenlet.greenlet object at 0x7f55c19dd048> to writer privilege escalation not allowed. Will retry later." [Undecided,Confirmed] | 15:26 |
| dansmith | ah, heh okay gotcha | 15:26 |
| *** huaqiang has quit IRC | 15:26 | |
| dansmith | oh, I didn't realize that bug was old, I thought you had just filed it | 15:27 |
| melwitt | I don't think so, I don't remember doing anything related to periodics and targeting. and the re-using context thing I never finished and it never landed | 15:27 |
| dansmith | melwitt: okay | 15:27 |
| melwitt | what's interesting is that you'd think this would always happen... because generally nova-api will set_target_cell and then the CheatingSerializer will pass it on to nova-compute already targeted, and generally we have used 'with target_cell' to run periodics | 15:28 |
| *** huaqiang has joined #openstack-nova | 15:29 | |
| dansmith | we can't use target_cell for periodics at least in reality because we don't (always) have the api db creds to be able to look up a mapping | 15:29 |
| *** JamesBenson has joined #openstack-nova | 15:29 | |
| melwitt | right, I mean in func tests only | 15:29 |
| dansmith | or do you mean in tests where we poke them.. okay | 15:29 |
| dansmith | well, it only matters when the periodic makes a call to something that calls target a second time | 15:30 |
| dansmith | which should never happen, | 15:30 |
| melwitt | I'm just saying if it's a problem to 'with target_cell' on something that has already set_target_cell then lots of stuff should be failing all the time I'd think. surprising that it's only sometimes | 15:30 |
| dansmith | because periodics need to be calling compute-only code that assume the local cell | 15:30 |
| dansmith | it's not a problem with target_cell, it's a problem of calling code that calls target_cell from on the compute node I think | 15:31 |
| melwitt | oh, I see, periodic is calling something that does a target_cell? I wonder why it's even doing that | 15:31 |
| dansmith | well, | 15:31 |
| dansmith | it's only that one change, | 15:31 |
| dansmith | and it's for auto-confirming a cross-cell migration :) | 15:31 |
| *** hamalq has joined #openstack-nova | 15:31 | |
| melwitt | I see, re-using (calling) a utility function that's normally only called by non nova-compute I guess? | 15:32 |
| melwitt | I guess I could just look :P /me looks | 15:32 |
| dansmith | exactly | 15:33 |
| melwitt | huh, no it's not doing anything like that. mysterious ... | 15:33 |
| dansmith | it's adding code that calls target | 15:34 |
| *** hamalq_ has joined #openstack-nova | 15:34 | |
| melwitt | in nova/compute/api.py confirm_resize method | 15:34 |
| dansmith | right, which is called by the periodic right? | 15:35 |
| melwitt | must be... I guess I hadn't expected a nova-compute periodic calling something in nova.compute.API? but it must be I guess | 15:35 |
| dansmith | we do that in places | 15:36 |
| dansmith | like we call compute.api.stop_instance9) | 15:36 |
| dansmith | and such | 15:36 |
| melwitt | yeah, true | 15:36 |
| dansmith | we probably shouldn't, but we've done that for a long time since before it mattered | 15:36 |
| dansmith | ideally we'd have an inner method for "assume you're targeted or don't need to" that compute-specific code always calls, and then the compute "external" API just targets and then calls those things | 15:37 |
| melwitt | yup there it is https://github.com/openstack/nova/blob/1fa6799e4171d7c93f4d6330f2437891733d7d27/nova/compute/manager.py#L9093 | 15:37 |
| dansmith | unmysterious :) | 15:37 |
| melwitt | MYSTERY SOLVED | 15:37 |
| *** hamalq has quit IRC | 15:38 | |
| *** martinkennelly has joined #openstack-nova | 15:40 | |
| dansmith | sean-k-mooney: IIRC, we have some local hook mechanism we use for checking the devstack deployment, maybe after tempest runs? we used to (or still do) run db archiving I think.. is that still a thing? | 15:52 |
| dansmith | oh, is that post-run? | 15:54 |
| melwitt | https://github.com/openstack/nova/blob/master/gate/post_test_hook.sh ? | 15:54 |
| *** k_mouza has quit IRC | 15:55 | |
| melwitt | only run by certain jobs https://github.com/openstack/nova/search?q=post_test_hook&unscoped_q=post_test_hook | 15:56 |
| melwitt | nova-next, nova-live-migration, nova-grenade-multinode | 15:57 |
| dansmith | ah yeah, but what runs that? | 15:57 |
| dansmith | or is that baked into d-g? | 15:57 |
| melwitt | actually maybe it's only nova-next. I don't understand what the other two are doing | 15:58 |
| melwitt | it appears you run it by specifying this role in the post.yaml for the job https://github.com/openstack/nova/blob/89e65bd976be8e02fdb6932bb710b5165d7e00a3/playbooks/nova-next/post.yaml | 15:58 |
| melwitt | and the role is defined here https://github.com/openstack/nova/blob/89e65bd976be8e02fdb6932bb710b5165d7e00a3/roles/run-post-test-hook/tasks/main.yaml | 15:59 |
| dansmith | um, | 15:59 |
| dansmith | but something has to tell the nova-next job to run that post.yaml right? | 15:59 |
| melwitt | oh, yeah. that would be something in infra | 16:00 |
| melwitt | I don't know which thing | 16:00 |
| melwitt | maybe it is d-g | 16:00 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: compute: Validate a BDMs disk_bus when provided https://review.opendev.org/724842 | 16:01 |
| *** mkrai has quit IRC | 16:07 | |
| sean-k-mooney | dansmith: we used to in devstack gate yes | 16:08 |
| sean-k-mooney | a post job would be the zuuv3 way | 16:08 |
| sean-k-mooney | *playbook | 16:09 |
| sean-k-mooney | dansmith: this used to be defiend project-config | 16:11 |
| sean-k-mooney | which hook to run | 16:11 |
| sean-k-mooney | but i think it got moved in repo a long time ago | 16:11 |
| *** k_mouza has joined #openstack-nova | 16:11 | |
| sean-k-mooney | dansmith: melwitt we used to do something like this https://review.opendev.org/#/c/701051/1/playbooks/legacy/tempest-dsvm-networking-ovs-dpdk/run.yaml@a58 | 16:14 |
| sean-k-mooney | we defiend a pre_test_hook or post_test_hook function | 16:15 |
| sean-k-mooney | which was invoked by devstack gate if it existed | 16:15 |
| *** k_mouza has quit IRC | 16:16 | |
| sean-k-mooney | we still do that for the grenate multi node https://github.com/openstack/nova/blob/master/playbooks/legacy/nova-grenade-multinode/run.yaml#L57-L60 | 16:16 |
| *** maciejjozefczyk has quit IRC | 16:18 | |
| sean-k-mooney | i may have missed the point of your question | 16:19 |
| sean-k-mooney | dansmith: did any of ^ help | 16:19 |
| dansmith | sean-k-mooney: on a call, biab | 16:20 |
| sean-k-mooney | crap im ment to be on a call too i thnk | 16:20 |
| *** mgariepy has quit IRC | 16:22 | |
| *** k_mouza has joined #openstack-nova | 16:24 | |
| *** k_mouza has quit IRC | 16:24 | |
| *** k_mouza has joined #openstack-nova | 16:24 | |
| *** k_mouza has quit IRC | 16:29 | |
| openstackgerrit | Merged openstack/nova stable/ussuri: catch libvirt exception when nodedev not found. https://review.opendev.org/739593 | 16:34 |
| *** ociuhandu_ has joined #openstack-nova | 16:36 | |
| *** udesale_ has quit IRC | 16:38 | |
| *** ociuhandu has quit IRC | 16:39 | |
| *** ociuhandu_ has quit IRC | 16:40 | |
| *** psachin has quit IRC | 16:58 | |
| *** bbowen has quit IRC | 17:00 | |
| *** derekh has quit IRC | 17:00 | |
| *** mgariepy has joined #openstack-nova | 17:09 | |
| *** dtantsur is now known as dtantsur|afk | 17:18 | |
| openstackgerrit | Merged openstack/nova stable/ussuri: Correct reported system memory https://review.opendev.org/740666 | 17:24 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add emulated TPM support to Nova https://review.opendev.org/631363 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add docs for vTPM support https://review.opendev.org/739213 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Move single use constants to their callers https://review.opendev.org/741280 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Define constants in '_IntegratedTestBase' https://review.opendev.org/741281 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Remove 'test_servers.ServersTestBase' https://review.opendev.org/741282 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add 'PlacementHelperMixin', 'PlacementInstanceHelperMixin' https://review.opendev.org/741283 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Make '_IntegratedTestBase' subclass 'PlacementInstanceHelperMixin' https://review.opendev.org/741284 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add helpers for suspend, resume and reboot of server https://review.opendev.org/741285 | 17:29 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Pass context, instance to '_create_domain' https://review.opendev.org/741286 | 17:30 |
| *** gyee has joined #openstack-nova | 17:30 | |
| stephenfin | cleaning up after reverted resizes is hard work :( | 17:30 |
| gmann | dansmith: melwitt sean-k-mooney nova-next is zuulv3 native jobs and d-g not in pic. d-g run post script for legacy jobs only. zuulv3 run three phase in order 1. 'pre-run' (in hierarchical inheritance order, parent >child) 2. 'run' (this is not run in inheritance order means only parent run if child define one). and last is 3 'post-run' (in hierarchical inheritance order, parent >child) here it trigger the | 17:31 |
| gmann | playbooks/nova-next/post.yaml which run that script (https://github.com/openstack/nova/blob/89e65bd976be8e02fdb6932bb710b5165d7e00a3/.zuul.yaml#L199) | 17:31 |
| sean-k-mooney | gmann: yep i know | 17:32 |
| dansmith | gmann: ack, that's what I was suspecting, | 17:32 |
| dansmith | that we had to call the post playbook, I didn't even look | 17:32 |
| dansmith | I'm trying to get a glance job that will do some smoke testing of the devstack stuff, since baking devstack assumptions into tempest doesn't really matter | 17:32 |
| dansmith | er, doesn't really make sense | 17:32 |
| dansmith | and I just want to like validate that the cirros image got imported as expected, with auto-added metadata and stuff | 17:33 |
| dansmith | and that's kinda like what we do in that post job where we check the db state | 17:33 |
| gmann | dansmith: that is little tricky if you do in post pahse as parent post is run first where all logs etc are collected | 17:34 |
| sean-k-mooney | ah ok you want to just dump some extra info after the job run so we can validate the setup was correct | 17:35 |
| dansmith | sean-k-mooney: well, I want to validate it, not just dump it but yeah | 17:35 |
| sean-k-mooney | gmann: isint our post job run before our parent | 17:35 |
| dansmith | gmann: nova-next overrides post-run right? | 17:35 |
| sean-k-mooney | i though it was like an onion or stack call | 17:35 |
| dansmith | I don't care the order, as long as I can fail and cause the job to fail | 17:36 |
| sean-k-mooney | all our parent pre playbooks run firrst then ours then the un playbook then post run in reverse order | 17:36 |
| gmann | right child post-run playbook -> parent post-run playbook sean-k-mooney is right. | 17:36 |
| gmann | so we are good to run in nova-next post | 17:36 |
| sean-k-mooney | dansmith: the post playbook run regardless of if run passes or fails | 17:37 |
| dansmith | yeah, that's fine, I don't care the order | 17:37 |
| dansmith | sean-k-mooney: right, but if post-run fails, the job will fail right? | 17:37 |
| sean-k-mooney | but if a pre playbook fails wee dont run the run playbook | 17:37 |
| dansmith | like our db archive test | 17:37 |
| sean-k-mooney | oh am yes | 17:37 |
| gmann | yeas, it will cause job failure | 17:37 |
| dansmith | right, I just want the job to fail if I decide | 17:37 |
| sean-k-mooney | but you can ignore failure in the ansible playbook if you wan | 17:37 |
| sean-k-mooney | e.g. the ansible task can be marked to ignore any failures | 17:38 |
| dansmith | basically, I will configure glance to auto-add a piece of metadata to images that get added, and then in post, just check that the cirros image has that metadata set, and if not, explode | 17:38 |
| sean-k-mooney | so you can decied if a taks should mark the post playbook as a failure or not | 17:38 |
| sean-k-mooney | dansmith: yep that should be failrly simple | 17:38 |
| dansmith | yup | 17:38 |
| sean-k-mooney | you are already doing most of it with your multistore job | 17:39 |
| sean-k-mooney | in this case you need to enable the metadata plugin | 17:39 |
| sean-k-mooney | then have a post job that just checks for it | 17:39 |
| dansmith | yup | 17:39 |
| dansmith | but I'm going to do it for glance jobs, not mine | 17:39 |
| sean-k-mooney | sure | 17:39 |
| dansmith | just because they're not testing any of that stuff right now, | 17:39 |
| sean-k-mooney | it would be nice to do multi store for them too | 17:40 |
| dansmith | and it requires config, not something tempest can do, so I want to example them towards actually testing that stuff :) | 17:40 |
| dansmith | I think we're going to make the nova job global so they can run it too, right gmann ? | 17:40 |
| gmann | if you do it via role the nova or other job can reuse that | 17:40 |
| sean-k-mooney | dansmith: it already is | 17:40 |
| sean-k-mooney | they just need to add it to there check pipeline | 17:40 |
| sean-k-mooney | any repo can run any others jobs | 17:40 |
| gmann | dansmith: right, even on devstack-plugin-ceph gate | 17:40 |
| dansmith | sean-k-mooney: well, not yet because it's not landed | 17:40 |
| dansmith | but yeah | 17:41 |
| sean-k-mooney | dansmith: it can with depend on but ya | 17:41 |
| dansmith | well, anyway :) | 17:41 |
| sean-k-mooney | zuul is awsome that way | 17:41 |
| *** nightmare_unreal has quit IRC | 17:44 | |
| *** tesseract has quit IRC | 17:45 | |
| openstackgerrit | Merged openstack/nova stable/queens: Add functional test for bug 1550919 https://review.opendev.org/732715 | 17:59 |
| openstack | bug 1550919 in OpenStack Compute (nova) queens "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Elod Illes (elod-illes) | 17:59 |
| *** martinkennelly has quit IRC | 18:08 | |
| *** larsks has joined #openstack-nova | 18:14 | |
| *** bbowen has joined #openstack-nova | 18:16 | |
| *** admin0 has left #openstack-nova | 18:17 | |
| *** vishalmanchanda has quit IRC | 18:20 | |
| *** slaweq has joined #openstack-nova | 18:21 | |
| *** tosky has quit IRC | 18:25 | |
| *** tosky has joined #openstack-nova | 18:25 | |
| *** jsuchome has quit IRC | 18:39 | |
| *** ociuhandu has joined #openstack-nova | 18:46 | |
| *** ociuhandu has quit IRC | 18:50 | |
| *** jdillaman has joined #openstack-nova | 18:53 | |
| *** KeithMnemonic has joined #openstack-nova | 19:12 | |
| *** haleyb has joined #openstack-nova | 19:29 | |
| openstackgerrit | Merged openstack/nova stable/queens: libvirt: Fix misleading debug msg "Instance is running" https://review.opendev.org/732716 | 19:29 |
| *** ralonsoh has quit IRC | 19:49 | |
| artom | mordred, just wanted to bring your attention to https://review.opendev.org/#/c/741295/ and the corresponding story | 19:56 |
| * artom -> kids pick up | 19:57 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: Reject boot request for unsupported images https://review.opendev.org/708609 | 19:59 |
| *** raildo has quit IRC | 20:16 | |
| *** raildo has joined #openstack-nova | 20:16 | |
| *** raildo has quit IRC | 20:22 | |
| *** raildo has joined #openstack-nova | 20:24 | |
| *** slaweq has quit IRC | 20:29 | |
| *** raildo has quit IRC | 20:30 | |
| *** jangutter has quit IRC | 20:40 | |
| *** jangutter_ has joined #openstack-nova | 20:40 | |
| *** hamalq_ has quit IRC | 20:46 | |
| *** hamalq has joined #openstack-nova | 20:48 | |
| *** hamalq has quit IRC | 20:52 | |
| *** hamalq has joined #openstack-nova | 20:52 | |
| openstackgerrit | Sean McGinnis proposed openstack/nova master: Use unittest.mock instead of third party mock https://review.opendev.org/714676 | 20:52 |
| *** gyee has quit IRC | 21:10 | |
| *** gyee has joined #openstack-nova | 21:11 | |
| *** mriedem has left #openstack-nova | 21:46 | |
| *** xek has quit IRC | 21:50 | |
| mordred | artom: awesome - lgtm - thanks! | 21:59 |
| openstackgerrit | Merged openstack/nova master: virt: Add 'context', drop 'network_info' parameters for 'unrescue' https://review.opendev.org/730382 | 22:09 |
| *** logan- has quit IRC | 22:17 | |
| *** rcernin has joined #openstack-nova | 22:19 | |
| *** logan- has joined #openstack-nova | 22:19 | |
| *** rcernin has quit IRC | 22:20 | |
| *** rcernin has joined #openstack-nova | 22:21 | |
| *** prometheanfire has joined #openstack-nova | 22:26 | |
| prometheanfire | it looks like it's just nova that fails with mock-4.0.2 https://review.opendev.org/741091 | 22:27 |
| *** spatel has joined #openstack-nova | 22:29 | |
| *** vapjes has quit IRC | 22:33 | |
| *** spatel has quit IRC | 22:34 | |
| openstackgerrit | Merged openstack/nova master: crypto: Add type hints https://review.opendev.org/739207 | 22:35 |
| *** hamalq has quit IRC | 22:35 | |
| *** tosky has quit IRC | 22:48 | |
| melwitt | prometheanfire: will look | 22:52 |
| melwitt | I wrote that test 😝 | 22:53 |
| *** tkajinam has joined #openstack-nova | 22:55 | |
| *** tkajinam has quit IRC | 22:55 | |
| *** xiaolin has quit IRC | 22:56 | |
| prometheanfire | :D | 22:57 |
| *** tkajinam has joined #openstack-nova | 22:58 | |
| *** mlavalle has quit IRC | 22:59 | |
| *** avolkov has quit IRC | 23:02 | |
| *** markvoelker has joined #openstack-nova | 23:11 | |
| *** markvoelker has quit IRC | 23:15 | |
| *** JamesBenson has quit IRC | 23:56 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!