| *** dpawlik has quit IRC | 00:03 | |
| *** tbachman has joined #openstack-nova | 00:06 | |
| *** med_ has quit IRC | 00:17 | |
| openstackgerrit | Merged openstack/nova stable/pike: Filter out instances without a host when populating AZ https://review.openstack.org/594184 | 00:21 |
|---|---|---|
| openstackgerrit | Merged openstack/nova stable/ocata: Make ResourceTracker.stats node-specific https://review.openstack.org/588077 | 00:21 |
| *** med_ has joined #openstack-nova | 00:33 | |
| *** mriedem_away has quit IRC | 00:40 | |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Default zero disk flavor to RULE_ADMIN_API in Stein https://review.openstack.org/603910 | 00:41 |
| *** med_ has quit IRC | 00:48 | |
| *** erlon has quit IRC | 01:09 | |
| *** imacdonn has quit IRC | 01:15 | |
| *** imacdonn has joined #openstack-nova | 01:16 | |
| *** mhen has quit IRC | 01:17 | |
| *** mhen has joined #openstack-nova | 01:20 | |
| *** erlon has joined #openstack-nova | 01:21 | |
| *** mschuppert has quit IRC | 01:27 | |
| *** erlon has quit IRC | 01:27 | |
| *** mrsoul has quit IRC | 01:27 | |
| *** tiendc has joined #openstack-nova | 01:36 | |
| *** tiendc has quit IRC | 01:36 | |
| openstackgerrit | Tao Li proposed openstack/python-novaclient master: Remove the unused instance-name https://review.openstack.org/602520 | 01:40 |
| *** tetsuro has quit IRC | 01:43 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:50 | |
| *** hongbin has joined #openstack-nova | 01:58 | |
| *** efried1 has joined #openstack-nova | 02:02 | |
| *** efried has quit IRC | 02:04 | |
| *** efried1 is now known as efried | 02:04 | |
| gmann | mnaser: yeah, there can be more instance like that. as matt mentioned i can add this in my API improvement spec. exact number of fields always in response is better API even with empty value | 02:07 |
| *** dpawlik has joined #openstack-nova | 02:12 | |
| *** dpawlik has quit IRC | 02:17 | |
| *** Dinesh_Bhor has quit IRC | 02:26 | |
| *** Dinesh_Bhor has joined #openstack-nova | 02:38 | |
| openstackgerrit | huanhongda proposed openstack/nova master: Fix instance delete stuck in deleting task_state https://review.openstack.org/598084 | 02:41 |
| *** cfriesen has quit IRC | 02:47 | |
| *** hoonetorg has quit IRC | 02:48 | |
| *** Bhujay has joined #openstack-nova | 03:01 | |
| *** hoonetorg has joined #openstack-nova | 03:01 | |
| *** jiaopengju has quit IRC | 03:19 | |
| *** jiaopengju has joined #openstack-nova | 03:20 | |
| *** tiendc has joined #openstack-nova | 03:26 | |
| *** vivsoni has quit IRC | 03:26 | |
| *** vivsoni has joined #openstack-nova | 03:27 | |
| *** gcb_ has joined #openstack-nova | 03:27 | |
| *** sdake has quit IRC | 03:38 | |
| *** moshele has joined #openstack-nova | 03:42 | |
| *** sdake has joined #openstack-nova | 03:43 | |
| *** rcernin has quit IRC | 03:48 | |
| *** rcernin has joined #openstack-nova | 03:49 | |
| *** alaski has quit IRC | 03:56 | |
| *** edmondsw has quit IRC | 03:56 | |
| *** Bhujay has quit IRC | 03:58 | |
| *** moshele has quit IRC | 03:58 | |
| *** Dinesh_Bhor has quit IRC | 03:58 | |
| *** sambetts|afk has quit IRC | 04:08 | |
| *** sambetts_ has joined #openstack-nova | 04:10 | |
| *** brinzhang has quit IRC | 04:26 | |
| *** brinzhang has joined #openstack-nova | 04:26 | |
| *** udesale has joined #openstack-nova | 04:29 | |
| *** jhesketh_ has joined #openstack-nova | 04:32 | |
| *** vivsoni has quit IRC | 04:33 | |
| *** vivsoni has joined #openstack-nova | 04:33 | |
| *** Dinesh_Bhor has joined #openstack-nova | 04:34 | |
| *** d34dh0r53 has quit IRC | 04:34 | |
| *** d34dh0r53 has joined #openstack-nova | 04:34 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova stable/rocky: Imported Translations from Zanata https://review.openstack.org/604260 | 04:35 |
| *** jhesketh has quit IRC | 04:36 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/601047 | 04:42 |
| *** hongbin has quit IRC | 04:45 | |
| *** hshiina has joined #openstack-nova | 04:58 | |
| *** swamireddy has joined #openstack-nova | 05:08 | |
| *** swamireddy has quit IRC | 05:08 | |
| *** swamireddy has joined #openstack-nova | 05:08 | |
| *** ratailor has joined #openstack-nova | 05:11 | |
| *** Dinesh_Bhor has quit IRC | 05:12 | |
| *** Dinesh_Bhor has joined #openstack-nova | 05:16 | |
| *** Bhujay has joined #openstack-nova | 05:18 | |
| *** hoangcx has quit IRC | 05:21 | |
| *** hoangcx has joined #openstack-nova | 05:21 | |
| *** jhesketh_ is now known as jhesketh | 05:25 | |
| *** hoangcx has quit IRC | 05:32 | |
| *** tiendc has quit IRC | 05:32 | |
| *** hoangcx has joined #openstack-nova | 05:32 | |
| *** sridharg has joined #openstack-nova | 05:47 | |
| *** icey has quit IRC | 05:52 | |
| *** icey has joined #openstack-nova | 05:52 | |
| openstackgerrit | Naichuan Sun proposed openstack/nova master: os-xenapi(n-rp)(WIP): add traits for vgpu n-rp https://review.openstack.org/604269 | 06:03 |
| *** janki has joined #openstack-nova | 06:04 | |
| *** mschuppert has joined #openstack-nova | 06:08 | |
| *** dpawlik has joined #openstack-nova | 06:11 | |
| openstackgerrit | Tao Li proposed openstack/nova master: Rollback instance vm_state to original where instance claims failed https://review.openstack.org/592252 | 06:17 |
| *** slaweq has joined #openstack-nova | 06:35 | |
| *** litao__ has joined #openstack-nova | 06:36 | |
| *** litao__ has quit IRC | 06:38 | |
| *** litao_ has joined #openstack-nova | 06:39 | |
| openstackgerrit | YI-JIE,SYU proposed openstack/nova master: Because nova-consoleauth had deprecated since version 18.0.0, fix verify operation documentation file context. https://review.openstack.org/604277 | 06:42 |
| *** belmoreira has joined #openstack-nova | 06:43 | |
| *** sambetts_ has quit IRC | 06:53 | |
| *** sambetts_ has joined #openstack-nova | 06:55 | |
| *** rcernin has quit IRC | 07:02 | |
| *** tssurya has joined #openstack-nova | 07:04 | |
| gibi | efried: thanks for the fup on the cleanup I've +A-d it | 07:05 |
| *** sahid has joined #openstack-nova | 07:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova stable/rocky: Imported Translations from Zanata https://review.openstack.org/604260 | 07:13 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/601047 | 07:23 |
| *** ejat has joined #openstack-nova | 07:24 | |
| *** holser_ has joined #openstack-nova | 07:34 | |
| *** jpena|off is now known as jpena | 07:48 | |
| *** ralonsoh has joined #openstack-nova | 07:50 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add API ref guideline for examples https://review.openstack.org/604060 | 07:53 |
| *** janki has quit IRC | 07:53 | |
| *** hshiina has quit IRC | 08:00 | |
| *** helenafm has joined #openstack-nova | 08:04 | |
| *** dims has quit IRC | 08:11 | |
| *** dims has joined #openstack-nova | 08:12 | |
| *** tssurya has quit IRC | 08:16 | |
| *** gcb_ has quit IRC | 08:18 | |
| *** alexchadin has joined #openstack-nova | 08:22 | |
| *** alexchadin has quit IRC | 08:26 | |
| *** alexchadin has joined #openstack-nova | 08:28 | |
| *** s10 has joined #openstack-nova | 08:33 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612 | 08:35 |
| openstackgerrit | Vlad Gusev proposed openstack/nova stable/queens: libvirt: Use os.stat and os.path.getsize for RAW disk inspection https://review.openstack.org/604295 | 08:37 |
| *** belmoreira has quit IRC | 08:37 | |
| *** Dinesh_Bhor has quit IRC | 08:37 | |
| *** derekh has joined #openstack-nova | 08:39 | |
| *** belmoreira has joined #openstack-nova | 08:46 | |
| *** Dinesh_Bhor has joined #openstack-nova | 08:55 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Specifies the storage backend to boot instance https://review.openstack.org/579360 | 08:57 |
| *** sapd1 has quit IRC | 09:00 | |
| *** sapd1_ has joined #openstack-nova | 09:01 | |
| *** dpawlik has quit IRC | 09:02 | |
| *** mdbooth has joined #openstack-nova | 09:02 | |
| *** dpawlik has joined #openstack-nova | 09:04 | |
| *** hoonetorg has quit IRC | 09:08 | |
| *** jaosorior is now known as jaosorior_lunch | 09:09 | |
| *** dillaman has joined #openstack-nova | 09:09 | |
| *** jdillaman has quit IRC | 09:11 | |
| *** tssurya has joined #openstack-nova | 09:15 | |
| *** Dinesh_Bhor has quit IRC | 09:16 | |
| openstackgerrit | Merged openstack/nova stable/rocky: Fix image-defined numa claims during evacuate https://review.openstack.org/599062 | 09:19 |
| openstackgerrit | Merged openstack/nova stable/rocky: Configure placement DB context manager for nova-manage/status https://review.openstack.org/600464 | 09:20 |
| openstackgerrit | Merged openstack/nova stable/rocky: Document unset/reset wrinkle for *_allocation_ratio options https://review.openstack.org/599673 | 09:20 |
| openstackgerrit | Naichuan Sun proposed openstack/nova master: os-xenapi(n-rp)(WIP): add traits for vgpu n-rp https://review.openstack.org/604269 | 09:22 |
| openstackgerrit | Naichuan Sun proposed openstack/nova master: os-xenapi(n-rp)(WIP): add traits for vgpu n-rp https://review.openstack.org/604269 | 09:23 |
| *** Luzi has joined #openstack-nova | 09:23 | |
| *** hoonetorg has joined #openstack-nova | 09:25 | |
| *** vabada has joined #openstack-nova | 09:26 | |
| *** sambetts_ is now known as sambetts | 09:31 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612 | 09:33 |
| *** takashin has left #openstack-nova | 09:34 | |
| *** mdbooth has quit IRC | 09:40 | |
| *** mdbooth has joined #openstack-nova | 09:40 | |
| openstackgerrit | Naichuan Sun proposed openstack/nova master: os-xenapi(n-rp)(WIP): add traits for vgpu n-rp https://review.openstack.org/604269 | 09:42 |
| openstackgerrit | Merged openstack/os-traits master: clean up CUDA traits https://review.openstack.org/597170 | 09:46 |
| *** hoangcx has quit IRC | 09:48 | |
| *** alexchadin has quit IRC | 09:51 | |
| openstackgerrit | Naichuan Sun proposed openstack/nova master: os-xenapi(n-rp): add traits for vgpu n-rp https://review.openstack.org/604269 | 09:52 |
| *** tssurya has quit IRC | 09:54 | |
| *** alexchadin has joined #openstack-nova | 09:56 | |
| *** Dinesh_Bhor has joined #openstack-nova | 09:56 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612 | 10:13 |
| openstackgerrit | YI-JIE,SYU proposed openstack/nova master: Remove deprecated nova-consoleauth reference from doc https://review.openstack.org/604277 | 10:14 |
| *** cdent has joined #openstack-nova | 10:17 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove deprecated nova-consoleauth reference from doc https://review.openstack.org/604277 | 10:18 |
| *** vivsoni has quit IRC | 10:19 | |
| *** Dinesh_Bhor has quit IRC | 10:20 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612 | 10:26 |
| *** eandersson has quit IRC | 10:30 | |
| *** jiteka has quit IRC | 10:30 | |
| *** alexchadin has quit IRC | 10:35 | |
| *** alexchadin has joined #openstack-nova | 10:36 | |
| *** alexchadin has quit IRC | 10:36 | |
| *** alexchadin has joined #openstack-nova | 10:36 | |
| *** alexchadin has quit IRC | 10:37 | |
| *** alexchadin has joined #openstack-nova | 10:37 | |
| *** alexchadin has quit IRC | 10:37 | |
| *** alexchadin has joined #openstack-nova | 10:38 | |
| *** alexchadin has quit IRC | 10:38 | |
| *** tbachman has quit IRC | 10:44 | |
| *** erlon has joined #openstack-nova | 10:47 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Add regression for bug 1787606 https://review.openstack.org/593073 | 10:52 |
| openstack | bug 1787606 in OpenStack Compute (nova) "Multi instance creation rescheduling fails due to a lack of alternates" [Medium,In progress] https://launchpad.net/bugs/1787606 - Assigned to Lee Yarwood (lyarwood) | 10:52 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: scheduler: Increase alternate count in smaller environments https://review.openstack.org/593074 | 10:52 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: fixtures: Track volume attachments within CinderFixtureNewAttachFlow https://review.openstack.org/587013 | 10:52 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Add regression test for bug#1784353 https://review.openstack.org/587014 | 10:52 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: conductor: Recreate volume attachments during a reschedule https://review.openstack.org/587071 | 10:52 |
| s10 | Please review backports for the https://bugs.launchpad.net/nova/+bug/1785827 | 10:55 |
| openstack | Launchpad bug 1785827 in OpenStack Compute (nova) "Performance regression in libvirt get_available_resource()" [Undecided,Fix released] - Assigned to Lee Yarwood (lyarwood) | 10:55 |
| s10 | https://review.openstack.org/#/q/topic:bug/1785827+(status:open+OR+status:merged) | 10:55 |
| *** dpawlik has quit IRC | 11:02 | |
| *** dpawlik has joined #openstack-nova | 11:03 | |
| *** udesale has quit IRC | 11:05 | |
| *** helenafm has quit IRC | 11:08 | |
| *** cdent has quit IRC | 11:12 | |
| *** udesale has joined #openstack-nova | 11:13 | |
| *** alexchadin has joined #openstack-nova | 11:15 | |
| *** alexchadin has quit IRC | 11:20 | |
| *** jpena is now known as jpena|lunch | 11:23 | |
| *** alex_xu has quit IRC | 11:23 | |
| *** mschuppert is now known as mschuppert|lunch | 11:24 | |
| *** janki has joined #openstack-nova | 11:28 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Specifies the storage backend to boot instance https://review.openstack.org/579360 | 11:32 |
| *** litao_ has quit IRC | 11:33 | |
| *** udesale has quit IRC | 11:34 | |
| *** udesale has joined #openstack-nova | 11:34 | |
| *** alexchadin has joined #openstack-nova | 11:37 | |
| s10 | Please reopen bug https://bugs.launchpad.net/nova/+bug/1209101 , it was reintroduced in https://github.com/openstack/nova/commit/2a81337a#diff-46d9a1430ccd8406e9dae8eee9e1f699R81 | 11:38 |
| openstack | Launchpad bug 1209101 in OpenStack Compute (nova) "Non-public flavor cannot be used in created tenant" [High,Fix released] - Assigned to Sumanth Nagadavalli (sumanth-nagadavalli) | 11:38 |
| *** brinzhang has quit IRC | 11:43 | |
| *** cdent has joined #openstack-nova | 11:44 | |
| openstackgerrit | Merged openstack/nova master: Resource retrieving: add changes-before filter https://review.openstack.org/599276 | 11:48 |
| *** mschuppert|lunch is now known as mschuppert | 12:00 | |
| openstackgerrit | Merged openstack/nova stable/pike: Add unshelve instance error info to fault table https://review.openstack.org/583426 | 12:01 |
| openstackgerrit | Merged openstack/nova stable/rocky: Delete instance_group_member records from API DB during archive https://review.openstack.org/598953 | 12:01 |
| openstackgerrit | Merged openstack/nova stable/queens: Remove noisy DEBUG log https://review.openstack.org/600372 | 12:01 |
| openstackgerrit | Merged openstack/nova master: Optimize AZ lookup during schedule_and_build_instances https://review.openstack.org/588665 | 12:10 |
| *** tssurya has joined #openstack-nova | 12:10 | |
| *** jaosorior_lunch is now known as jaosorior | 12:11 | |
| *** tbachman has joined #openstack-nova | 12:15 | |
| *** gibi is now known as giblet | 12:17 | |
| giblet | stephenfin: rubasov published the blogpost about the bandwidth demo https://rubasov.github.io/2018/09/21/openstack-qos-min-bw-demo.html | 12:17 |
| *** tssurya has quit IRC | 12:21 | |
| *** jpena|lunch is now known as jpena | 12:26 | |
| *** udesale has quit IRC | 12:26 | |
| *** tbachman has quit IRC | 12:32 | |
| cdent | +many on that blog post | 12:33 |
| *** ratailor has quit IRC | 12:34 | |
| *** alexchadin has quit IRC | 12:43 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Consumer gen support for delete instance allocations https://review.openstack.org/591597 | 12:43 |
| *** panda|off is now known as panda | 12:53 | |
| openstackgerrit | Merged openstack/nova master: libvirt: Drop MIN_LIBVIRT_PF_WITH_NO_VFS_CAP_VERSION https://review.openstack.org/565517 | 12:59 |
| openstackgerrit | Merged openstack/nova master: Filter deleted computes from get_all_by_uuids() https://review.openstack.org/604108 | 12:59 |
| openstackgerrit | Merged openstack/nova master: Delete instance_id_mappings record in instance_destroy https://review.openstack.org/591558 | 12:59 |
| *** tbachman has joined #openstack-nova | 13:00 | |
| openstackgerrit | Merged openstack/nova master: Remove an unnecessary comment https://review.openstack.org/603924 | 13:00 |
| openstackgerrit | Merged openstack/nova master: Imported Translations from Zanata https://review.openstack.org/601047 | 13:00 |
| *** cdent has quit IRC | 13:02 | |
| *** liuyulong has joined #openstack-nova | 13:02 | |
| *** jaypipes-ooo has quit IRC | 13:25 | |
| *** jaypipes has joined #openstack-nova | 13:25 | |
| mdbooth | Oh, fun. LVM encryption is useless :) | 13:27 |
| * johnthetubaguy is very curious now | 13:27 | |
| mdbooth | johnthetubaguy: Just writing up a detailed response to that bug | 13:28 |
| lyarwood | which bug? | 13:28 |
| mdbooth | https://bugs.launchpad.net/nova/+bug/1793159 | 13:28 |
| openstack | Launchpad bug 1793159 in OpenStack Compute (nova) "no signature check for cached images" [Undecided,New] | 13:28 |
| *** jaypipes is now known as leakypipes | 13:28 | |
| *** mriedem has joined #openstack-nova | 13:29 | |
| *** cdent has joined #openstack-nova | 13:30 | |
| mdbooth | johnthetubaguy: A para from the wip response: | 13:30 |
| mdbooth | Unfortunately, 'encrypted' LVM as currently implemented is similarly vulnerable. When starting the instance we first created an LV for the disk which will store encrypted data. However, we then create a dm-crypt device which we initialise with the key we obtained from Barbican. This presents an unencrypted block device to the host, which we then present to the instance. Any attacker needs only use the dm-crypt device rather than the | 13:30 |
| mdbooth | underlying device, both of which have the same access. The dm-crypt device is only removed if the instance is deleted, or implicitly if the compute host is rebooted. | 13:30 |
| *** jistr is now known as jistr|call | 13:31 | |
| mdbooth | lyarwood: Native ephemeral encryption would improve ^^^ | 13:31 |
| johnthetubaguy | well, I still think once you have root on the hypervisor its game over, any which way | 13:32 |
| *** s10 has quit IRC | 13:32 | |
| mdbooth | johnthetubaguy: Right. I'm addressing that, too. | 13:32 |
| * mdbooth is probably spending too much time on this | 13:33 | |
| johnthetubaguy | I think they were thinking about an external storage system being compromised, and providing some protection against that | 13:33 |
| *** belmoreira has quit IRC | 13:33 | |
| mriedem | dansmith: i'll update the stable release patches once maya gets off to school. everything is merged that was approved except for https://review.openstack.org/#/c/592310/ but that's super latent anyway so i won't hold for it. | 13:33 |
| *** sridharg has quit IRC | 13:33 | |
| dansmith | sweet | 13:33 |
| lyarwood | mdbooth: yeah on my list for T overall, might try to get to rbd in S | 13:34 |
| mdbooth | johnthetubaguy lyarwood: https://bugs.launchpad.net/nova/+bug/1793159 | 13:40 |
| openstack | Launchpad bug 1793159 in OpenStack Compute (nova) "no signature check for cached images" [Undecided,New] | 13:40 |
| mdbooth | Commented | 13:40 |
| *** burt has joined #openstack-nova | 13:40 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Consumer gen support for put allocations https://review.openstack.org/591647 | 13:46 |
| mdbooth | johnthetubaguy: So the point about being hosed if you have root on the hypervisor is obviously valid, but there's real-world practical gain to be had by requiring the admin to reconfigure the system first, as you have a chance to put additional controls around that. It'll also stop pretty much all 'innocent curiosity'. | 13:47 |
| openstackgerrit | Mohammed Naser proposed openstack/nova stable/rocky: Filter deleted computes from get_all_by_uuids() https://review.openstack.org/604367 | 13:47 |
| *** stephenfin is now known as finucannot | 13:48 | |
| finucannot | giblet: Sweet, cheers | 13:48 |
| *** efried is now known as fried_rice | 13:49 | |
| *** dansmith is now known as SteelyDan | 13:50 | |
| mdbooth | SteelyDan: I think DonSmith might be more appropriate :D | 13:50 |
| SteelyDan | eh? | 13:50 |
| SteelyDan | Who is Don Smith? | 13:50 |
| mdbooth | A mafioso, obviously | 13:51 |
| openstackgerrit | Merged openstack/nova master: Making instance/migration listing skipping down cells configurable https://review.openstack.org/592428 | 13:53 |
| mriedem | nova stable releases https://review.openstack.org/#/q/topic:nova-stable-releases-sept-2018+(status:open+OR+status:merged) | 13:54 |
| SteelyDan | mriedem: thanks for doing that | 13:55 |
| lyarwood | about to jump on a call but I can take a look afterwards, thanks mriedem | 13:55 |
| SteelyDan | mriedem: do we bug dims and smcginnis to look at those? | 13:58 |
| openstackgerrit | Merged openstack/nova master: Add get_by_cell_and_project() method to InstanceMappingList https://review.openstack.org/591656 | 13:58 |
| openstackgerrit | Merged openstack/nova master: Fix missing specifying doctrees directory https://review.openstack.org/604068 | 13:58 |
| smcginnis | I looked at a couple. I can keep going if it helps. | 13:58 |
| openstackgerrit | Merged openstack/nova master: Remove mox in test_compute_api.py (4) https://review.openstack.org/568462 | 13:58 |
| openstackgerrit | Merged openstack/nova master: Remove deprecated hide_server_address_states option https://review.openstack.org/603831 | 13:58 |
| openstackgerrit | Merged openstack/nova master: Remove mox in libvirt/test_driver.py (6) https://review.openstack.org/571330 | 13:59 |
| SteelyDan | smcginnis: we haven't had releases in a while and there are like a hundred important pending fixes | 14:01 |
| SteelyDan | so yeah it would be good if you can | 14:01 |
| smcginnis | SteelyDan: Cool, I can spend a little time this morning going through there then. | 14:02 |
| SteelyDan | thanks | 14:02 |
| smcginnis | No problem | 14:02 |
| *** openstackgerrit has quit IRC | 14:07 | |
| *** mchlumsky has quit IRC | 14:09 | |
| mriedem | SteelyDan: i'm confused by tssurya's change here https://review.openstack.org/#/c/567785/ which looks like it adds the new microversion handling, but the microversion isn't actually introduced in that change, it's spread throughout several other patches after that | 14:10 |
| mriedem | is the idea that none of this works until the end of the series? | 14:11 |
| SteelyDan | yeah, that's generally how we do this right? | 14:11 |
| mriedem | well, | 14:12 |
| mriedem | we generally plumb the lower layers with flags and such, | 14:12 |
| mriedem | but that change is actually checking the version the user passed in is 2.66 and if so, does something | 14:12 |
| mriedem | i just don't know if that would actually work yet until the MAX_VERSION is updated later | 14:13 |
| mriedem | the risk is that 2.66 is already approved in another change | 14:13 |
| mriedem | iow, normally the change that introduces the actual microversion is at the end | 14:13 |
| mriedem | i think i'm going to procedurally -2 this until the rest of the series is +W | 14:13 |
| SteelyDan | right, that's what I asked for earlier .. is that no tthis? | 14:14 |
| * SteelyDan is in a meeting and distracted | 14:14 | |
| mriedem | i left comments and a -2, can discuss with tssurya later | 14:16 |
| *** cfriesen has joined #openstack-nova | 14:16 | |
| *** openstackgerrit has joined #openstack-nova | 14:18 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Delete instance_id_mappings record in instance_destroy https://review.openstack.org/604373 | 14:18 |
| *** mchlumsky has joined #openstack-nova | 14:18 | |
| mriedem | jroll: we should probably make this VirtDriverNotReady thing for ironic a warning yeah? http://logs.openstack.org/27/602127/2/check/ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa/4238d0f/controller/logs/screen-n-cpu.txt.gz?level=TRACE#_Sep_20_21_52_03_587436 | 14:20 |
| mriedem | or info? | 14:20 |
| mriedem | it's just a case of n-cpu starting up before ironic api right? | 14:21 |
| mriedem | and it's self-healing? | 14:21 |
| mriedem | gmann: i'm going to pull https://blueprints.launchpad.net/nova/+spec/api-extensions-merge-stein out of the runway slot since there are no open changes | 14:23 |
| *** lbragstad is now known as elbragstad | 14:24 | |
| *** munimeha1 has joined #openstack-nova | 14:28 | |
| *** alex_xu has joined #openstack-nova | 14:30 | |
| *** janki has quit IRC | 14:33 | |
| *** dpawlik has quit IRC | 14:33 | |
| *** maciejjozefczyk has quit IRC | 14:34 | |
| openstackgerrit | Merged openstack/nova-specs master: Placement: any traits in allocation_candidate query https://review.openstack.org/565730 | 14:36 |
| *** Bhujay has quit IRC | 14:38 | |
| openstackgerrit | Merged openstack/nova-specs master: Placement: support mixing required traits with any traits https://review.openstack.org/565741 | 14:38 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add abstract OVSDB API https://review.openstack.org/476612 | 14:39 |
| *** mchlumsky has quit IRC | 14:40 | |
| *** mrjk has quit IRC | 14:40 | |
| *** mchlumsky has joined #openstack-nova | 14:41 | |
| *** mrjk has joined #openstack-nova | 14:41 | |
| *** Bhujay has joined #openstack-nova | 14:41 | |
| *** pmannidi has joined #openstack-nova | 14:42 | |
| *** Luzi has quit IRC | 14:42 | |
| cdent | How disabled is a compute node that's been administratively disabled (compute service disable...)? Can an admin stil force a migration there? | 14:47 |
| cdent | SteelyDan, mriedem ^ ? | 14:47 |
| mriedem | i think they can | 14:47 |
| mriedem | b/c a force would bypass the ComputeFilter | 14:47 |
| *** mlavalle has joined #openstack-nova | 14:48 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Ignore VirtDriverNotReady in _sync_power_states periodic task https://review.openstack.org/604376 | 14:49 |
| mriedem | jroll: ^ | 14:49 |
| cdent | mriedem: is my understanding correct that there are two different kinds of force? one checks that the destination has the required resourcdes (and thus uses the sheduler/placement) and the other does not? | 14:49 |
| mriedem | well, | 14:49 |
| mriedem | there are a few ways to confuse 'force' here, | 14:50 |
| mriedem | depends on the microversion used in the live migration api | 14:50 |
| mriedem | https://developer.openstack.org/api-ref/compute/#live-migrate-server-os-migratelive-action | 14:50 |
| cdent | anytime someone says "well" i want to run and hide | 14:50 |
| mriedem | so before microversion 2.30, specifying the host bypasses the scheduler and forces it, | 14:50 |
| mriedem | after microversion 2.30, if you specify a host but not force=true, the scheduler validates the host, | 14:51 |
| mriedem | >=2.30 + host + force=true means bypass the scheduler | 14:51 |
| mriedem | yes, it's terrible; mordred can attest when i explained this when he fixed it in the sdk | 14:51 |
| mriedem | and it's also the reason i was -5 on adding force to cold migration | 14:51 |
| mriedem | among other reasons | 14:51 |
| * cdent attempts to digest this information | 14:52 | |
| mriedem | if it doesn't matter, always pass host=None | 14:52 |
| mriedem | so the scheduler always picks | 14:52 |
| mriedem | otheriwse use microversion >=2.30 so the scheduler validates the specified host | 14:52 |
| mordred | yeah. I really didn't enjoy this one | 14:53 |
| cdent | so: in >= 2.30 if i want to target a disabled compute node I can host + force = true and really truly force. That's the thing I'm after in this case. | 14:53 |
| mriedem | yes i think so | 14:53 |
| mriedem | there are big red warnings in the api ref about it too | 14:53 |
| cdent | cool, thank you very much. I think I've just learned a lot in a very short space of time, which is pleasing. | 14:54 |
| mordred | cdent: http://git.openstack.org/cgit/openstack/openstacksdk/tree/openstack/compute/v2/server.py#n366 if you want to see it all in python | 14:54 |
| * mriedem grabs dull knife | 14:54 | |
| * cdent calls ... someone | 14:55 | |
| mriedem | ghostbusters? | 14:55 |
| mriedem | i guess that would actually be bad for their business | 14:55 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Optimize AZ lookup during schedule_and_build_instances https://review.openstack.org/604378 | 14:57 |
| mriedem | SteelyDan: lyarwood: when you get a chance, could use reviews on these simple backports to run tempest-slow in queens and pike https://review.openstack.org/#/q/topic:nova-slow+(status:open+OR+status:merged) | 14:58 |
| mriedem | b/c we merged a change in tempest to move several tests from tempest-full to tempest-slow so we should make sure we still have the test coverage on stable | 14:59 |
| *** swamireddy has quit IRC | 15:01 | |
| *** dpawlik has joined #openstack-nova | 15:03 | |
| lyarwood | mriedem: ack looking now | 15:04 |
| mriedem | alex_xu: have you talked with the cyborg devs at all about your nvdimm stuff to see if that could work with cyborg as a generic way to model those devices and integrate with nova for the plug/unplug that's needed via the os-acc library? | 15:07 |
| *** dpawlik has quit IRC | 15:07 | |
| mriedem | alex_xu: in general, i think we could really use someone that knows how nova works helping the cyborg team directly; fried_rice has been doing that but is also really busy with other stuff too. | 15:08 |
| lyarwood | mriedem: remind me, nova-next is the cellsv2 job right? | 15:09 |
| mriedem | lyarwood: in the olden times nova-next was cells v2 and placement while those were optional (in newton yeah) | 15:09 |
| SteelyDan | all jobs should be cellsv2 with superconductor these days no? | 15:09 |
| mriedem | yes | 15:09 |
| lyarwood | yeah I just couldn't recall what we used it for now | 15:09 |
| mriedem | that's why we don't run nova-next on ocata/pike | 15:09 |
| mriedem | it's also the only job that runs our post-test scripts | 15:10 |
| mriedem | the job description in .zuul.yaml explains what it runs | 15:10 |
| mriedem | it runs the tls console proxy code since queens | 15:10 |
| mriedem | and the service user token stuff since pike | 15:10 |
| mriedem | so it's a smorgasbord of optional nova things not tested elsewhere | 15:11 |
| lyarwood | cool, thanks | 15:11 |
| *** jistr|call is now known as jistr | 15:11 | |
| openstackgerrit | Michal Arbet proposed openstack/nova master: Add check if api_database.connection is set. https://review.openstack.org/604388 | 15:20 |
| *** rpittau has quit IRC | 15:21 | |
| *** bnemec is now known as beekneemech | 15:21 | |
| openstackgerrit | Michal Arbet proposed openstack/nova master: Add check if api_database.connection is set. https://review.openstack.org/604388 | 15:25 |
| *** sapd1 has joined #openstack-nova | 15:27 | |
| mriedem | giblet: i have a few questions in https://review.openstack.org/#/c/591597/ | 15:27 |
| mriedem | i think it's OK for the most part, except we could be leaking a 500 response in the DELETE /servers/{server_id} API | 15:28 |
| cdent | fried_rice, giblet : besides this topic https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/use-nested-allocation-candidates is there other "make nested work" stuff in progress? | 15:29 |
| fried_rice | heh, probably. | 15:30 |
| fried_rice | I assume you mean besides the stuff that consumes it (e.g. vgpu) | 15:30 |
| cdent | (yes to second question) I figured, and I'll endeavor to find it, but thought I'd check for a quickref first | 15:31 |
| alex_xu | mriedem: I'm thinking nova first since I think it is memory device. but actually, I never thing about how to integrate with cyborg | 15:31 |
| alex_xu | mriedem: I didn't contribute too much for cyborg spec, but Sundar used to catch me discuss something | 15:32 |
| *** tbachman has quit IRC | 15:35 | |
| mriedem | if it could work with cyborg, then it would be a win-win if you worked on it that way since they need someone that understands nova to help them integrate i think | 15:35 |
| mriedem | obviously not on your friday night | 15:35 |
| alex_xu | mriedem: got it, i will take a look at that | 15:37 |
| *** med_ has joined #openstack-nova | 15:40 | |
| *** pmannidi has quit IRC | 15:41 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add volume-backed evacuate test https://review.openstack.org/604397 | 15:46 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add native implementation OVSDB API https://review.openstack.org/482226 | 15:47 |
| *** Bhujay has quit IRC | 15:48 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Run evacuate tests with local/lvm and shared/rbd storage https://review.openstack.org/604400 | 15:50 |
| mriedem | mdbooth: ^ those two additional changes should give a pretty healthy evacuate integration test coverage | 15:50 |
| *** med_ has quit IRC | 15:50 | |
| mriedem | local/lvm and shared/rbd | 15:50 |
| *** sapd1 has quit IRC | 15:51 | |
| mriedem | image-backed and volume-backed | 15:51 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add volume-backed evacuate test https://review.openstack.org/604397 | 15:54 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Run evacuate tests with local/lvm and shared/rbd storage https://review.openstack.org/604400 | 15:54 |
| *** cdent has quit IRC | 15:59 | |
| *** tbachman has joined #openstack-nova | 15:59 | |
| mdbooth | mriedem: Looking now, thanks | 16:02 |
| openstackgerrit | Michal Arbet proposed openstack/nova master: Add check if api_database.connection is set. https://review.openstack.org/604388 | 16:02 |
| sean-k-mooney | mriedem: hi i meant to ask you at the ptg but can you take a look at https://review.openstack.org/#/c/587615/ and comment if you want us to squash this and the next change into one or if this is ok. melwitt and finucannot wanted your input before merging for backport reasons | 16:02 |
| *** sahid has quit IRC | 16:03 | |
| *** med_ has joined #openstack-nova | 16:03 | |
| mriedem | sean-k-mooney: without looking at details i'd squash them, | 16:05 |
| mriedem | we normally only do the 2-change dance for a regression test when it's a much larger functional regression test that is actually hitting the api and such and shows a regression | 16:05 |
| mriedem | otherwise if it's just unit tests, removing the code fix should show the test fails | 16:05 |
| *** ShilpaSD has quit IRC | 16:14 | |
| *** ralonsoh has quit IRC | 16:18 | |
| *** holser_ has quit IRC | 16:18 | |
| *** fried_rice is now known as fried_rolls | 16:18 | |
| sean-k-mooney | mriedem: ya that was thw main question. i know the two step thing is normally only dont for functional test. ill squash them | 16:19 |
| sean-k-mooney | mriedem: it will make finucannot happy anyway | 16:19 |
| mdbooth | mriedem: How about testing NFS/LVM ;) https://bugs.launchpad.net/nova/+bug/1793159/comments/9 | 16:22 |
| openstack | Launchpad bug 1793159 in OpenStack Compute (nova) "no signature check for cached images" [Undecided,New] | 16:22 |
| mriedem | lvm will be tested by the last change in that series now, | 16:22 |
| mriedem | or do you mean lvm as the image backend, not volume-based? | 16:22 |
| mriedem | we have an lvm job in the experimental queue i think but it's not multinode | 16:23 |
| mriedem | same for nfs | 16:23 |
| mdbooth | mriedem: I wasn't being serious, but I was suggesting instances on NFS and image backend=lvm | 16:23 |
| mriedem | well, we could do those if we made the jobs multinode | 16:23 |
| mdbooth | As that was the scenario described in that bug | 16:23 |
| * mdbooth had never previously considered it | 16:23 | |
| mriedem | i seem to remember having a todo to investigate integrating lvm into some other job of ours for additional coverage, related to something lyarwood was fixing awhile back | 16:24 |
| mriedem | i'd have to dig that up | 16:24 |
| *** janki has joined #openstack-nova | 16:25 | |
| mdbooth | mriedem: Incidentally, afaik there's absolutely no reason to ever use that configuration. | 16:25 |
| mriedem | imagebackend=lvm? | 16:26 |
| mriedem | windriver loved it until recently | 16:26 |
| mdbooth | mriedem: Combined with instances on nfs | 16:26 |
| mriedem | oh | 16:26 |
| mdbooth | It seems like the worst of everything | 16:26 |
| mdbooth | Yeah, lvm is a thing. I wonder what the performance advantage over raw files is, though. Bet it's minimal. | 16:28 |
| mriedem | i guess i was thinking of this https://review.openstack.org/#/c/567860/ maybe | 16:28 |
| mriedem | config drive with vfat | 16:28 |
| mriedem | because of https://bugs.launchpad.net/nova/+bug/1771700 | 16:29 |
| openstack | Launchpad bug 1771700 in nova (Ubuntu Bionic) "nova-lvm tempest job failing with InvalidDiskInfo" [High,Fix committed] | 16:29 |
| mriedem | http://paste.openstack.org/show/730550/ | 16:30 |
| mriedem | tl;dr: "so we could get images_type live migration coverage for qcow2, raw and rbd image types, with and without config drive, and with config drive vfat and iso9660." | 16:30 |
| mriedem | right now our live migration jobs only test qcow2/rbd and without config drive ever | 16:31 |
| mdbooth | It's quite a matrix :/ | 16:32 |
| mdbooth | mriedem: Add in with/without kernel/ramdisk | 16:32 |
| mriedem | no one cares about those | 16:32 |
| mdbooth | Hehe, they were a bug in my evacuate patch, though :) | 16:32 |
| mdbooth | The fake imageservice was non-deterministically returning an image with kernel/ramdisk, which caused a failure in my functional test. The bug was real. | 16:34 |
| mdbooth | mriedem: How much of test_evacuate.sh is boilerplate, btw? | 16:35 |
| mriedem | as in virt agnostic? | 16:35 |
| mdbooth | As in copied almost verbatim from another well-tested script | 16:35 |
| mriedem | 0 | 16:36 |
| mdbooth | k | 16:36 |
| mriedem | i wrote it while at the ptg last week | 16:36 |
| mriedem | in between bouts of yelling "argh" at people | 16:36 |
| mriedem | you were my muse you know | 16:36 |
| mdbooth | You must have typed quickly in those gaps | 16:36 |
| mriedem | heh | 16:36 |
| mriedem | btw, assuming these evacuate test patches pass, we should run your fix on top of them, | 16:37 |
| mriedem | could probably do that without rebasing | 16:37 |
| mriedem | if you put a change on top of your fix that depends-on my stack | 16:37 |
| mdbooth | mriedem: ack. I'll add a Depends-on to it now | 16:37 |
| mriedem | don't put the depends-on your fix directly, | 16:37 |
| mriedem | put it on a DNM patch on top of yours | 16:37 |
| mdbooth | Oh, ok. Got it. | 16:37 |
| mdbooth | I'll have to leave the detailed review for Monday now, though. I need to be somewhere in 50 minutes, and I need to both get home, eat, and travel there first. | 16:38 |
| mriedem | yup, sure | 16:38 |
| mriedem | ttyl | 16:38 |
| mriedem | SteelyDan: stacking up a new devstack atm to test your placement db copy script thing, should hopefully be able to start writing grenade changes this afternoon | 16:39 |
| SteelyDan | mriedem: ack.. did the projects thing for the grenade job get resolved? | 16:40 |
| SteelyDan | I haven't really been paying attention | 16:40 |
| mriedem | don't know | 16:40 |
| *** dtantsur is now known as dtantsur|afk | 16:41 | |
| *** derekh has quit IRC | 16:48 | |
| *** jpena is now known as jpena|off | 16:54 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Add regression test for bug 1550919 https://review.openstack.org/591733 | 16:55 |
| openstack | bug 1550919 in OpenStack Compute (nova) "[Libvirt]Evacuate fail may cause disk image be deleted" [Medium,In progress] https://launchpad.net/bugs/1550919 - Assigned to Matthew Booth (mbooth-9) | 16:55 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Don't delete disks on shared storage during evacuate https://review.openstack.org/578846 | 16:55 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: DNM: Test against mdbooth's evacuate patch https://review.openstack.org/604423 | 16:55 |
| *** sambetts is now known as sambetts|afk | 16:56 | |
| *** gyee has joined #openstack-nova | 17:01 | |
| *** mdbooth has quit IRC | 17:02 | |
| kashyap | mriedem: Shall I bump the MIN_VERSION_* to the advertized NEXT_MIN_* here: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L230 | 17:03 |
| *** nicolasbock_ has joined #openstack-nova | 17:03 | |
| kashyap | (Will get to it on Monday; but just wanted to double-check here.) | 17:03 |
| *** spsurya has quit IRC | 17:05 | |
| openstackgerrit | Merged openstack/nova stable/queens: Add tempest-slow job to run the tempest slow tests https://review.openstack.org/604134 | 17:06 |
| *** maciejjozefczyk has joined #openstack-nova | 17:12 | |
| *** med_ has quit IRC | 17:12 | |
| *** maciejjozefczyk has quit IRC | 17:14 | |
| *** maciejjozefczyk has joined #openstack-nova | 17:14 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (7) https://review.openstack.org/571992 | 17:17 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (8) https://review.openstack.org/571993 | 17:17 |
| *** med_ has joined #openstack-nova | 17:18 | |
| *** maciejjozefczyk has quit IRC | 17:19 | |
| openstackgerrit | Merged openstack/nova stable/pike: Add tempest-slow job to run the tempest slow tests https://review.openstack.org/604138 | 17:21 |
| openstackgerrit | Merged openstack/nova master: Remove deprecated nova-consoleauth reference from doc https://review.openstack.org/604277 | 17:21 |
| openstackgerrit | Merged openstack/nova master: ironic: stop hammering ironic API in power sync loop https://review.openstack.org/602127 | 17:24 |
| kashyap | mriedem: When you're around, I should we should update it. Since we advertized them as the NEXT_MIN. | 17:25 |
| kashyap | (That means, some more tedious, but necessary work, about selecting the NEXT_MIN_* for 'T' release.) | 17:26 |
| * kashyap --> home | 17:26 | |
| *** manjeets has joined #openstack-nova | 17:26 | |
| openstackgerrit | sean mooney proposed openstack/nova master: libvirt: fix disk_bus handling for root disk https://review.openstack.org/584999 | 17:31 |
| *** etp has quit IRC | 17:41 | |
| *** etp has joined #openstack-nova | 17:44 | |
| *** med_ has quit IRC | 17:54 | |
| *** etp has quit IRC | 17:55 | |
| *** etp has joined #openstack-nova | 17:56 | |
| *** janki has quit IRC | 17:57 | |
| openstackgerrit | Merged openstack/os-vif master: Fix upper-constraints link in tox file https://review.openstack.org/604158 | 17:59 |
| *** panda has quit IRC | 18:08 | |
| *** panda has joined #openstack-nova | 18:10 | |
| *** nicolasbock_ has quit IRC | 18:12 | |
| openstackgerrit | Mohammed Naser proposed openstack/nova stable/queens: Filter deleted computes from get_all_by_uuids() https://review.openstack.org/604448 | 18:47 |
| openstackgerrit | Mohammed Naser proposed openstack/nova stable/pike: Filter deleted computes from get_all_by_uuids() https://review.openstack.org/604449 | 18:48 |
| openstackgerrit | Mohammed Naser proposed openstack/nova stable/ocata: Filter deleted computes from get_all_by_uuids() https://review.openstack.org/604451 | 18:52 |
| *** dave-mccowan has quit IRC | 19:03 | |
| *** dave-mccowan has joined #openstack-nova | 19:14 | |
| *** erlon has quit IRC | 19:25 | |
| *** maciejjozefczyk has joined #openstack-nova | 19:28 | |
| *** dpawlik has joined #openstack-nova | 19:33 | |
| cfriesen | is there an API equivalent to "nova-manage cell_v2 discover_hosts"? | 19:37 |
| *** dpawlik has quit IRC | 19:38 | |
| mriedem | cfriesen: no | 19:38 |
| mriedem | there is a config option to run it periodically in the scheduler | 19:39 |
| *** med_ has joined #openstack-nova | 19:45 | |
| *** slaweq has quit IRC | 19:54 | |
| *** slaweq has joined #openstack-nova | 19:57 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add volume-backed evacuate test https://review.openstack.org/604397 | 20:00 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Run evacuate tests with local/lvm and shared/rbd storage https://review.openstack.org/604400 | 20:00 |
| *** mdbooth has joined #openstack-nova | 20:10 | |
| imacdonn | mriedem: can I ask a "dumb" question or two about upgrade checks? | 20:18 |
| mriedem | yes | 20:18 |
| imacdonn | so ... my understanding of "nova-status upgrade check" is that it's something that would be run *after* upgrading (db sync, etc), to check that everything is OK to proceed | 20:19 |
| imacdonn | now it sounds like you want to do checks for things that should be done BEFORE the upgrade in there | 20:19 |
| imacdonn | which is it? | 20:19 |
| mriedem | it's meant to be run before starting new code | 20:22 |
| mriedem | but that's not really a hard and fast rule | 20:22 |
| imacdonn | but ... before or after upgrading the database ? | 20:22 |
| *** mdbooth has quit IRC | 20:22 | |
| mriedem | i.e. you can also run the nova-status upgrade check as a post-install step for a fresh install | 20:22 |
| imacdonn | (databases) | 20:22 |
| mriedem | after | 20:22 |
| imacdonn | OK, so then ... it seems like it's not the right place for pre-upgrade checks | 20:23 |
| *** med_ has quit IRC | 20:23 | |
| imacdonn | there could be things you'd want to check before you try to upgrade the databases | 20:23 |
| mriedem | well, depends on what your db upgrades are doing | 20:23 |
| mriedem | are they doing data migrations during db sync? | 20:23 |
| mriedem | b/c they really shouldn't | 20:24 |
| imacdonn | hmm, I guess I don't know enough about the db internals ... I kinda thought that that's what db sync does | 20:24 |
| imacdonn | if not where, where should the migrations happen? on first startup after upgrade ? | 20:25 |
| mriedem | nova's db sync routines which lay down schema are primarily only for additive schema changes, like adding tables, columns and indexes/constraints | 20:25 |
| mriedem | not for things that involve data migrations | 20:25 |
| mriedem | during runtime | 20:25 |
| mriedem | e.g. if you need to change the format for some value stored in the db, check on read and update on write | 20:26 |
| mriedem | nova has moved several things from the 'cell' db to the api db, and during that process our routine is to read from the api db first, if not there, read from the cell db, and then migrate | 20:26 |
| mriedem | we also have commands for performing those data migration in batches | 20:26 |
| mriedem | nova-manage db online_data_migrations | 20:26 |
| imacdonn | hmmm ... that's probably documented somewhere, but I haven't run across it yet ... is there a good description of the right sequence of events for upgrades somewhere? | 20:28 |
| mriedem | have you read through https://docs.openstack.org/nova/latest/reference/upgrade-checks.html ? | 20:28 |
| imacdonn | don't think I'd seen that one ... will study it | 20:29 |
| mriedem | so having said that, this is how nova does upgrades, | 20:30 |
| imacdonn | but back to the checks ... I think it may be a good idea, especially if you're trying to make this a generic framework, so have places for pre- and post-upgrade checking | 20:30 |
| mriedem | which isn't going to be how everyone does it | 20:30 |
| mriedem | well, pre and post upgrade are pretty fuzzy, | 20:30 |
| mriedem | the general rule i try to follow is how idempotent can i write the check such that anyone can run it at anytime | 20:30 |
| mriedem | be that before the db sync with new code, after that but before new code starts running, after new code is running, etc | 20:31 |
| mriedem | i know that doesn't help you much here... | 20:31 |
| mriedem | it makes more sense when you have something you actually need to write a check for, | 20:31 |
| mriedem | e.g. if your release notes say, "make sure you do x before upgrading to stein" then that's a pretty obvious thing to check (if you can) | 20:32 |
| imacdonn | I can sortof see that ... but it seems that what constitutes "OK to start the upgrade" vs. "the upgrade did everything it was supposed to, and you can start services now" may be different | 20:32 |
| imacdonn | Looking at the output of 'nova-status upgrade check', I see one of the items is "Request Spec Migration" | 20:38 |
| imacdonn | haven't looked to see exactly what that is, but if it's a migration, and migrations don't happen during sync, and you're supposed to do the upgrade check before running the new code .............. | 20:38 |
| mriedem | so the story on that one is, | 20:39 |
| mriedem | nova added this thing called a request spec in newton, | 20:39 |
| mriedem | and has compatibility code all over the place for instances created before newton that don't have a related request spec record in the db, | 20:40 |
| mriedem | we have an online data migration that operators can run to create records for old instances, | 20:40 |
| mriedem | the upgrade check was added because i want to drop that compatibility code in stein, | 20:40 |
| mriedem | which means, before you get to stein and restart services with stein code (which could have dropped that compatibility), you can run that check and see, oh i better finish my homework - it's been due for 2 years | 20:41 |
| imacdonn | so ... if we're running this check after doing the db sync with the stein code, but before we can start up the services with the new code, can we still do the online migration ? | 20:42 |
| mriedem | so the way upgrades work in grenade for nova is: lay down new code, sync the db schema for the new models, run the online data migrations, restart the services | 20:43 |
| mriedem | *with nova-status upgrade check in between those last 2 | 20:43 |
| mriedem | sure you can | 20:43 |
| mriedem | the online data migrations CLI in nova just needs access to the config and db | 20:43 |
| mriedem | none of that goes over rpc or rest api | 20:44 |
| *** munimeha1 has quit IRC | 20:44 | |
| mriedem | otherwise you can't FFU | 20:44 |
| imacdonn | so "online" means that it's OK to do online, not that it has to be done online? or am I completely misunderstanding what "online" means ? | 20:45 |
| mriedem | the 'online_data_migrations' name might be a bad name | 20:46 |
| mriedem | it just means, | 20:47 |
| mriedem | not offline like during a db sync when the controller is down | 20:47 |
| mriedem | like how data migrations were done during db sync in the long ago | 20:47 |
| mriedem | which meant operators could have substantial downtime in the control plane | 20:47 |
| imacdonn | ok | 20:48 |
| mriedem | which project are you looking to do the upgrade checks for btw? | 20:49 |
| imacdonn | so I need to digest the whole upgrade flow a bit more (maybe after I digest lunch) .... but I still think it's worth at least considering pre/post-upgrade checks in general | 20:49 |
| imacdonn | I'm trying to be a good citizen and 'do the right thing' for all projects .. or all of the ones that I uses, which is mostly the core ones | 20:50 |
| imacdonn | based on your ML posts a while back, I added 'nova-status upgrade check', but it's not quite right, as I'm running it after starting the services | 20:51 |
| imacdonn | maybe mostly because currently placement is considered a nova service, at least in my deployment mechanism | 20:52 |
| imacdonn | I may try hacking in sometihng to start placement individually before doing the upgrade check, then start the rest of the (nova) services | 20:53 |
| imacdonn | or maybe I'll worry about that for stein | 20:53 |
| mriedem | "I added 'nova-status upgrade check', but it's not quite right, as I'm running it after starting the services" - as i said, that's also acceptable in the case of nova and it's even in our install docs: https://docs.openstack.org/nova/latest/install/verify.html | 20:54 |
| mriedem | for a fresh install i mean | 20:54 |
| mriedem | flexibility and idempotency are kind of the key things here | 20:55 |
| imacdonn | Well, the description does say "Performs a release-specific readiness check before restarting services with new code" | 20:55 |
| imacdonn | https://docs.openstack.org/nova/latest/cli/nova-status.html | 20:56 |
| imacdonn | but yeah ... I was basically using it as a validation that "everything went OK with the nova upgrade" | 20:56 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: DNM: Test against mdbooth's evacuate patch https://review.openstack.org/604423 | 20:56 |
| imacdonn | re fresh install vs upgrade, I basically perform the same steps in both cases (that idempotence thing) | 20:58 |
| mriedem | yeah some of the wording is a bit dated and was written when the initial change in ocata was really for before the new code was started up | 21:01 |
| mriedem | but that's still how we do it in grenade | 21:02 |
| imacdonn | wondering if there might be a case where you really do want to check for something before starting the new code, because otherwise the new code might try to do something that would turn out to be catastrophic | 21:03 |
| mriedem | well, yes, you want to run it after starting new code | 21:04 |
| mriedem | guh, sorry, typo | 21:05 |
| mriedem | before starting new code | 21:05 |
| mriedem | that's the idea | 21:05 |
| mriedem | put it in a venv or container and run it | 21:05 |
| *** sambetts|afk has quit IRC | 21:12 | |
| *** sambetts_ has joined #openstack-nova | 21:15 | |
| *** mchlumsky has quit IRC | 21:30 | |
| *** med_ has joined #openstack-nova | 21:31 | |
| *** ivve has quit IRC | 21:38 | |
| *** med_ has quit IRC | 21:44 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add some more docs for upgrade checkers https://review.openstack.org/604486 | 21:47 |
| *** dave-mccowan has quit IRC | 22:04 | |
| *** mriedem is now known as mriedem_away | 22:09 | |
| imacdonn | mriedem_away: On a slightly related note; is there a reason that this do like .where(compute_nodes.c.deleted==0) ? https://github.com/openstack/nova/blob/master/nova/cmd/status.py#L111-L130 | 22:26 |
| imacdonn | erm, I mean; ... that this doesn't* do like ..... | 22:29 |
| imacdonn | see also https://bugs.launchpad.net/nova/+bug/1757207 | 22:32 |
| openstack | Launchpad bug 1757207 in OpenStack Compute (nova) "compute resource providers not equal to compute nodes in deployment" [Undecided,Incomplete] | 22:32 |
| *** tbachman has quit IRC | 22:55 | |
| *** mrjk has quit IRC | 22:56 | |
| *** mrjk has joined #openstack-nova | 22:57 | |
| *** janki has joined #openstack-nova | 23:13 | |
| *** janki has quit IRC | 23:28 | |
| *** tbachman has joined #openstack-nova | 23:29 | |
| *** maciejjozefczyk has quit IRC | 23:31 | |
| *** dpawlik has joined #openstack-nova | 23:34 | |
| *** tbachman has quit IRC | 23:34 | |
| *** mschuppert has quit IRC | 23:35 | |
| *** tbachman has joined #openstack-nova | 23:37 | |
| *** dpawlik has quit IRC | 23:38 | |
| *** medberry has joined #openstack-nova | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!