| *** gmann_afk is now known as gmann | 00:01 | |
| *** threestrands has joined #openstack-nova | 00:01 | |
| *** hemna_ has quit IRC | 00:02 | |
| *** yamamoto has quit IRC | 00:03 | |
| *** itlinux has joined #openstack-nova | 00:03 | |
| gmann | mriedem: dansmith please check these stable branch patches for zuulv3 in novaclient - https://review.openstack.org/#/q/I43a8435485751748ca6228f67d401945cb32652e | 00:04 |
|---|---|---|
| gmann | project config patch is waiting for those to merge - https://review.openstack.org/#/c/522101/ | 00:04 |
| *** liverpooler has joined #openstack-nova | 00:04 | |
| *** hongbin has quit IRC | 00:08 | |
| openstackgerrit | Armando Migliaccio proposed openstack/nova stable/pike: VMware: Factor out relocate_vm() https://review.openstack.org/526549 | 00:08 |
| *** liverpooler has quit IRC | 00:09 | |
| *** chyka has quit IRC | 00:11 | |
| *** baoli has quit IRC | 00:13 | |
| *** slaweq has joined #openstack-nova | 00:22 | |
| *** rcernin has quit IRC | 00:25 | |
| *** rcernin has joined #openstack-nova | 00:25 | |
| *** gouthamr has quit IRC | 00:25 | |
| *** hshiina has joined #openstack-nova | 00:26 | |
| *** slaweq has quit IRC | 00:26 | |
| *** sdague has quit IRC | 00:29 | |
| *** mlavalle has quit IRC | 00:36 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in delete (3) https://review.openstack.org/526557 | 00:36 |
| openstackgerrit | Sam Morrison proposed openstack/nova master: Allow ability for non admin users to use all filters on server list. https://review.openstack.org/526558 | 00:41 |
| openstackgerrit | zhangyanxian proposed openstack/nova-specs master: Sriov-bond https://review.openstack.org/463526 | 00:46 |
| *** slaweq has joined #openstack-nova | 00:57 | |
| *** huanxie has joined #openstack-nova | 00:59 | |
| *** yamamoto has joined #openstack-nova | 00:59 | |
| *** gyee has quit IRC | 01:01 | |
| *** slaweq has quit IRC | 01:02 | |
| *** phuongnh has joined #openstack-nova | 01:04 | |
| *** yamamoto has quit IRC | 01:04 | |
| *** jichen has joined #openstack-nova | 01:08 | |
| *** yangyapeng has joined #openstack-nova | 01:09 | |
| *** esberglu has joined #openstack-nova | 01:16 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add aggregates check in allocation candidates https://review.openstack.org/522407 | 01:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add functional test for get_all_with_shared() https://review.openstack.org/522408 | 01:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Fix missing rps in allocation candidates https://review.openstack.org/522409 | 01:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: placement: func tests for multiple shared RPs https://review.openstack.org/498737 | 01:17 |
| *** david-lyle has joined #openstack-nova | 01:18 | |
| *** esberglu has quit IRC | 01:21 | |
| *** mikal has quit IRC | 01:24 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:28 | |
| *** mikal has joined #openstack-nova | 01:30 | |
| *** slaweq has joined #openstack-nova | 01:32 | |
| *** Apoorva_ has joined #openstack-nova | 01:33 | |
| *** fragatina has quit IRC | 01:33 | |
| *** Apoorva has quit IRC | 01:36 | |
| *** david-lyle has quit IRC | 01:36 | |
| *** slaweq has quit IRC | 01:37 | |
| *** Apoorva_ has quit IRC | 01:38 | |
| *** fragatina has joined #openstack-nova | 01:46 | |
| *** fragatina has quit IRC | 01:51 | |
| openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add instance action db and obj pagination support. https://review.openstack.org/526422 | 01:52 |
| *** yamamoto has joined #openstack-nova | 02:00 | |
| *** harlowja has quit IRC | 02:03 | |
| *** yamamoto has quit IRC | 02:04 | |
| *** hongbin has joined #openstack-nova | 02:05 | |
| *** slaweq has joined #openstack-nova | 02:12 | |
| *** slaweq has quit IRC | 02:17 | |
| openstackgerrit | wanghongxu proposed openstack/nova master: Update cinder info when attach volume that attached https://review.openstack.org/525540 | 02:17 |
| *** annp has joined #openstack-nova | 02:26 | |
| openstackgerrit | Jiao Pengju proposed openstack/python-novaclient master: Fix listing available images example error in nova.rst https://review.openstack.org/526569 | 02:27 |
| *** dgonzalez_ has joined #openstack-nova | 02:27 | |
| openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add pagination and changes-since for instance-actions https://review.openstack.org/326326 | 02:28 |
| *** dgonzalez_ has quit IRC | 02:29 | |
| *** namnh has joined #openstack-nova | 02:38 | |
| *** zhurong has joined #openstack-nova | 02:47 | |
| *** slaweq has joined #openstack-nova | 02:48 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Document Queens review priorities https://review.openstack.org/526573 | 02:49 |
| *** slaweq has quit IRC | 02:52 | |
| *** fragatina has joined #openstack-nova | 03:01 | |
| *** yamamoto has joined #openstack-nova | 03:01 | |
| *** yamamoto has quit IRC | 03:06 | |
| *** bbbbzhao_ has joined #openstack-nova | 03:14 | |
| *** slaweq has joined #openstack-nova | 03:19 | |
| *** fragatina has quit IRC | 03:19 | |
| *** tidwellr has joined #openstack-nova | 03:19 | |
| *** abhishekk has joined #openstack-nova | 03:24 | |
| *** slaweq has quit IRC | 03:24 | |
| *** slaweq has joined #openstack-nova | 03:25 | |
| *** Dinesh_Bhor has quit IRC | 03:25 | |
| *** dave-mccowan has quit IRC | 03:26 | |
| *** slaweq has quit IRC | 03:30 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: doc: link in some Sydney summit content https://review.openstack.org/526587 | 03:30 |
| *** Apoorva has joined #openstack-nova | 03:33 | |
| *** mriedem has quit IRC | 03:36 | |
| *** links has joined #openstack-nova | 03:42 | |
| *** coreywright has quit IRC | 03:53 | |
| *** udesale has joined #openstack-nova | 03:54 | |
| *** slaweq has joined #openstack-nova | 03:57 | |
| *** BryanS68 has joined #openstack-nova | 04:01 | |
| *** slaweq has quit IRC | 04:02 | |
| *** yamamoto has joined #openstack-nova | 04:03 | |
| *** Dinesh_Bhor has joined #openstack-nova | 04:06 | |
| *** yamamoto has quit IRC | 04:06 | |
| *** armax has quit IRC | 04:06 | |
| *** yamamoto has joined #openstack-nova | 04:06 | |
| *** armax has joined #openstack-nova | 04:07 | |
| *** armax has quit IRC | 04:07 | |
| *** coreywright has joined #openstack-nova | 04:10 | |
| *** trungnv has quit IRC | 04:10 | |
| *** TuanLA has joined #openstack-nova | 04:13 | |
| *** chyka has joined #openstack-nova | 04:13 | |
| openstackgerrit | Chen Hanxiao proposed openstack/nova master: trivial: more suitable log in set_admin_password https://review.openstack.org/526592 | 04:13 |
| *** gbarros has quit IRC | 04:17 | |
| *** chyka has quit IRC | 04:19 | |
| *** Dinesh_Bhor has quit IRC | 04:19 | |
| *** Dinesh_Bhor has joined #openstack-nova | 04:19 | |
| *** andreas_s has joined #openstack-nova | 04:22 | |
| *** zhurong has quit IRC | 04:26 | |
| *** andreas_s has quit IRC | 04:27 | |
| *** huanxie has quit IRC | 04:32 | |
| *** gbarros has joined #openstack-nova | 04:32 | |
| *** adisky_ has joined #openstack-nova | 04:34 | |
| *** slaweq has joined #openstack-nova | 04:36 | |
| *** gbarros has quit IRC | 04:39 | |
| *** slaweq has quit IRC | 04:41 | |
| *** hieulq has quit IRC | 04:43 | |
| *** hieulq has joined #openstack-nova | 04:44 | |
| *** adreznec has joined #openstack-nova | 04:54 | |
| *** adreznec has quit IRC | 04:55 | |
| *** karthiks has joined #openstack-nova | 04:56 | |
| *** adreznec has joined #openstack-nova | 04:57 | |
| *** diga has joined #openstack-nova | 04:59 | |
| *** sree has joined #openstack-nova | 05:00 | |
| *** tidwellr has quit IRC | 05:02 | |
| *** huanxie has joined #openstack-nova | 05:02 | |
| *** BryanS68 has quit IRC | 05:06 | |
| *** slaweq has joined #openstack-nova | 05:07 | |
| *** Apoorva has quit IRC | 05:09 | |
| *** rcernin has quit IRC | 05:11 | |
| *** slaweq has quit IRC | 05:11 | |
| *** BryanS68 has joined #openstack-nova | 05:12 | |
| *** slaweq_ has joined #openstack-nova | 05:13 | |
| *** ratailor has joined #openstack-nova | 05:13 | |
| *** Apoorva has joined #openstack-nova | 05:13 | |
| *** Apoorva has quit IRC | 05:15 | |
| *** slaweq_ has quit IRC | 05:18 | |
| *** yikun_ has quit IRC | 05:22 | |
| *** Lingwu has joined #openstack-nova | 05:24 | |
| *** threestrands has quit IRC | 05:24 | |
| *** takashin has left #openstack-nova | 05:30 | |
| *** harlowja has joined #openstack-nova | 05:37 | |
| *** hongbin has quit IRC | 05:38 | |
| *** threestrands has joined #openstack-nova | 05:39 | |
| *** threestrands has quit IRC | 05:39 | |
| *** threestrands has joined #openstack-nova | 05:39 | |
| *** BryanS68 has quit IRC | 05:39 | |
| *** slaweq has joined #openstack-nova | 05:40 | |
| *** zhurong has joined #openstack-nova | 05:40 | |
| *** slaweq has quit IRC | 05:45 | |
| *** armax has joined #openstack-nova | 05:45 | |
| *** fragatina has joined #openstack-nova | 05:52 | |
| *** armax has quit IRC | 05:55 | |
| *** armax has joined #openstack-nova | 05:55 | |
| *** armax has quit IRC | 05:55 | |
| *** armax has joined #openstack-nova | 05:56 | |
| *** armax has quit IRC | 05:56 | |
| *** fragatina has quit IRC | 05:57 | |
| *** mikal has quit IRC | 05:57 | |
| *** mikal has joined #openstack-nova | 05:59 | |
| *** janki has joined #openstack-nova | 05:59 | |
| *** rcernin has joined #openstack-nova | 06:11 | |
| *** sree has quit IRC | 06:12 | |
| *** sree has joined #openstack-nova | 06:13 | |
| *** harlowja has quit IRC | 06:14 | |
| *** slaweq has joined #openstack-nova | 06:16 | |
| *** sree has quit IRC | 06:17 | |
| *** _heena_py_ has joined #openstack-nova | 06:19 | |
| _heena_py_ | Hi team, could anyone please help me with command where I have the server name and I need to know the floating IP | 06:20 |
| *** slaweq has quit IRC | 06:21 | |
| *** xinliang has quit IRC | 06:22 | |
| *** diga has quit IRC | 06:24 | |
| openstackgerrit | jichenjc proposed openstack/nova master: deprecate fping_path config option https://review.openstack.org/526602 | 06:26 |
| *** threestrands has quit IRC | 06:28 | |
| *** fragatina has joined #openstack-nova | 06:29 | |
| *** fragatina has quit IRC | 06:30 | |
| *** fragatina has joined #openstack-nova | 06:30 | |
| *** xinliang has joined #openstack-nova | 06:35 | |
| *** hui has joined #openstack-nova | 06:36 | |
| *** Shunli has joined #openstack-nova | 06:39 | |
| masber | _heena_py_, jut run ¨openstack server list¨ and it will show your server and the floating IP | 06:43 |
| *** sree has joined #openstack-nova | 06:43 | |
| masber | good afternoon, I have a host which is misbehaving. How could I tell nova not to schedule new vms into that host? I tried disabling the nova service but didnt work | 06:46 |
| masber | any idea? | 06:46 |
| *** zhurong has quit IRC | 06:47 | |
| openstackgerrit | Yikun Jiang (Kero) proposed openstack/nova master: Add pagination and changes-since for instance-actions https://review.openstack.org/326326 | 06:47 |
| *** vivsoni__ has quit IRC | 06:51 | |
| *** slaweq has joined #openstack-nova | 06:55 | |
| *** andreas_s has joined #openstack-nova | 06:57 | |
| *** sree has quit IRC | 06:57 | |
| *** sree has joined #openstack-nova | 06:58 | |
| *** slaweq has quit IRC | 07:00 | |
| *** sree has quit IRC | 07:03 | |
| -openstackstatus- NOTICE: Due to some unforseen Zuul issues the gate is under very high load and extremely unstable at the moment. This is likely to persist until PST morning | 07:03 | |
| *** ChanServ changes topic to "Due to some unforseen Zuul issues the gate is under very high load and extremely unstable at the moment. This is likely to persist until PST morning" | 07:03 | |
| *** serverascode has quit IRC | 07:07 | |
| *** serverascode has joined #openstack-nova | 07:07 | |
| *** kencjohnston has quit IRC | 07:08 | |
| *** melwitt has quit IRC | 07:08 | |
| *** sree has joined #openstack-nova | 07:10 | |
| *** melwitt has joined #openstack-nova | 07:10 | |
| *** melwitt is now known as Guest9054 | 07:11 | |
| *** kencjohnston has joined #openstack-nova | 07:13 | |
| *** bbbbzhao_ has quit IRC | 07:14 | |
| *** sree has quit IRC | 07:17 | |
| *** john51 has quit IRC | 07:17 | |
| *** john51 has joined #openstack-nova | 07:17 | |
| *** sree has joined #openstack-nova | 07:17 | |
| *** jamesdenton has quit IRC | 07:18 | |
| *** sree has quit IRC | 07:25 | |
| *** mdnadeem has joined #openstack-nova | 07:25 | |
| *** phuongnh has quit IRC | 07:27 | |
| *** moshele has joined #openstack-nova | 07:27 | |
| *** swamireddy has quit IRC | 07:30 | |
| *** huanxie has quit IRC | 07:31 | |
| *** slaweq has joined #openstack-nova | 07:34 | |
| *** slaweq has quit IRC | 07:39 | |
| *** sahid has joined #openstack-nova | 07:44 | |
| *** swamireddy has joined #openstack-nova | 07:46 | |
| *** slaweq has joined #openstack-nova | 07:46 | |
| *** sree has joined #openstack-nova | 07:47 | |
| *** slaweq has quit IRC | 07:50 | |
| *** slaweq has joined #openstack-nova | 07:50 | |
| *** slaweq has quit IRC | 07:51 | |
| *** david-lyle has joined #openstack-nova | 07:52 | |
| *** slaweq has joined #openstack-nova | 07:53 | |
| openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: vSCSI https://review.openstack.org/526094 | 08:06 |
| *** slaweq has quit IRC | 08:12 | |
| *** moshele has quit IRC | 08:15 | |
| *** zhurong has joined #openstack-nova | 08:16 | |
| *** huanxie has joined #openstack-nova | 08:16 | |
| *** tesseract has joined #openstack-nova | 08:20 | |
| openstackgerrit | Eli Qiao proposed openstack/nova master: Api-guide: Add Block Device Mapping https://review.openstack.org/522084 | 08:31 |
| *** sapd__ has joined #openstack-nova | 08:31 | |
| *** sapd_ has quit IRC | 08:31 | |
| *** adreznec has quit IRC | 08:32 | |
| *** adreznec has joined #openstack-nova | 08:32 | |
| *** vivsoni has joined #openstack-nova | 08:34 | |
| *** damien_r has joined #openstack-nova | 08:41 | |
| *** qsyqian has joined #openstack-nova | 08:47 | |
| *** hshiina has quit IRC | 08:47 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: Add instance action record for backup instances https://review.openstack.org/523676 | 08:48 |
| *** lucas-afk is now known as lucasagomes | 08:57 | |
| *** gszasz has joined #openstack-nova | 09:00 | |
| *** itlinux has quit IRC | 09:01 | |
| *** hoangcx has quit IRC | 09:05 | |
| *** owalsh has quit IRC | 09:10 | |
| *** andreas_s has quit IRC | 09:11 | |
| *** sapd__ has quit IRC | 09:11 | |
| *** owalsh has joined #openstack-nova | 09:12 | |
| *** andreas_s has joined #openstack-nova | 09:12 | |
| *** sapd has joined #openstack-nova | 09:12 | |
| *** andreas_s has quit IRC | 09:17 | |
| *** alexchadin has joined #openstack-nova | 09:17 | |
| *** karthiks is now known as karthiks_afk | 09:20 | |
| *** sapd_ has joined #openstack-nova | 09:22 | |
| *** sapd has quit IRC | 09:22 | |
| *** moshele has joined #openstack-nova | 09:22 | |
| *** andreas_s has joined #openstack-nova | 09:23 | |
| *** phuongnh has joined #openstack-nova | 09:24 | |
| *** andreas_s has quit IRC | 09:28 | |
| *** Shunli has quit IRC | 09:29 | |
| *** andreas_s has joined #openstack-nova | 09:37 | |
| *** Dinesh_Bhor has quit IRC | 09:37 | |
| *** yamamoto has quit IRC | 09:41 | |
| *** andreas_s has quit IRC | 09:42 | |
| *** yamamoto has joined #openstack-nova | 09:43 | |
| *** moshele has quit IRC | 09:46 | |
| *** gmann is now known as gmann_afk | 09:48 | |
| *** andreas_s has joined #openstack-nova | 09:51 | |
| *** andreas_s has quit IRC | 09:56 | |
| *** Alex_Staf has joined #openstack-nova | 09:56 | |
| *** andreas_s has joined #openstack-nova | 10:03 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add regression test for bug 1735407 https://review.openstack.org/526095 | 10:05 |
| openstack | bug 1735407 in OpenStack Compute (nova) "[Nova] Evacuation doesn't respect anti-affinity rules" [Medium,In progress] https://launchpad.net/bugs/1735407 - Assigned to Balazs Gibizer (balazs-gibizer) | 10:05 |
| *** annp has quit IRC | 10:07 | |
| *** brault has quit IRC | 10:11 | |
| *** brault has joined #openstack-nova | 10:11 | |
| *** gszasz has quit IRC | 10:12 | |
| *** brault has quit IRC | 10:12 | |
| *** andreas_s has quit IRC | 10:12 | |
| *** dtantsur|afk is now known as dtantsur | 10:16 | |
| *** Alex_Staf has quit IRC | 10:16 | |
| *** pooja-jadhav is now known as pooja_jadhav | 10:19 | |
| *** namnh has quit IRC | 10:19 | |
| openstackgerrit | Jiao Pengju proposed openstack/python-novaclient master: Fix listing available images example error in nova.rst https://review.openstack.org/526569 | 10:19 |
| *** andreas_s has joined #openstack-nova | 10:19 | |
| *** ociuhandu has joined #openstack-nova | 10:20 | |
| openstackgerrit | Theodoros Tsioutsias proposed openstack/python-novaclient master: CommandError is raised for invalid server fields https://review.openstack.org/525110 | 10:21 |
| *** andreas_s has quit IRC | 10:24 | |
| *** sdague has joined #openstack-nova | 10:28 | |
| *** gszasz has joined #openstack-nova | 10:31 | |
| *** jichen has quit IRC | 10:33 | |
| *** andreas_s has joined #openstack-nova | 10:33 | |
| *** phuongnh has quit IRC | 10:39 | |
| *** ociuhandu has quit IRC | 10:40 | |
| *** zhurong has quit IRC | 10:40 | |
| *** derekh has joined #openstack-nova | 10:40 | |
| *** danpawlik has quit IRC | 10:42 | |
| *** danpawlik has joined #openstack-nova | 10:43 | |
| *** hoonetorg has quit IRC | 10:44 | |
| *** abhishekk has quit IRC | 10:45 | |
| *** gszasz has quit IRC | 10:45 | |
| *** hoonetorg has joined #openstack-nova | 10:45 | |
| *** andreas_s has quit IRC | 10:48 | |
| *** andreas_s has joined #openstack-nova | 10:48 | |
| *** gibi is now known as giblet | 10:57 | |
| *** andreas_s has quit IRC | 10:57 | |
| *** andreas_s has joined #openstack-nova | 10:58 | |
| *** danpawlik has quit IRC | 10:59 | |
| *** danpawlik has joined #openstack-nova | 11:00 | |
| *** danpawlik has quit IRC | 11:01 | |
| *** danpawlik has joined #openstack-nova | 11:03 | |
| *** qsyqian has quit IRC | 11:05 | |
| *** danpawlik has quit IRC | 11:06 | |
| *** karthiks_afk is now known as karthiks | 11:08 | |
| *** danpawlik has joined #openstack-nova | 11:10 | |
| *** qsyqian has joined #openstack-nova | 11:17 | |
| *** alexchadin has quit IRC | 11:17 | |
| *** openstackgerrit has quit IRC | 11:17 | |
| *** yamamoto has quit IRC | 11:21 | |
| *** sree has quit IRC | 11:25 | |
| *** chyka has joined #openstack-nova | 11:26 | |
| *** links has quit IRC | 11:28 | |
| *** claudiub has joined #openstack-nova | 11:28 | |
| *** udesale has quit IRC | 11:29 | |
| *** qsyqian has quit IRC | 11:30 | |
| *** chyka has quit IRC | 11:30 | |
| *** brault has joined #openstack-nova | 11:38 | |
| *** artom_ has joined #openstack-nova | 11:38 | |
| *** artom_ has quit IRC | 11:39 | |
| *** artom_ has joined #openstack-nova | 11:39 | |
| *** artom has quit IRC | 11:41 | |
| *** links has joined #openstack-nova | 11:41 | |
| *** liuzz has quit IRC | 11:44 | |
| *** dave-mccowan has joined #openstack-nova | 11:45 | |
| *** gszasz has joined #openstack-nova | 11:47 | |
| *** yamamoto has joined #openstack-nova | 11:49 | |
| *** tesseract has quit IRC | 11:50 | |
| *** tesseract has joined #openstack-nova | 11:51 | |
| *** tbachman has quit IRC | 11:52 | |
| *** adisky_ has quit IRC | 11:53 | |
| *** cdent has joined #openstack-nova | 11:59 | |
| *** kalyan has quit IRC | 12:02 | |
| *** andreas_s has quit IRC | 12:04 | |
| *** andreas_s has joined #openstack-nova | 12:05 | |
| *** andreas_s has quit IRC | 12:05 | |
| *** andreas_s has joined #openstack-nova | 12:05 | |
| *** artom_ has quit IRC | 12:06 | |
| giblet | what a slow day | 12:07 |
| cdent | gah! now you’ve done it | 12:07 |
| cdent | the quiet peace will now be broken | 12:07 |
| *** janki has quit IRC | 12:07 | |
| cdent | Is the gate still unhappy | 12:08 |
| giblet | yeah the gate contributes a lot to the slowness of the day | 12:09 |
| *** claudiub has quit IRC | 12:14 | |
| *** andreas_s has quit IRC | 12:17 | |
| *** andreas_s has joined #openstack-nova | 12:17 | |
| *** yangyapeng has quit IRC | 12:24 | |
| *** TuanLA has quit IRC | 12:26 | |
| *** andreas_s has quit IRC | 12:27 | |
| *** efried is now known as fried_rice | 12:29 | |
| fried_rice | Morning folks. | 12:29 |
| fried_rice | Wow, still nothing merged since last night. | 12:30 |
| *** openstackgerrit has joined #openstack-nova | 12:30 | |
| openstackgerrit | Merged openstack/nova master: XenAPI: update the picture in Xen hypervisor document https://review.openstack.org/523360 | 12:30 |
| giblet | fried_rice: welcome in the slow Friday land :) | 12:30 |
| cdent | fried_rice: gate has issues that likely won’t be resolved until morning time pst | 12:30 |
| *** yikun_ has joined #openstack-nova | 12:31 | |
| cdent | dansmith: when you come on scene can you help decide if the cells-related issue in https://bugs.launchpad.net/nova/+bug/1736101 ought to be considered a proper bug (comments welcome from everyone else too) | 12:31 |
| openstack | Launchpad bug 1736101 in OpenStack Compute (nova) "nova placement resource_providers DBDuplicateEntry when name repeat" [Undecided,New] | 12:31 |
| *** ratailor has quit IRC | 12:32 | |
| fried_rice | Just caught up on -infra, looks like they might have sorted it out around 3h ago. Now we just gotta wait for the backlog to push through. | 12:36 |
| fried_rice | Sokay, I only had 17 patches in queue. | 12:39 |
| cdent | clearly this is all your fault. you gotta baby the queue | 12:40 |
| *** andreas_s has joined #openstack-nova | 12:42 | |
| *** salv-orlando has joined #openstack-nova | 12:42 | |
| giblet | I hope by rechecking all the bounced patches we will DDoS the whole gate down again ;] | 12:45 |
| fried_rice | I'm deliberately holding off on 10 of them. | 12:45 |
| *** pchavva has joined #openstack-nova | 12:46 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add regression test for bug 1735407 https://review.openstack.org/526095 | 12:46 |
| openstack | bug 1735407 in OpenStack Compute (nova) "[Nova] Evacuation doesn't respect anti-affinity rules" [Medium,In progress] https://launchpad.net/bugs/1735407 - Assigned to Balazs Gibizer (balazs-gibizer) | 12:46 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add late server group policy check to rebuild https://review.openstack.org/525242 | 12:46 |
| giblet | oops, two more :) | 12:46 |
| fried_rice | cdent I've got about 4-4.5h in me today, then on vacation. Were you serious in your offer to babysit my code while I'm out? | 12:47 |
| *** andreas__ has joined #openstack-nova | 12:48 | |
| cdent | fried_rice: a) yes (in part because I want to understand it better ), b) I thought you were supposed to already be gone | 12:48 |
| sean-k-m2 | giblet: it wont ddos the gate. zuul will just enqueue them and run them when nodepool says there is an env availably to it | 12:48 |
| *** sean-k-m2 is now known as sean-k-mooney | 12:49 | |
| giblet | sean-k-mooney: what a relief | 12:49 |
| *** andreas_s has quit IRC | 12:49 | |
| * cdent tries coffee as cold medicine | 12:49 | |
| giblet | fried_rice, cdent: I'm here and active until 20th of Dec so I can offer at least my review help on those patches | 12:50 |
| sean-k-mooney | cdent: im pretty sure you can by coffee enfused cold medicine | 12:50 |
| fried_rice | cdent giblet Okay cool, so what I was gonna say is, if y'all got time over the next couple hours to look at those, I can answer any questions or explain any weirdness. | 12:50 |
| *** alexchadin has joined #openstack-nova | 12:50 | |
| fried_rice | sean-k-mooney Don't they call that "Irish coffee"? ;P | 12:51 |
| *** andreas_s has joined #openstack-nova | 12:51 | |
| sean-k-mooney | cdent: but if that does not work you can try the irish solution of brandy/wisky infused coffe | 12:51 |
| *** yamamoto has quit IRC | 12:51 | |
| *** andreas__ has quit IRC | 12:52 | |
| fried_rice | cdent Can you explain this delta? https://review.openstack.org/#/c/414230/2..6/nova/scheduler/client/report.py@180 In particular, why did we feel it necessary to check for both `openstack-request-id` and `x-openstack-request-id`? | 12:52 |
| cdent | sean-k-mooney: that will be later in the day, while I think a weekly resource providers update supported by whisky might be entertaining, it might not be all that useful | 12:54 |
| *** alexchadin has quit IRC | 12:54 | |
| cdent | fried_rice: because there was a brief time when both was going to be supported (as in moving to not using the x- prefix) but then that didn’t happen | 12:55 |
| *** alexchadin has joined #openstack-nova | 12:55 | |
| *** yangyapeng has joined #openstack-nova | 12:55 | |
| fried_rice | Okay, so https://review.openstack.org/#/c/523606/ is kosher? | 12:55 |
| *** andreas_s has quit IRC | 12:55 | |
| sean-k-mooney | x-openstack-request-id is the http header we use for the osprofiler stuff right? | 12:56 |
| fried_rice | oh, you're saying we still might move to the non-x version at some point, whereupon the above would have to be reverted? | 12:56 |
| sean-k-mooney | i taught all non standard http headers were required to have an x prefix | 12:56 |
| cdent | it is kosher, as I said on my +1 on ps3 | 12:56 |
| cdent | sean-k-mooney: they got rid of that requirement somewhere in the last 3 years or so | 12:57 |
| *** andreas_s has joined #openstack-nova | 12:57 | |
| cdent | fried_rice: even if we did move to wanting the new style, we would have to continue producing the old style, because of compatibility between version | 12:57 |
| fried_rice | Oh, I didn't see your review. Cool. | 12:57 |
| sean-k-mooney | cdent: i guess to prevent peopel from updating all there code to support both when its standariesed | 12:57 |
| cdent | it’s like how the microversion headers in nova always return and support both the old and new styles | 12:58 |
| cdent | sean-k-mooney: yes | 12:58 |
| sean-k-mooney | in anycase key.endswith('-request-id') was a little too broad before i like the v6 version more that said wont this rais an exception if neither form are found | 13:00 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: [placement] Add x-openstack-request-id in API ref https://review.openstack.org/523007 | 13:01 |
| *** andreas_s has quit IRC | 13:01 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2) https://review.openstack.org/430608 | 13:02 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: List/show all server migration types (2/2) https://review.openstack.org/459483 | 13:04 |
| *** liverpooler has joined #openstack-nova | 13:05 | |
| *** stephenfin is now known as finucannot | 13:05 | |
| sean-k-mooney | actully never mind if .get does not find the key and no default is given it returns None instead of raising an exception | 13:05 |
| *** diga has joined #openstack-nova | 13:06 | |
| *** jaosorior has quit IRC | 13:12 | |
| *** jaosorior has joined #openstack-nova | 13:13 | |
| openstackgerrit | Silvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts https://review.openstack.org/522245 | 13:17 |
| *** claudiub has joined #openstack-nova | 13:20 | |
| *** jaypipes has joined #openstack-nova | 13:28 | |
| *** salv-orlando has quit IRC | 13:29 | |
| *** jaypipes is now known as leakypipes | 13:29 | |
| *** salv-orlando has joined #openstack-nova | 13:30 | |
| *** trozet has quit IRC | 13:31 | |
| giblet | fried_rice: regarding https://review.openstack.org/#/c/521605 Can we change the traits of an RP without bumping its generation? | 13:32 |
| giblet | fried_rice: I wondering about having the default None in https://review.openstack.org/#/c/521605/5/nova/compute/provider_tree.py@134 is necessary | 13:33 |
| *** salv-orlando has quit IRC | 13:34 | |
| fried_rice | giblet So | 13:35 |
| fried_rice | IIUC, the generation is intended to be used to detect async updates when we push stuff back to placement. | 13:35 |
| fried_rice | at the moment, I feel like it's not well defined how it's supposed to get used. | 13:36 |
| *** edleafe is now known as figleaf | 13:36 | |
| fried_rice | I.e. who's responsible for bumping the generation, who's responsible for checking for inconsistencies, what the result of an inconsistency should be (exception? HTTP error? Retry? ...) | 13:36 |
| giblet | fried_rice: I think the client should send in bumped generation and the server is responsible to reject update with outdated generation | 13:37 |
| fried_rice | I think that's about the gist of it, yeah. | 13:37 |
| leakypipes | fried_rice: the nova.objects.resource_provider._increment_provider_generation() function pretty well explains it. | 13:38 |
| * fried_rice looks... | 13:38 | |
| *** lpetrut has joined #openstack-nova | 13:39 | |
| fried_rice | leakypipes So the server side is responsible for incrementing the generation. | 13:39 |
| fried_rice | It should never be done by the client | 13:40 |
| *** markvoelker has quit IRC | 13:40 | |
| fried_rice | The client is supposed to take the generation from the server side, make whatever changes to the *other* fields, and push back the object with the *same* generation, which lets the server detect whether a concurrent update occurred. | 13:40 |
| fried_rice | IOW, the only time we should be updating the generation in ProviderTree is when we've freshly retrieved the object from placement. | 13:41 |
| giblet | leakypipes, fried_rice: thanks, now it make sense | 13:42 |
| giblet | fried_rice: so the case when generation is None in https://review.openstack.org/#/c/521605/5/nova/compute/provider_tree.py@134 is when the client update the trait of the RP | 13:43 |
| *** markvoelker has joined #openstack-nova | 13:43 | |
| leakypipes | fried_rice: the client is responsible for refreshing its knowledge of the latest generation, yes. | 13:43 |
| fried_rice | giblet Yeah, so the code path (in a subsequent patch in the series) is going to be a GET of the RP's traits. Uhm, I need to check whether that response includes the RP generation I suppose. | 13:44 |
| leakypipes | fried_rice: it does not. | 13:44 |
| fried_rice | Okay. Then we would have no way to know what to update it to. | 13:44 |
| fried_rice | In fact, I'm not even sure we should have the generation arg there at all... | 13:45 |
| fried_rice | Same for aggregates, I assume. | 13:45 |
| *** _heena_py_ has quit IRC | 13:45 | |
| leakypipes | fried_rice: changing aggregates doesn't increment the generation. | 13:46 |
| *** peter-hamilton has joined #openstack-nova | 13:47 | |
| leakypipes | fried_rice: because when you associate or disassociate an aggregate, you aren't changing the state of the resource provider's own attributes (like traits or inventories). there could be an argument made for incrementing the generation when aggs change on a rp, but I'm a bit skeptical about that | 13:47 |
| *** tesseract has quit IRC | 13:47 | |
| fried_rice | leakypipes But generation would change if you update traits? | 13:48 |
| giblet | leakypipes: the worst thing is that two parallel aggregate update on the same RP overwrites each others | 13:48 |
| *** mriedem has joined #openstack-nova | 13:49 | |
| leakypipes | giblet: sure, understood. | 13:50 |
| *** mlavalle has joined #openstack-nova | 13:50 | |
| leakypipes | fried_rice: yes. | 13:50 |
| leakypipes | fried_rice: for instance, if/when we support setting distances between aggregates (for affinity/anti-affinity), some attribute of the aggregate is being changed that could affect placement decisions and therefore the generation should be incremented. | 13:50 |
| *** tesseract has joined #openstack-nova | 13:50 | |
| fried_rice | That makes sense. But in that case, the RP generation should really come back with the GET of traits/aggs, nah? | 13:51 |
| fried_rice | leakypipes But then we run into the issue of: if I have my RP cached at generation 1, and I GET traits and they come back at generation 2, I really ought to re-GET the RP - and on and on until the generation agrees on all the GETs. | 13:52 |
| *** yamamoto has joined #openstack-nova | 13:52 | |
| leakypipes | fried_rice: https://github.com/openstack/nova/blob/master/nova/api/openstack/placement/handlers/trait.py#L215 | 13:52 |
| leakypipes | fried_rice: the generation is return when listing traits. | 13:52 |
| mriedem | so i guess zuul just totally crapped out last night huh | 13:53 |
| leakypipes | mriedem: yup | 13:53 |
| leakypipes | fried_rice: correctomundo. | 13:53 |
| leakypipes | fried_rice: which is why your proposed (GET /resource_providers_and_all_the_subresources call makes sense to me. | 13:53 |
| fried_rice | mriedem They restarted some stuff that may have fixed it around 4h ago, but backlog will take a while to clear. | 13:55 |
| mriedem | yikes | 13:55 |
| mriedem | we've had things sitting in the gate for 2 days now | 13:55 |
| mriedem | but ok | 13:55 |
| *** yamahata has joined #openstack-nova | 13:55 | |
| fried_rice | leakypipes Okay, I had gotten the impression that that guy wasn't going to get traction; the code in my current series does all of that via a dozen separate calls (but without the generation checking). | 13:56 |
| *** yamamoto has quit IRC | 13:58 | |
| leakypipes | fried_rice: ack. which is fine, frankly. it may seem like there's a potential for data corruption, but aside from the aggregates, there isn't. | 13:58 |
| cdent | fried_rice: i think the lack of traction is not because it is a fundamentally bad idea, but because there’s insufficient sharing of info on what all that stuff’s doing (as I’ve noted in the rp updated pushed out just now) | 13:58 |
| *** alexchadin has quit IRC | 14:00 | |
| *** ChanServ changes topic to "This channel is for Nova development. For support of Nova deployments, please use #openstack. Please see: https://wiki.openstack.org/wiki/Nova/Queens_Release_Schedule" | 14:00 | |
| -openstackstatus- NOTICE: The issues have been fixed, Zuul is operating fine again but has a large backlog. You can recheck jobs that failed. | 14:00 | |
| fried_rice | cdent sorry, noted where? | 14:00 |
| *** yamamoto has joined #openstack-nova | 14:01 | |
| cdent | fried_rice: in the nested providers section of http://lists.openstack.org/pipermail/openstack-dev/2017-December/125318.html | 14:01 |
| *** kuzko has quit IRC | 14:02 | |
| leakypipes | fried_rice: cdent is referring to his weekly status email | 14:03 |
| leakypipes | oh, jinx | 14:03 |
| fried_rice | aha | 14:03 |
| *** baoli has joined #openstack-nova | 14:04 | |
| leakypipes | cdent, giblet, fried_rice: in any case, to just wrap this conversation up in a tidy little bow, the resource provider generation protects the resource provider's important attributes (inventories, traits, allocations, etc) against concurrent writes. It means that yes, you need to occasionally refresh that generation if you get a 409 Conflict about a concurrent write. | 14:04 |
| cdent | leakypipes: we still haven’t done server side retry of allocations have we? reckon we should get that in sooner than later or does it matter? | 14:05 |
| fried_rice | leakypipes At the moment the only thing that's actually doing RP updates (qua updates - not fresh creates) is inventory. And there's no concurrency issues yet because it's always 1:1 compute node to RP. | 14:06 |
| leakypipes | but also keep in mind that that refresh/concurrent update on the provider generation is exceedingly rare in the ProviderTree (since the compute host is what houses the ProviderTree and all operations against it are currently done while holding a semaphore in the compute manager. The scheduler's claim_resources() block is where the contention is, but it's designed for quick retries when it receives a 409 Conflict for either a concurrent update | 14:06 |
| leakypipes | or a capacity exceeded due to a concurrent claim | 14:06 |
| leakypipes | fried_rice: claim_resources() in the scheduler is the only hot-spot currently. | 14:07 |
| leakypipes | fried_rice: and that is by design. | 14:07 |
| fried_rice | wait, claims don't update a RP generation, do they?? | 14:07 |
| leakypipes | cdent: nope, only client-side (in the reportclient.claim_resources(*) | 14:07 |
| leakypipes | fried_rice: they absolutely do. | 14:08 |
| fried_rice | oh, dang, then we definitely need to do more than we're doing. | 14:08 |
| leakypipes | fried_rice: https://github.com/openstack/nova/blob/master/nova/objects/resource_provider.py#L2098 | 14:08 |
| cdent | leakypipes: Is that “no we shouldn’t bother” or “ack, I agree we aren’t” | 14:09 |
| *** kuzko has joined #openstack-nova | 14:09 | |
| *** tbachman has joined #openstack-nova | 14:09 | |
| figleaf | mriedem: heh, you are already rechecking the alternate host series | 14:09 |
| leakypipes | cdent: on the server-side retry thing? that was a "no, we just haven't gotten to that yet and I'm not too worried because we have client-side retries" | 14:10 |
| leakypipes | cdent: i.e. the server-side would be a welcome optimization but isn't unsafe | 14:10 |
| figleaf | mriedem: I had them open, and didn't see your update until I rechecked 'em | 14:10 |
| cdent | ✔ | 14:10 |
| mriedem | i have rechecked all things | 14:10 |
| figleaf | mriedem: cool, thanks | 14:10 |
| *** sahid has quit IRC | 14:11 | |
| leakypipes | diga: whatup? :) | 14:11 |
| *** trozet has joined #openstack-nova | 14:11 | |
| diga | leakypipes: Hi | 14:11 |
| leakypipes | diga: the thing ew've been discussing -- server-side retries of the claim_resources() logic -- is something I think you might be interested in contributing. | 14:11 |
| *** lyan has joined #openstack-nova | 14:12 | |
| leakypipes | diga: it all stems from the following TODO left in the code: | 14:12 |
| leakypipes | https://github.com/openstack/nova/blob/master/nova/objects/resource_provider.py#L2148-L2152 | 14:12 |
| diga | leakypipes: yep, this can be gr8 opportunity for me to pitch in :) | 14:12 |
| diga | leakypipes: let me go through this | 14:12 |
| diga | leakypipes: How can I test this first | 14:13 |
| *** kuzko has quit IRC | 14:13 | |
| leakypipes | diga: currently, we are doing a retry of the "claim resources" transaction from the client side (i.e. in the scheduler). It would be an optimization to be able to retry the transaction from the placement server side (thus saving the round-trip HTTP call) | 14:13 |
| *** sahid has joined #openstack-nova | 14:14 | |
| fried_rice | diga The real life scenario would be to have two separate threads doing claims, and make sure they do them in a certain sequence. | 14:15 |
| openstackgerrit | Surya Seetharaman proposed openstack/nova master: update_cell allows more than once cell to have the same db/transport url https://review.openstack.org/518556 | 14:15 |
| fried_rice | diga For test purposes, you could simulate it by cloning the local RP data, doing a claim with one copy, and then doing a different claim with the second copy (which will have the old generation). | 14:15 |
| *** kuzko has joined #openstack-nova | 14:16 | |
| *** sahid has quit IRC | 14:19 | |
| cdent | sigh, I was wrong about deploy.deploy() being a public interface, it’s deploy.loadapp() that I was thinking of. I’ll fix things appropriately. | 14:19 |
| *** kuzko has quit IRC | 14:20 | |
| *** tssurya has quit IRC | 14:22 | |
| *** gbarros has joined #openstack-nova | 14:23 | |
| *** sahid has joined #openstack-nova | 14:24 | |
| *** esberglu has joined #openstack-nova | 14:24 | |
| *** dansmith is now known as superdan | 14:25 | |
| *** tbachman_ has joined #openstack-nova | 14:31 | |
| *** diga has quit IRC | 14:32 | |
| *** tbachman has quit IRC | 14:33 | |
| *** tbachman_ is now known as tbachman | 14:33 | |
| *** links has quit IRC | 14:37 | |
| *** trozet has quit IRC | 14:38 | |
| *** kuzko has joined #openstack-nova | 14:38 | |
| mriedem | fried_rice: do we need to hold https://review.openstack.org/#/c/522112/ for https://review.openstack.org/#/c/508345/ or can yours be stacked on top? | 14:39 |
| mriedem | diga left huh | 14:40 |
| mriedem | was going to point them at how we deal with this client-side today https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L98 | 14:40 |
| *** awaugama has joined #openstack-nova | 14:41 | |
| *** kuzko has quit IRC | 14:43 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: [placement] annotate loadapp as public interface https://review.openstack.org/526691 | 14:45 |
| *** diga has joined #openstack-nova | 14:47 | |
| diga | leakypipes: Hi | 14:47 |
| *** yamahata has quit IRC | 14:47 | |
| diga | sorry got disconnected | 14:47 |
| *** gbarros has quit IRC | 14:48 | |
| diga | leakypipes: I will try to test it as per your suggestion, if anything requires, will ping you | 14:48 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: fixes mock autospec usage https://review.openstack.org/447505 | 14:51 |
| *** artom has joined #openstack-nova | 14:51 | |
| fried_rice | mriedem Either way will work. Mine isn't working yet, so do the other first. | 14:52 |
| diga | cdent: I will ping you if need help | 14:52 |
| *** Guest9054 is now known as melwitt | 14:52 | |
| mriedem | fried_rice: ok - did you see anything wrong with the way they are doing the ksa options in that one? | 14:52 |
| fried_rice | mriedem No, looks fine. Mine will need a manual rebase to accomodate, of course. | 14:53 |
| *** rcernin has quit IRC | 14:55 | |
| fried_rice | mriedem I left a +1. | 14:56 |
| mriedem | thanks | 14:56 |
| *** andreas_s has joined #openstack-nova | 15:02 | |
| openstackgerrit | Jackie Truong proposed openstack/python-novaclient master: Microversion 2.57 - Add trusted_image_certificates https://review.openstack.org/500396 | 15:02 |
| *** salv-orlando has joined #openstack-nova | 15:04 | |
| *** hongbin has joined #openstack-nova | 15:07 | |
| *** BryanS68 has joined #openstack-nova | 15:09 | |
| *** hemna_ has joined #openstack-nova | 15:09 | |
| *** burt has joined #openstack-nova | 15:10 | |
| *** tssurya has joined #openstack-nova | 15:12 | |
| *** tidwellr has joined #openstack-nova | 15:13 | |
| *** diga has quit IRC | 15:14 | |
| *** tssurya has quit IRC | 15:17 | |
| *** hemna_ has quit IRC | 15:18 | |
| openstackgerrit | Jackie Truong proposed openstack/python-novaclient master: Microversion 2.57 - Add trusted_image_certificates https://review.openstack.org/500396 | 15:19 |
| *** trozet has joined #openstack-nova | 15:20 | |
| *** tssurya has joined #openstack-nova | 15:20 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate file injection https://review.openstack.org/522027 | 15:22 |
| *** Apoorva has joined #openstack-nova | 15:22 | |
| *** tssurya has quit IRC | 15:23 | |
| *** burt has quit IRC | 15:23 | |
| *** armax has joined #openstack-nova | 15:24 | |
| openstackgerrit | Jackie Truong proposed openstack/python-novaclient master: Microversion 2.57 - Add trusted_image_certificates https://review.openstack.org/500396 | 15:25 |
| *** tssurya has joined #openstack-nova | 15:25 | |
| *** burt has joined #openstack-nova | 15:26 | |
| *** hemna_ has joined #openstack-nova | 15:27 | |
| *** claudiub has quit IRC | 15:28 | |
| *** Apoorva has quit IRC | 15:28 | |
| melwitt | mriedem: I utilitized the quotas follow up patch https://review.openstack.org/#/c/524234 | 15:30 |
| mriedem | ok, currently re-reviewing superdan's live migration allocations thing | 15:31 |
| melwitt | sahid: would you be able to take a look at this review where there's a bug with disk device addresses being changed during a live migration? https://review.openstack.org/#/c/518022 | 15:34 |
| *** salv-orlando has quit IRC | 15:34 | |
| melwitt | it looks like it's a regression caused by a different bug fix from the past https://review.openstack.org/#/c/459741 | 15:35 |
| *** salv-orlando has joined #openstack-nova | 15:35 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: [placement] add name to resource provider create error https://review.openstack.org/526710 | 15:36 |
| sahid | melwitt: i think i did that this morning | 15:36 |
| sahid | is there something not clear you want to discuss? | 15:37 |
| melwitt | mdbooth: it would be good to have your review on this change for fixing the cache value with regard to O_DIRECT support during an image conversion https://review.openstack.org/#/c/523554 | 15:37 |
| mdbooth | melwitt: I vaguely recall having looked at something similar, if not this | 15:38 |
| melwitt | sahid: oh geez, sorry. I didn't see that you reviewed it this morning. I had it on my todo to ask you the past few days and just did it blindly. thanks | 15:38 |
| *** amodi has joined #openstack-nova | 15:38 | |
| *** liangy has joined #openstack-nova | 15:39 | |
| *** lyan has quit IRC | 15:39 | |
| *** salv-orlando has quit IRC | 15:40 | |
| openstackgerrit | Merged openstack/nova master: Avoid stashed connector lookup for new style detach https://review.openstack.org/526182 | 15:43 |
| melwitt | mdbooth: most of the patch is moving the supports_direct_io check to nova/utils.py so that it could be used by both virt/images and libvirt/driver. but the main part of the fix is in here https://review.openstack.org/#/c/523554/6/nova/virt/images.py | 15:43 |
| * superdan opens the live migration patch and waits for mriedem's next -1 to roll in | 15:44 | |
| mriedem | superdan: done | 15:47 |
| mriedem | however, it's an easy one | 15:47 |
| superdan | received | 15:47 |
| *** felipemonteiro_ has joined #openstack-nova | 15:47 | |
| *** tssurya has left #openstack-nova | 15:49 | |
| *** tssurya has joined #openstack-nova | 15:49 | |
| *** tssurya has quit IRC | 15:50 | |
| leakypipes | mriedem: I'm trying to square your code comment here: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L4591-L4595 with the following in the functional test case setup here: https://github.com/openstack/nova/blob/master/nova/tests/functional/compute/test_host_api.py#L76-L79 | 15:50 |
| leakypipes | mriedem: the reason is I have some PoC code I'm working on locally that would allow host aggregates to be associated to compute nodes (via UUID) in addition to compute *services* by service hostname | 15:51 |
| cdent | superdan: not sure if you saw my earlier ping on this potentially cells related bug that needs some evaluation: https://bugs.launchpad.net/nova/+bug/1736101 | 15:51 |
| openstack | Launchpad bug 1736101 in OpenStack Compute (nova) "nova placement resource_providers DBDuplicateEntry when name repeat" [Undecided,New] | 15:51 |
| superdan | I did not | 15:51 |
| mriedem | leakypipes: https://github.com/openstack/nova/blob/master/nova/tests/functional/compute/test_host_api.py#L78 is just test data, the hostmapping.host wouldn't be a uuid | 15:51 |
| *** felipemonteiro__ has joined #openstack-nova | 15:51 | |
| mriedem | leakypipes: at least, not a uuid that matches the computenode.uuid | 15:52 |
| leakypipes | mriedem: sorry, I'm not following you... | 15:52 |
| leakypipes | mriedem: you're creating a HostMapping with the host attribute of the mapping set to the compute node's UUID. | 15:52 |
| superdan | cdent: so, two hosts with the same name is not a valid thing, but it sounds like they only have that situation because they're moving between cells? | 15:53 |
| leakypipes | mriedem: which is what the code comment in HostsAPI says we're not doing? | 15:53 |
| mriedem | leakypipes: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L4591-L4595 is just saying, we're looking up the compute node (in a cell) by uuid, (microversion >=2.53), if we had the uuid stored in the host mapping (api db), then we wouldn't have to iterate the cells querying each db for a compute node with that uuid | 15:53 |
| mriedem | we could just get the host mapping from the api db which would tell us which cell that compute node is in | 15:53 |
| cdent | superdan: I struggled to parse it and wasn’t entirely clear on the naming constraints | 15:53 |
| mriedem | leakypipes: https://github.com/openstack/nova/blob/master/nova/tests/functional/compute/test_host_api.py#L78 - the hostmapping.host field could be "turd.ferguson" | 15:53 |
| superdan | cdent: the conflict is in the placement db, right? | 15:53 |
| cdent | yes, rp names are supposed to be unique | 15:54 |
| superdan | cdent: the rp name comes from what though? | 15:54 |
| leakypipes | mriedem: ok. so are we adding HostMapping records for compute nodes now or are we not doing that? | 15:54 |
| mriedem | leakypipes: the test is just using something random, which in this case is the uuid that matches the compute node in the cell - confusing the test i guess | 15:54 |
| leakypipes | mriedem: note: compute nodes, not service hosts. | 15:54 |
| mriedem | leakypipes: we are adding host mappings for service hosts | 15:54 |
| mriedem | not compute nodes | 15:54 |
| leakypipes | mriedem: ok, that's what I needed to know, thanks man | 15:54 |
| cdent | superdan: that’s what I’m not clear on. some portion of the host? If it’s not the fqdn, then that’s a clear potential for problems | 15:54 |
| leakypipes | mriedem: I'm writing func tests for this code I'm proposing, thus the question... | 15:55 |
| mriedem | leakypipes: this is what creates the host mappings http://git.openstack.org/cgit/openstack/nova/tree/nova/objects/host_mapping.py#n192 | 15:55 |
| superdan | cdent: I wonder if it is something like the hostname, and that moving between cells has caused the name to be the same but the uuid change? | 15:55 |
| mriedem | http://git.openstack.org/cgit/openstack/nova/tree/nova/objects/host_mapping.py#n177 | 15:55 |
| leakypipes | mriedem: I copied test_hosts_api.py and saw that after reading the comment in api.py about us not storing compute node records in host mappings table and was like... uh? | 15:55 |
| mriedem | host_mapping = HostMapping( ctxt, host=compute.host, cell_mapping=cm) host_mapping.create() | 15:55 |
| *** felipemonteiro_ has quit IRC | 15:55 | |
| mriedem | leakypipes: the api comment says "We wouldn't have to loop over cells if we stored the ComputeNode.uuid in the HostMapping" - specifically the uuid, but yeah i can see how that's confusing now | 15:56 |
| mriedem | because even if we did, it'd be wrong | 15:56 |
| mriedem | we should probably remove that comment | 15:56 |
| * mriedem patches | 15:56 | |
| leakypipes | mriedem: right. I'm hacking around the host mapping stuff, though, and calling ComputeNode.get_all() on all cells via the context.scatter_gather_skip_cell0() function, since I figured it would not be possible to look up compute node information in the host_mappings table. | 15:57 |
| cdent | superdan: I haven’t had much of a chance to think about it, I was just going through placement tagged bugs and that one was making a dance between invalid and new and thought I better try to make sure it wasn’t lost if it is a real issue | 15:57 |
| leakypipes | s/get_all()/get_by_uuid() | 15:57 |
| mriedem | leakypipes: yeah correct - only the compute_nodes.host field is in there, and if you're ironic that doesn't help you | 15:57 |
| mriedem | because 1:M | 15:57 |
| superdan | cdent: well, I imagine it's a real issue, but I'm not sure exactly what to tell them.. I guess you could provide them the curl commands to grab a token and delete the old provider record to confirm that it heals up properly afterwards right? | 15:58 |
| leakypipes | mriedem: FYI, this is to support PUT /os-aggregates/{agg_id} and passing a compute node UUID instead of a service hostname. | 15:58 |
| leakypipes | mriedem: since our friends in Ironic can't use host aggregates due to the service host coupling. | 15:58 |
| leakypipes | mriedem: and our Ironic friends can't use any aggregate or affinity filter that relies on host aggregates being associated with an Ironic resource node. | 15:59 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove confusing comment in compute_node_get API method https://review.openstack.org/526718 | 16:00 |
| mriedem | leakypipes: see if this floats your boat ^ | 16:00 |
| *** tbachman has quit IRC | 16:00 | |
| leakypipes | mriedem: of course, I am going to need to backport all of this to Ocata, when none of the context.scatter_gather stuff was in. so that should be super-fun. | 16:00 |
| mriedem | leakypipes: you can't add aggregates support for ironic in nova because mogan | 16:00 |
| sean-k-mooney | leakypipes: FYI i talked to infra and at least for now they are ok with enableing legacy jobs so ill submit a patch for os-vif fuctional jobs on monday. ill see if i can migrate os-vif to in repo zuul v3 jobs before the PTG | 16:00 |
| leakypipes | mriedem: hmm? | 16:00 |
| mriedem | leakypipes: you mean backport internally... | 16:00 |
| leakypipes | mriedem: correcto. | 16:00 |
| mriedem | fun | 16:00 |
| mriedem | oath? | 16:00 |
| leakypipes | yuppers. | 16:00 |
| mriedem | great | 16:00 |
| cdent | superdan: when I say “ I haven’t had much of a chance to think about it” I really mean that. I’ve not processed what they are doing at all. If you’re trying to encourage me to take some responsibility for flagging something up, that’s great, and I can, but it’s not on my mental stack, despite the presumption of trying to stick ti on yours… :) | 16:00 |
| leakypipes | I blame the penick. | 16:01 |
| mriedem | leakypipes: oh right you weren't at the mogen session in sydney | 16:01 |
| mriedem | aggregates support in mogan was one of the differentiators | 16:01 |
| leakypipes | mriedem: I wasn't in Sydney. period. :) | 16:01 |
| mriedem | i could have sworn... | 16:01 |
| mriedem | leakypipes: so, | 16:01 |
| mriedem | are you going to do this upstream with a new microversion? | 16:01 |
| leakypipes | mriedem: well, frankly, a placement aggregate doesn't have the service host coupling that a nova host-aggregate currently does, so I'm just trying to get a patch in place for Ocata Nova that kind of gets nova host-aggs looking/functioning more like placement aggs | 16:02 |
| mriedem | what happens when oath upgrades to pike or queens? | 16:02 |
| leakypipes | mriedem: I'll throw my code up and have you take a looksie, of course. | 16:02 |
| mriedem | well, | 16:02 |
| leakypipes | mriedem: when oath upgrades to pike/queens, we'll likely forward-port this thing I'm wokring on | 16:03 |
| mriedem | my point is, backporting microversions opens up some bad issues unless you backport all intermediate microversions too | 16:03 |
| leakypipes | mriedem: until placement aggs are fully utilized. | 16:03 |
| *** artom has quit IRC | 16:03 | |
| leakypipes | mriedem: I'm not doing this as a microversion :) | 16:03 |
| mriedem | ok | 16:03 |
| leakypipes | mriedem: I'm not changing the API at all. | 16:03 |
| mriedem | "this is to support PUT /os-aggregates/{agg_id} and passing a compute node UUID instead of a service hostname." | 16:03 |
| leakypipes | mriedem: just looking to see if hostname is like a UUID and if so, looking up a compute node instead of a service host. | 16:03 |
| mriedem | ok, | 16:04 |
| mriedem | was just going to say that :) | 16:04 |
| leakypipes | it's cheeky, I know. | 16:04 |
| mriedem | tricksy | 16:04 |
| leakypipes | cheeky cheeky monkey. | 16:04 |
| mriedem | does oath also want to be able to cold migrate/resize baremetal instances? | 16:04 |
| sean-k-mooney | superdan: i was under the impression that we required the hostID to be gloally unique across the entire cloud not jsut within a cell so if they are useing hostnames as the hostid .i.e the default then they would be required to have unique host names across all cells | 16:05 |
| superdan | sean-k-mooney: correct, but they're moving a host between cells | 16:05 |
| mriedem | leakypipes: because https://review.openstack.org/#/c/449155/ | 16:06 |
| sean-k-mooney | superdan: yes but i was under the impression that the host id had to be unique period e.g. its invalid to have to hosts with the same hostname in different cells if you are using the hostname as the hostid | 16:06 |
| leakypipes | mriedem: that hasn't been brought up, no | 16:06 |
| superdan | sean-k-mooney: yes, that's true, I'm not sure what you're getting at | 16:07 |
| superdan | sean-k-mooney: they are moving hosts between cells, which means the host gets a new UUID, and I think the problem is they're reporting a new uuid with the same name, and placement is kicking it out as a result | 16:07 |
| finucannot | melwitt: Could you take another look at this some time today/early next week? https://review.openstack.org/#/q/topic:bp/websocket-proxy-to-host-security+status:open | 16:09 |
| cdent | how does someone move a host between cells? Is that a command they run to which “remove the resource provider” needs to be added? | 16:09 |
| sean-k-mooney | superdan: oh sorry they are moving the physical host between cells. not that they actlly have 2 phyical host with the same hostname | 16:09 |
| superdan | correct | 16:09 |
| dmsimard | mriedem, kashyap: would love if we could iron out https://bugs.launchpad.net/nova/+bug/1735823 soon, it's making a non-negligible amount of jobs flap | 16:09 |
| openstack | Launchpad bug 1735823 in OpenStack Compute (nova) "Nova can hang when creating a VM with disk injection" [Medium,In progress] - Assigned to Matt Riedemann (mriedem) | 16:09 |
| mriedem | cdent: very carefully | 16:09 |
| cdent | as is so often the case | 16:09 |
| superdan | cdent: that's my point | 16:09 |
| melwitt | finucannot: been meaning to do that, yes I will | 16:09 |
| kashyap | dmsimard: Hey, I've seen it; but a bit overwhelmed today | 16:09 |
| kashyap | dmsimard: I haven't forgotten it | 16:09 |
| superdan | cdent: if they're doing this thing, its going to leave residue we likely don't have a cleanup command for | 16:09 |
| dmsimard | kashyap: I know the feeling, just a friendly reminder :) | 16:09 |
| mriedem | dmsimard: are these only centos jobs? | 16:09 |
| kashyap | Yeah, was wondering the same. | 16:10 |
| cdent | superdan: ah okay, I think I’m _finally_ caught up then. I was way way back there in the weeds. | 16:10 |
| mriedem | i'm assuming the problem is some difference in the guestfs package versions | 16:10 |
| *** liangy has quit IRC | 16:10 | |
| cdent | I blame society. Or my head cold. Or just being stupid. | 16:10 |
| mriedem | dmsimard: because https://review.openstack.org/#/c/524748/ has been busted for a long time, and we know it doesn't fix the issue | 16:10 |
| dmsimard | mriedem: the data that we have has problems on centos and opensuse, I couldn't find any on Ubuntu but that doesn't mean it doesn't happen there | 16:10 |
| mriedem | i think the lock issue is the fact we're using eventlet.tpool | 16:11 |
| *** liangy has joined #openstack-nova | 16:11 | |
| dmsimard | mriedem: well, it kind of fixes the issue but then generates new ones ? right ? the patch with the depends-on fails horribly | 16:11 |
| *** amodi has quit IRC | 16:11 | |
| mriedem | https://review.openstack.org/#/c/524750/ ? | 16:11 |
| kashyap | Yeah, that seems to be the test | 16:12 |
| finucannot | sean-k-mooney: What's this about functional tests? | 16:12 |
| finucannot | I assume you mean enabling ralonsoh's patches? | 16:12 |
| mriedem | oh boy http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-full/eb8d805/logs/screen-n-api.txt.gz?level=TRACE#_Dec_04_13_34_20_635874 | 16:13 |
| sean-k-mooney | finucannot: yes im going to add a job to the experimental pipline for os-vif so we can run rodolfos tests | 16:13 |
| sean-k-mooney | finucannot: once they are merged i can move it to the gate pipline | 16:13 |
| *** jaosorior has quit IRC | 16:13 | |
| finucannot | sean-k-mooney: Ideal | 16:13 |
| finucannot | gate for os-vif or nova too? | 16:14 |
| sean-k-mooney | check and gate for os-vif | 16:14 |
| superdan | leakypipes: finucannot: melwitt: either one of you can mindlessly approve this trivial thing without worry: https://review.openstack.org/#/c/507874/1 | 16:14 |
| *** itlinux has joined #openstack-nova | 16:14 | |
| leakypipes | superdan: BFV with BDMs? | 16:14 |
| finucannot | superdan: If there's one thing I know how to do... | 16:14 |
| superdan | leakypipes: not even | 16:15 |
| leakypipes | :) | 16:15 |
| superdan | finucannot: I knew I could count on you | 16:15 |
| finucannot | sean-k-mooney: Cool. We already run oslo.versionedobjects tests in the experimental pipeline for nova. Might be worth adding os-vif there too | 16:15 |
| finucannot | Eventually, anyway | 16:15 |
| *** weshay|ruck is now known as weshay|ruck|MOD | 16:15 | |
| sean-k-mooney | i would also like to move all our os-vif jobs to zuul v3 in repo config and add a linux-bridge job for os-vif but it will be a few week before i have time to do that | 16:15 |
| sean-k-mooney | finucannot: os-vif is already in the nova experimental pipline | 16:16 |
| sean-k-mooney | check experimtal on a nova patch will test that patch with the head of the os-vif master branch in a seperate job | 16:16 |
| finucannot | superdan: I know you asked for a mindless approval but...no bug? https://review.openstack.org/#/c/507874/ | 16:16 |
| superdan | finucannot: it's not a bug yet | 16:17 |
| sean-k-mooney | finucannot: the one thing it does not do is work with depend-on where the depends-on is an unmerged os-vif change | 16:17 |
| superdan | finucannot: it will be after my live migrations patch merges, it's just an omission right now, the last of a few fixes ahead of time | 16:17 |
| finucannot | superdan: Yup, just saw jianghuaw_'s comment now. Cool, +Wd | 16:17 |
| superdan | finucannot: thanks | 16:18 |
| mriedem | well it might be a bug | 16:18 |
| finucannot | sean-k-mooney: All the os-vif jobs or just the unit tests? | 16:18 |
| finucannot | *unit test jobs | 16:18 |
| mriedem | if the driver doesn't pass migrate_data, we don't set the migration.status to 'error' | 16:18 |
| sean-k-mooney | finucannot: ill try and move them all i think. we will see what infra advise | 16:19 |
| sean-k-mooney | finucannot: what has nova done? | 16:19 |
| mriedem | nothing | 16:19 |
| mriedem | novaclient on the other hand https://review.openstack.org/#/q/I43a8435485751748ca6228f67d401945cb32652e | 16:19 |
| finucannot | mriedem: Well https://review.openstack.org/#/q/status:open+branch:master+topic:zuulv3-nova | 16:19 |
| mriedem | sean-k-mooney: there are very few ci jobs that nova runs which are just run against nova | 16:19 |
| finucannot | We don't have to move much, thankfully | 16:19 |
| mriedem | i think the nova-next job is the only one | 16:19 |
| finucannot | Because of that, yes | 16:19 |
| mriedem | oh lxc/lvm | 16:20 |
| mriedem | umm, we can drop the wsgi-full job, unless we run that on stable | 16:20 |
| *** tidwellr has quit IRC | 16:20 | |
| mriedem | wsgi is the default in devstack since pike | 16:20 |
| finucannot | We don't run any of them on stable. Drop that entirely so? | 16:20 |
| sean-k-mooney | ah cool well i dont think it will be that hard to move os-vif. is there a plan to move nova eventually or just rely on the infra maintained jobs | 16:21 |
| mriedem | idk, i'd have to go through this with a comb | 16:21 |
| mriedem | ci job shit is always a minefield | 16:21 |
| finucannot | sean-k-mooney: Those patches simply copy in the legacy ones. I haven't wrapped my head around translating a job to zuulv3 yet | 16:21 |
| finucannot | despite mordred's best efforts | 16:22 |
| melwitt | would appreciate reviews on this relatively simple bug fix to set the right cache mode during image conversion. most of the patch is moving a function that checks for O_DIRECT support to nova/utils so it can be used by both virt/images and libvirt/driver. the actual fix is in https://review.openstack.org/#/c/523554/6/nova/virt/images.py | 16:22 |
| *** MasterOfBugs has joined #openstack-nova | 16:22 | |
| mordred | I didn't do it | 16:22 |
| *** pramodrj07 has joined #openstack-nova | 16:22 | |
| sean-k-mooney | finucannot: ya just too a look at your patches now. | 16:23 |
| mordred | finucannot: oh - well - I'm working on converting glanceclient jobs to be native at the moment (and finding a weird issue) - once I'm done with that, I'll circle back around and help update yours | 16:23 |
| finucannot | mriedem: The migration made it especially fun https://review.openstack.org/#/c/514310/3/zuul.d/zuul-legacy-jobs.yaml@6092 | 16:24 |
| sean-k-mooney | finucannot: there is a sports and social Q4 party tonight in the stand so ime going to head home soon but ill take a look at what you have started on monday | 16:24 |
| finucannot | mordred: I'd appreciate that. Will do whatever I can to help | 16:24 |
| finucannot | sean-k-mooney: Enjoy. We've the Dublin office party tonight too | 16:24 |
| sean-k-mooney | oh fun dublin has more interesting venues then the strand to choose form | 16:25 |
| openstackgerrit | Merged openstack/python-novaclient master: inject file: add method of showing quota value of injecting files for 'rebuild' command https://review.openstack.org/524588 | 16:25 |
| *** takedakn has joined #openstack-nova | 16:26 | |
| *** jangutter has quit IRC | 16:26 | |
| mordred | finucannot: it should actually be *fairly* easy ... once I figure out why the glanceclient job is somehow getting a broken mysql - but shade and python-openstackclient work fine | 16:26 |
| finucannot | Aye. The biggest issue I had was figuring out who was using what (and why). That's probably general infra- inexperience though | 16:27 |
| sean-k-mooney | mordred: is it insatlling in a venv and therefor may be using a different pymysql package | 16:27 |
| finucannot | and I haven't gone further down the zuulv3 rabbit hole that attempting to diff jobs before and after (I didn't get far) | 16:27 |
| *** baoli has quit IRC | 16:27 | |
| *** felipemonteiro__ has quit IRC | 16:28 | |
| *** felipemonteiro__ has joined #openstack-nova | 16:28 | |
| *** takedakn has quit IRC | 16:29 | |
| mriedem | dmsimard: are the centos or opensuse jobs voting anywhere? | 16:29 |
| dmsimard | mriedem: the devstack ones ? I'm not sure, but it's impacting non-devstack stuff like packstack | 16:30 |
| dmsimard | mriedem: tripleo doesn't run disk injection tests in tempest so they're not running into the issue | 16:31 |
| mriedem | dmsimard: so, has anyone tried downgrading the libguestfs package on centos to the version that we have in the UCA jobs and see if they still fail? | 16:31 |
| *** gszasz has quit IRC | 16:31 | |
| dmsimard | I haven't really tried reaching out to OSA and kolla if they've been noticing it too. | 16:31 |
| *** gszasz has joined #openstack-nova | 16:31 | |
| dmsimard | mriedem: no, not that I am aware of -- I'm not sure if that version is even available on EL7, I'd need to check. | 16:32 |
| mordred | sean-k-mooney: well - eventually it should - but it's actually failing in the devstack layer before it even gets there. it's super SUPER strange, but I'm certain I just left out a comma or something | 16:34 |
| *** lyan_ has joined #openstack-nova | 16:35 | |
| *** mlavalle has quit IRC | 16:35 | |
| *** liangy has quit IRC | 16:35 | |
| *** yangyape_ has joined #openstack-nova | 16:36 | |
| dmsimard | mriedem: yeah we don't have 1.32.2, closest I see is libguestfs-1.32.7-3.el7.centos.x86_64.rpm and that's in an archived release | 16:36 |
| *** lyan_ has quit IRC | 16:36 | |
| *** yangyapeng has quit IRC | 16:36 | |
| dmsimard | mriedem: it might be complicated to downgrade the package, it probably doesn't come on its own | 16:36 |
| finucannot | melwitt: I'll take a look now | 16:37 |
| finucannot | leakypipes: Could you take a look at https://review.openstack.org/#/c/390520/ again at some point in the next few days? | 16:38 |
| * finucannot figures you're the man for the job | 16:38 | |
| melwitt | thanks finucannot | 16:38 |
| *** baoli has joined #openstack-nova | 16:38 | |
| leakypipes | finucannot: yes, wil try my best | 16:38 |
| finucannot | leakypipes: Excellent. Appreciate it | 16:39 |
| *** AlexeyAbashkin has joined #openstack-nova | 16:40 | |
| mriedem | sdague: superdan: methinks we should consider deprecating the libvirt.inject_key option | 16:41 |
| mriedem | (1) depends on file injection | 16:41 |
| mriedem | (2) doesn't work for volume-backed instances | 16:42 |
| mriedem | (3) use config drive | 16:42 |
| *** itlinux has quit IRC | 16:42 | |
| superdan | i | 16:42 |
| superdan | am down | 16:42 |
| mriedem | same with inject_password | 16:43 |
| *** itlinux has joined #openstack-nova | 16:43 | |
| *** lyan has joined #openstack-nova | 16:43 | |
| mriedem | oh fun we pass down the libvirt virt_type when injecting network info because of lxc | 16:44 |
| *** AlexeyAbashkin has quit IRC | 16:44 | |
| finucannot | I've got a rather simply bugfix here that resolves an issue we've seen with dodgy glance configurations, if anyone's bored https://review.openstack.org/#/c/526329 | 16:45 |
| *** lyan has quit IRC | 16:46 | |
| *** gszasz has quit IRC | 16:46 | |
| finucannot | mriedem: Do we still support lxc? | 16:46 |
| *** lyan has joined #openstack-nova | 16:47 | |
| mriedem | finucannot: where does https://docs.openstack.org/python-glanceclient/latest/reference/api/glanceclient.v2.images.html say something about 0 data? | 16:47 |
| finucannot | mriedem: https://docs.openstack.org/python-glanceclient/latest/reference/api/glanceclient.v2.images.html#glanceclient.v2.images.Controller.data | 16:47 |
| finucannot | I trimmed the URL because I didn't want to pollute everyone's git-log | 16:48 |
| finucannot | (more than I'd already done) | 16:48 |
| finucannot | Ha! python-glanceclient is broken itself | 16:54 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Change RPC for select_destinations() https://review.openstack.org/516707 | 16:56 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Move the claim_resources method to scheduler utils https://review.openstack.org/511357 | 16:56 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Make conductor pass and use host_lists https://review.openstack.org/511358 | 16:56 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Change compute RPC to use alternates for resize https://review.openstack.org/526436 | 16:56 |
| *** tidwellr has joined #openstack-nova | 16:57 | |
| *** fragatina has quit IRC | 16:57 | |
| *** david-lyle has quit IRC | 16:57 | |
| *** liusheng has quit IRC | 16:57 | |
| *** liusheng has joined #openstack-nova | 16:57 | |
| *** baoli_ has joined #openstack-nova | 16:59 | |
| *** fragatina has joined #openstack-nova | 17:00 | |
| *** itlinux has quit IRC | 17:00 | |
| *** andreas_s has quit IRC | 17:00 | |
| *** chyka has joined #openstack-nova | 17:00 | |
| *** andreas_s has joined #openstack-nova | 17:00 | |
| *** baoli has quit IRC | 17:01 | |
| *** amodi has joined #openstack-nova | 17:02 | |
| *** tbachman has joined #openstack-nova | 17:03 | |
| melwitt | mriedem: I noticed you abandoned these https://review.openstack.org/#/c/508163 did we not want to keep them and not skip that test now that test_volume_backed_live_migration works? to guard against regression? | 17:04 |
| mriedem | fewer double negatives please | 17:04 |
| *** fragatina has quit IRC | 17:05 | |
| *** itlinux has joined #openstack-nova | 17:05 | |
| melwitt | sorry. like, we were using those DNM patches to test a fix for volume backed live migration | 17:05 |
| *** damien_r has quit IRC | 17:05 | |
| *** andreas_s has quit IRC | 17:05 | |
| melwitt | now that volume backed live migration is fixed, shouldn't we have removed DNM from those to unskip the test? | 17:05 |
| melwitt | don't we want to unskip the test now that it will pass, that is | 17:07 |
| *** felipemonteiro_ has joined #openstack-nova | 17:07 | |
| mriedem | now that volume backed live migration is fixed.... | 17:07 |
| mriedem | it is? | 17:07 |
| mriedem | you mean https://review.openstack.org/#/c/507202/ ? | 17:08 |
| melwitt | yeah, I thought that fixed it. you had those DNM patches depend on it and the unskipped tests passed | 17:08 |
| melwitt | did I miss something? | 17:08 |
| mriedem | melwitt: i believe those tests will intermittently pass regardless of that patch | 17:08 |
| melwitt | okay, that's what I was missing then | 17:09 |
| mriedem | the volume-backed and iscsi live migration tests were skipped in tempest long ago due to random failures | 17:09 |
| mriedem | that no one could sleuth because qemu | 17:09 |
| mriedem | using UCA now might have changed that | 17:09 |
| melwitt | I see | 17:09 |
| mriedem | bug 1524898 | 17:09 |
| openstack | bug 1524898 in OpenStack Compute (nova) "Volume based live migration aborted unexpectedly" [High,In progress] https://launchpad.net/bugs/1524898 | 17:09 |
| melwitt | okay, I thought maybe that was related to the recent fix we had | 17:10 |
| mriedem | even if we enable the test in tempest again, | 17:10 |
| mriedem | tempest is branchless so it would run on stable branches and could fail there if we're using older packages that had a problem | 17:10 |
| mriedem | hard to tell | 17:10 |
| mriedem | i would very much like to get volume-backed live migration testing enabled in the gate though yes | 17:10 |
| melwitt | oh, right. | 17:10 |
| mriedem | we should probably change the global skip on the test in tempest to a config option, | 17:11 |
| mriedem | so that we can enable the test per-branch | 17:11 |
| melwitt | I can't recall if we skip certain tests for stable-only, or if that's even possible | 17:11 |
| mriedem | we can configure tempest per-branch, per-job | 17:11 |
| *** felipemonteiro__ has quit IRC | 17:11 | |
| mriedem | we can configure tempest in our in-tree live migration scripts | 17:11 |
| mriedem | the post_test_hook | 17:11 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/tests/live_migration/hooks/run_tests.sh#L33 | 17:12 |
| mriedem | with the magic of ansible | 17:12 |
| *** derekh has quit IRC | 17:12 | |
| *** lucasagomes is now known as lucas-afk | 17:12 | |
| melwitt | okay, so we'd add another option volume_backed_live_migration or such | 17:13 |
| mriedem | so if you wanted to be cool, propose a change to tempest to add a config option for volume-backed live migration to replace that skip line, default to false for stable, and then set it to true in our live migration job post_test_hook script | 17:13 |
| mriedem | and then be available when it melts the gate | 17:13 |
| *** lyan has quit IRC | 17:14 | |
| melwitt | can do | 17:14 |
| *** gyee has joined #openstack-nova | 17:14 | |
| melwitt | thanks | 17:14 |
| mriedem | it is my pleasure | 17:16 |
| mriedem | that's a weird phrase | 17:16 |
| mriedem | dmsimard: interesting, even when it appears that we do set force_tcg mode, guestfs returns None rather than an int | 17:18 |
| mriedem | http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-full-centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_13_43_19_923098 | 17:18 |
| melwitt | lots of zvm:recheck spam going on | 17:18 |
| mriedem | Failed to force guestfs TCG mode. guestfs_set_backend_settings returned: None | 17:18 |
| dmsimard | weird | 17:18 |
| dmsimard | mriedem: it seems like there's two set_backend_settings happening there | 17:19 |
| *** karthiks has quit IRC | 17:19 | |
| dmsimard | event=trace eh=0 buf='set_backend_settings "force_tcg"' array=[] | 17:19 |
| dmsimard | and event=trace eh=0 buf='set_backend_settings = 0' array=[] ? | 17:20 |
| *** baoli_ has quit IRC | 17:20 | |
| mriedem | i think one is the value being set? | 17:21 |
| mriedem | or the return value? | 17:21 |
| mriedem | i don't know what the array is | 17:21 |
| mriedem | but like "add_drive = 0" | 17:21 |
| *** lyan has joined #openstack-nova | 17:24 | |
| mriedem | http://libguestfs.org/guestfs.3.html#guestfs_set_event_callback | 17:24 |
| *** dtantsur is now known as dtantsur|afk | 17:24 | |
| *** mdnadeem has quit IRC | 17:26 | |
| *** baoli has joined #openstack-nova | 17:27 | |
| *** mlavalle has joined #openstack-nova | 17:28 | |
| *** tesseract has quit IRC | 17:29 | |
| *** lpetrut has quit IRC | 17:30 | |
| *** tidwellr has quit IRC | 17:30 | |
| mriedem | ok this is where we have the eventlet switch that blows up http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-full-centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_13_43_19_941869 | 17:33 |
| mriedem | that's right in the middle of the guestfs.launch call | 17:33 |
| mriedem | error: cannot switch to a different thread | 17:33 |
| mriedem | heh http://logs.openstack.org/50/524750/1/check/legacy-tempest-dsvm-neutron-full-centos-7/a7f051e/logs/screen-n-cpu.txt.gz#_Dec_04_15_25_29_320753 | 17:34 |
| mriedem | so, i think the logging is what's maybe making it switch | 17:34 |
| mriedem | so you enable guestfs debug to get callbacks from guestfs to log them, | 17:35 |
| mriedem | meanwhile, guestfs is running in a thread | 17:35 |
| *** david-lyle has joined #openstack-nova | 17:36 | |
| mriedem | that doesn't explain why we were hanging before we enabled debug though | 17:36 |
| mriedem | but i can push a patch to only launch in a thread if guestfs.debug is False | 17:36 |
| fried_rice | cdent You going to be around in a couple hours? For a "warm handoff"? | 17:38 |
| mriedem | gross | 17:38 |
| * cdent shivers | 17:38 | |
| cdent | fried_rice: yessir, I should be around, or at least nearby, make a ping and I’ll (eventually) come running | 17:39 |
| fried_rice | cdent ack | 17:40 |
| fried_rice | cdent Leaving a bit more of a mess than anticipated, I'm afraid. But nothing insurmountable. | 17:40 |
| *** fried_rice is now known as fried_rolls | 17:41 | |
| *** fragatina has joined #openstack-nova | 17:43 | |
| *** smatzek has joined #openstack-nova | 17:47 | |
| *** gbarros has joined #openstack-nova | 17:47 | |
| mnaser | is there anyone here who worked on glance image signature verification? | 17:49 |
| mnaser | we have a change in puppet-nova to add the ability to configure it, but i'd like to know if its a global nova config (aka, api/etc need it) or nova-compute only | 17:50 |
| mnaser | from my understanding of code, the option seems to be used in the glance image download code which probably only happens on computes? | 17:50 |
| *** salv-orlando has joined #openstack-nova | 17:50 | |
| *** claudiub has joined #openstack-nova | 17:51 | |
| *** tbachman has quit IRC | 17:52 | |
| mriedem | peter-hamilton: ^ | 17:52 |
| mriedem | mnaser: dane-fichter is who i'd usually go to but haven't seem him in channel in a long time | 17:52 |
| alaski | mriedem: in case this didn't already get answered, instance actions are not synced between a cell and the api level in v1. There are spurious records at the API level that never get used, but it would have taken a lot of effort to not write them there. Requests for instance actions are proxied to the cell and results are returned from there. | 17:52 |
| mnaser | mriedem: ok cool, thanks for the info.. i'll research the code a tad bit more | 17:52 |
| mriedem | alaski: hmm, i'm not sure how @wrap_instance_event in the compute service would ever work then | 17:53 |
| mriedem | alaski: because the api creates the action record | 17:53 |
| mriedem | and the compute tries to look it up to record events against it | 17:53 |
| mriedem | but if the action isn't in the cell db, how are they ever found? | 17:53 |
| superdan | mriedem: they get replayed in the cell I imagine | 17:53 |
| mriedem | superdan: where/how? | 17:53 |
| superdan | so they're down there, but the api never looks at them, only at the stale ones that are left by the start | 17:53 |
| superdan | mriedem: because we replay calls into compute/api down in the cell | 17:54 |
| superdan | which would create them there too | 17:54 |
| mriedem | so if a compute service fails to find an action created in the api, it could just be because of a timing issue? | 17:54 |
| *** peter-hamilton has quit IRC | 17:54 | |
| superdan | no, I don't think so, | 17:54 |
| mriedem | ok, was going to say - i find that hard to believe | 17:55 |
| mriedem | because we'd be randomly failing all the time | 17:55 |
| superdan | because it should still get created in the cell before you kick off anything | 17:55 |
| mriedem | ok that doesn't seem to be happening | 17:56 |
| mriedem | i dug through the cellsv1 code for awhile yesterday and don't see anything special about instance actions in there thoguh | 17:56 |
| mriedem | *though, like how they are handled | 17:56 |
| superdan | right | 17:56 |
| superdan | because they're not | 17:56 |
| superdan | mriedem: for example, this one: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3098-L3099 | 17:57 |
| superdan | mriedem: we run that in the api node as part of the cells_api indirection | 17:57 |
| openstackgerrit | Merged openstack/nova-specs master: Document Queens review priorities https://review.openstack.org/526573 | 17:57 |
| superdan | mriedem: then we pass it down to the cell and run that same code again in the cell, | 17:57 |
| superdan | then we call to compute, which would finish it, but only in the cell | 17:57 |
| superdan | and then you're done | 17:58 |
| mriedem | and that's all synchronous? | 17:58 |
| superdan | if you go look at it from the api, you'd only see the start, because you never finished it in the api cell, and nothing ever sync'd the finish | 17:58 |
| superdan | mriedem: is what synchronous? the api cell before the child cell? sure, but they don't know about each other really anyway, so it doesn't matter | 17:58 |
| *** tbachman has joined #openstack-nova | 17:59 | |
| mriedem | is the replay of the instance action create from the top level to the child cell synchronous | 17:59 |
| mriedem | was my question | 18:00 |
| mriedem | trying to figure out what is special that makes this fail the cellsv1 job https://review.openstack.org/#/c/523676 | 18:00 |
| mriedem | http://logs.openstack.org/76/523676/11/check/legacy-tempest-dsvm-cells/1d155c6/logs/screen-n-cpu.txt.gz?level=TRACE#_Dec_07_09_02_09_579991 | 18:00 |
| superdan | the thing that is replayed is the call to compute/api::confirm_resize() | 18:00 |
| superdan | the action_create is | 18:00 |
| superdan | not replayed itself, | 18:00 |
| superdan | the whole call to that method is | 18:00 |
| *** salv-orlando has quit IRC | 18:01 | |
| mriedem | ok, so backup_instance would all be replayed in the child cell api too right? | 18:01 |
| *** tidwellr has joined #openstack-nova | 18:01 | |
| superdan | yeah | 18:01 |
| superdan | I don't see where the cells job ran on that | 18:01 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2594 | 18:02 |
| mriedem | the latest patch timed out in on the cells job | 18:02 |
| superdan | okay | 18:02 |
| superdan | would have thought that would be reported | 18:02 |
| mriedem | zuul in the last 48 hours... | 18:02 |
| *** yamamoto has quit IRC | 18:02 | |
| superdan | oh I see, it is i the comment | 18:02 |
| superdan | but not the summary | 18:02 |
| *** openstackgerrit has quit IRC | 18:03 | |
| alaski | stepped away, but it's exactly as superdan describes | 18:03 |
| superdan | \o/ | 18:03 |
| mriedem | so i wonder if it has something to do with this comment from comstud https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2594 | 18:04 |
| mriedem | but i think that is just about setting the task_state | 18:04 |
| superdan | mriedem: that's a remote call, so maybe it's calling to conductor which is looking in the wrong db? | 18:04 |
| superdan | er, a remote exception I mean | 18:05 |
| mriedem | well, | 18:06 |
| superdan | not sure that makes sense actually | 18:06 |
| mriedem | i don't know why it would just fail on that one lookup and not all of the others | 18:06 |
| mriedem | that's what i don't understand about this | 18:06 |
| mriedem | fried_rolls: have you ever mocked something that can't be imported? | 18:06 |
| superdan | oh Iknow why | 18:07 |
| superdan | mriedem: backup_instance doesn't get replayed in the cell for some reason, unlike everything else, which means it calls straight to the compute I think: https://github.com/openstack/nova/blob/master/nova/cells/messaging.py#L915-L925 | 18:07 |
| superdan | mriedem: compare that to rebuild right below | 18:07 |
| mriedem | ok yeah that's what i was looking at | 18:08 |
| mriedem | https://github.com/openstack/nova/commit/2be50f0f3f02830798c25442dc54e92faa18106b | 18:08 |
| superdan | I'd be extremely concerned about changing _any_ of that stuff at this point | 18:09 |
| mriedem | weird, ok, so i guess the note about "any changes made here have to be made there" is what that's about | 18:09 |
| mriedem | including creating the action record | 18:10 |
| mriedem | well i see 2 options | 18:10 |
| mriedem | 1. do the action create for backup and snapshot in the cells messaging code like it says, | 18:10 |
| mriedem | 2. do something in the EventReporter where if cells is enabled, we blacklist some actions | 18:10 |
| mriedem | we could remove #2 when we remove cellsv1 | 18:10 |
| *** peter-hamilton has joined #openstack-nova | 18:11 | |
| superdan | I don't love either of those | 18:11 |
| peter-hamilton | mriedem: thanks! | 18:11 |
| mriedem | superdan: me neither | 18:11 |
| peter-hamilton | mnaser: bpoulos is who you want to talk to; I'll ping her for you | 18:11 |
| mriedem | superdan: but i'm not sure what else to do at this point | 18:12 |
| mnaser | peter-hamilton: cool, thanks! | 18:12 |
| mriedem | so, i'll go to lunch and not think about this | 18:12 |
| *** r-daneel has joined #openstack-nova | 18:15 | |
| *** yangyape_ has quit IRC | 18:16 | |
| *** sdague has quit IRC | 18:16 | |
| *** yangyapeng has joined #openstack-nova | 18:16 | |
| *** yamamoto has joined #openstack-nova | 18:16 | |
| *** openstackgerrit has joined #openstack-nova | 18:17 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Don't launch guestfs in a thread pool if guestfs.debug is enabled https://review.openstack.org/526748 | 18:17 |
| mriedem | dmsimard: clarkb: ^ | 18:17 |
| *** tidwellr has quit IRC | 18:19 | |
| *** david-lyle has quit IRC | 18:21 | |
| dmsimard | mriedem: so we toggled the bug by setting it to debug ? :/ | 18:21 |
| *** tssurya has joined #openstack-nova | 18:21 | |
| *** yamamoto has quit IRC | 18:21 | |
| dmsimard | mriedem: the force_tcg patch would work without debug ? | 18:21 |
| *** bpoulos has joined #openstack-nova | 18:23 | |
| bpoulos | mnaser: only nova-compute needs the verify_glance_signatures option set in order to enable glance image signature verification | 18:25 |
| mnaser | bpoulos: awesome, thank you so much (fyi, asking in context of https://review.openstack.org/#/c/526514/ ) | 18:26 |
| bpoulos | mnaser: no problem, happy to help | 18:27 |
| *** yamamoto has joined #openstack-nova | 18:32 | |
| *** penick has joined #openstack-nova | 18:35 | |
| *** yamamoto has quit IRC | 18:36 | |
| *** bpoulos has quit IRC | 18:39 | |
| *** sdague has joined #openstack-nova | 18:40 | |
| *** yamamoto has joined #openstack-nova | 18:44 | |
| *** yamamoto has quit IRC | 18:44 | |
| *** yamamoto has joined #openstack-nova | 18:44 | |
| *** yamamoto has quit IRC | 18:45 | |
| *** penick has quit IRC | 18:55 | |
| openstackgerrit | Jay Pipes proposed openstack/nova master: allow compute nodes to be associated with host agg https://review.openstack.org/526753 | 18:56 |
| *** penick has joined #openstack-nova | 18:58 | |
| *** Guest99196 has quit IRC | 19:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/524795 | 19:10 |
| *** pramodrj07 has quit IRC | 19:13 | |
| *** MasterOfBugs has quit IRC | 19:13 | |
| *** BryanS68 has quit IRC | 19:13 | |
| *** BryanS68 has joined #openstack-nova | 19:14 | |
| openstackgerrit | Merged openstack/nova master: Updated from global requirements https://review.openstack.org/526386 | 19:16 |
| *** harlowja has joined #openstack-nova | 19:21 | |
| *** peter-hamilton has quit IRC | 19:23 | |
| *** moshele has joined #openstack-nova | 19:24 | |
| *** tidwellr has joined #openstack-nova | 19:25 | |
| openstackgerrit | Merged openstack/nova master: Fix 'force' parameter in os-quota-sets PUT schema https://review.openstack.org/523194 | 19:27 |
| *** MasterOfBugs has joined #openstack-nova | 19:27 | |
| *** pramodrj07 has joined #openstack-nova | 19:27 | |
| *** tssurya has quit IRC | 19:31 | |
| *** gbarros has quit IRC | 19:34 | |
| *** moshele has quit IRC | 19:36 | |
| *** moshele has joined #openstack-nova | 19:37 | |
| *** AlexeyAbashkin has joined #openstack-nova | 19:46 | |
| *** yamamoto has joined #openstack-nova | 19:46 | |
| mriedem | dmsimard: that's what i hope to find out | 19:47 |
| fried_rolls | mriedem No, I don't think I have ever (successfully) mocked something that can't be imported. It will fail at "compile" time. | 19:50 |
| fried_rolls | mriedem You can cheat by mocking it after it's imported, though. | 19:50 |
| fried_rolls | I assume this is in a situation where the thing imports correctly on the prod side but for some reason not on the test side? Which... why? | 19:52 |
| *** yamamoto has quit IRC | 19:52 | |
| *** fried_rolls is now known as fried_rice | 19:53 | |
| fried_rice | cdent How's now? | 19:53 |
| cdent | one moment caller | 19:53 |
| cdent | blargh, timeouts in migration tests: http://logs.openstack.org/39/521639/8/gate/openstack-tox-functional/f6880ac/testr_results.html.gz | 19:54 |
| *** AlexeyAbashkin has quit IRC | 19:56 | |
| *** tssurya has joined #openstack-nova | 19:57 | |
| *** tssurya has quit IRC | 19:58 | |
| cdent | fried_rice: okay, sorry, had to feed that cat and put dinner dishes aweay | 20:02 |
| fried_rice | How domestic. | 20:03 |
| fried_rice | Should we do this here, or maybe snag an empty meeting channel so we don't consume this one? | 20:03 |
| figleaf | stay here - I'm eavesdropping | 20:04 |
| fried_rice | ight | 20:04 |
| cdent | agree with figleaf, probably best to spread the word as much as possible | 20:04 |
| *** gouthamr has joined #openstack-nova | 20:04 | |
| *** jmlowe has quit IRC | 20:04 | |
| fried_rice | I'll lead by saying I've got several balls in the air, including the 'granular' effort and a non-placement-related bp for ksa adapter stuff; but I'll be happy if just the nrp series gets shepherded while I'm away. | 20:05 |
| fried_rice | That's the series starting at the hopefully-just-about-to-merge https://review.openstack.org/#/c/385693/91 | 20:05 |
| fried_rice | The culmination of this series will be the ability for the compute driver to tell the scheduler (almost) everything about how it wants to perceive its world, placement-wise. | 20:06 |
| cdent | “the scheduler”? or “placment” or both? | 20:07 |
| fried_rice | As currently conceived in the top patch in the series (https://review.openstack.org/#/c/520246/12/nova/compute/resource_tracker.py) the resource tracker will hand off the ProviderTree to the compute driver; the compute driver will mangle it as it sees fit; and the resource tracker will flush any changes back to placement. | 20:07 |
| *** lpetrut has joined #openstack-nova | 20:07 | |
| fried_rice | cdent I should have said "resource tracker" instead of "scheduler". The main point there is that the compute driver does not directly tell placement anything; that is brokered by resource tracker via ProviderTree. | 20:08 |
| fried_rice | ...via report client. | 20:08 |
| *** penick has quit IRC | 20:08 | |
| *** BryanS68 has quit IRC | 20:08 | |
| cdent | okay, so the virt driver tells the compute manager and its resource tracker about what it knows about its world | 20:08 |
| fried_rice | Yup. | 20:09 |
| cdent | ✔ | 20:09 |
| fried_rice | I think there are reasons we don't want virt talking to placement directly. leakypipes and superdan have those in their heads. But I think it has to do with narrowing the points of control. | 20:09 |
| leakypipes | fried_rice: ya | 20:10 |
| *** jmlowe has joined #openstack-nova | 20:10 | |
| leakypipes | fried_rice: the scheduler reporting client is a good bright-line interface for nova-compute with the placement API. | 20:10 |
| fried_rice | Nod. | 20:11 |
| cdent | yeah, my concern was not so much about violating that line, rather trying to clarify the use of the term scheduler, which is ambiguous, but it’s clear now | 20:11 |
| fried_rice | As currently written in the series, the first time around, resource tracker => report client will do get_provider_tree to bootstrap the ProviderTree, which will _ensure_resource_provider with the compute nodename & uuid. That will create the entry in placement for the compute host RP and populate the local ProviderTree with that. | 20:11 |
| fried_rice | And then pass that ProviderTree to virt, via ComputeDriver.update_provider_tree. | 20:12 |
| fried_rice | So it'll have just the compute RP as a root (no children), no inventory, no traits, no aggs, no sharing RPs. | 20:12 |
| *** BryanS68 has joined #openstack-nova | 20:13 | |
| cdent | ✔ | 20:13 |
| fried_rice | virt does whatever it wants, as long as it's possible via the public ProviderTree interfaces. This includes creating new root RPs (which will probably typically be sharing RPs, though we're not enforcing that) or child RPs (usually descendants of the compute RP, though we're not enforcing that). | 20:14 |
| fried_rice | It also includes setting traits and inventories on any of those guys. | 20:14 |
| fried_rice | And assigning aggregate UUIDs. | 20:14 |
| *** sbezverk has joined #openstack-nova | 20:14 | |
| fried_rice | virt is responsible for returning True or False from that method to let us know whether it changed anything. (TBH, I'm not sure that's necessary, cause it's pretty easy for us to figure it out, and that might be better than trusting virt to get it right.) | 20:15 |
| fried_rice | Anyway, resource tracker gets that provider tree back and diffs it with what it passed in originally (which, I should have mentioned, is a copy of what it had built up); and then makes report client calls to push those changes back to the placement db. | 20:16 |
| *** david-lyle has joined #openstack-nova | 20:16 | |
| cdent | can I pause you for a moment? | 20:16 |
| fried_rice | sho | 20:16 |
| *** ChanServ has quit IRC | 20:17 | |
| *** lpetrut has quit IRC | 20:18 | |
| cdent | this is making sense, mostly, so far, except for one thing. Can you give me a scenario or two in which the provider tree will have more than one root? Is this for something like ironic, of clustering hypervisor or something else or something in addition to that? | 20:18 |
| *** tssurya has joined #openstack-nova | 20:19 | |
| fried_rice | I can't say what'll happen for ironic. It's possible they'll have their root be their (I'm going to get the terminology wrong, but) compute "host" with children being the compute "nodes". Or maybe the compute "nodes" will all be roots. I dunno. | 20:19 |
| fried_rice | A real scenario for multiple roots is sharing RPs. | 20:20 |
| *** tssurya has quit IRC | 20:20 | |
| fried_rice | So if I have a shared disk provider that my virt driver knows about, the compute manager won't know about it yet. | 20:20 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add instance action record for backup instances https://review.openstack.org/523676 | 20:20 |
| *** tssurya has joined #openstack-nova | 20:21 | |
| fried_rice | virt will populate the provider tree with that shared disk provider as a new root with the MISC_SHARES_VIA_AGGREGATE trait and tag both it and the compute RP with the aggregate UUID. | 20:21 |
| cdent | let me know when you are EOF, so I can ask my followup, but I don’t want to do it if you are stil typing away | 20:21 |
| fried_rice | And the rt will have to handle the case where we go to create that guy in placement and find it already exists, because some other compute node that *also* shares with it already created it. | 20:21 |
| fried_rice | By the same token, the various virts sharing a given sharing RP will have to make sure they use the same name and/or UUID (and we should probably document some rules for them doing that) so that we can in fact do that coordination. | 20:22 |
| *** penick has joined #openstack-nova | 20:22 | |
| fried_rice | We've been saying we won't support sharing in Q, but we at least need to consider this stuff so we don't paint ourselves into a corner. | 20:22 |
| fried_rice | EOF | 20:22 |
| *** huanxie has quit IRC | 20:24 | |
| *** ChanServ has joined #openstack-nova | 20:24 | |
| *** barjavel.freenode.net sets mode: +o ChanServ | 20:24 | |
| cdent | Hmm, okay. I see there are benefits to that model for sharing, especially for virtdrivers from our respective employers, but it surprises me a bit. I was imagining that a compute manager would only ever be responsble for managing stuff that it can be said to own, and ownership in this case is weird because of the -shared- aspect of things. over | 20:25 |
| *** penick has quit IRC | 20:25 | |
| *** sahid has quit IRC | 20:26 | |
| fried_rice | I think the flexibility of this model is good here, because it allows for some third party to manage the sharing RPs, but if such a third party doesn't exist for a given driver, it also allows for it all to be coordinated by the virt driver. | 20:26 |
| *** huanxie has joined #openstack-nova | 20:26 | |
| fried_rice | I can speak for PowerVM, when it comes to our shared storage pool impl, that there's no third party process that's going to be able to do that stuff (unless it's at the behest of the virt driver itself under the covers, which would pretty much violate the "virt driver not talking to placement" thing). | 20:27 |
| fried_rice | over | 20:27 |
| fried_rice | or I can proceed with brain dump if we don't need to discuss this aspect further at the moment. | 20:28 |
| *** sridharg has quit IRC | 20:28 | |
| *** sbezverk has quit IRC | 20:29 | |
| cdent | I’ll just make one more comment, as I agree that the flexibility is appropriate given constraints, and then we can move on: I’d rather see the virt drivers have license to talk to placement, especially since we can’t (and shouldn’t want) to control that anyway. All you need is the right keystone token. carry on my wayward son | 20:30 |
| fried_rice | You'd have to take that up with not-me, for sure. | 20:30 |
| * cdent nods | 20:31 | |
| fried_rice | To the point of whether virt can create roots, we could document (and even enforce) what virt drivers are and are not allowed to do to the ProviderTree, if we wanted to. But note that the code that lets you create a new root in a ProviderTree is already merged :) | 20:31 |
| *** sbezverk has joined #openstack-nova | 20:31 | |
| fried_rice | So anyway, I pretty much started with that, wrote that patch first, and then worked my way backwards to all the plumbing we need underneath it to make it workable. | 20:32 |
| *** david-lyle has quit IRC | 20:32 | |
| * cdent nods | 20:32 | |
| fried_rice | It's in the neighborhood of 16 patches at the moment. I think we only need one more :) | 20:32 |
| fried_rice | Oh, two. | 20:33 |
| fried_rice | So here's where I go into my "state of the series" address. | 20:33 |
| * cdent sits comfortably | 20:34 | |
| fried_rice | The bottom-most WIP was only still WIP because I wanted to add some more meat to the functional test: https://review.openstack.org/#/c/521098/10/nova/tests/functional/api/openstack/placement/test_report_client.py@332 | 20:34 |
| fried_rice | That's a nice big functional test that creates a bunch of interconnected providers such as a (fairly complex) virt driver might, including inventory. See ascii diagram at the top. | 20:35 |
| * cdent nods | 20:36 | |
| fried_rice | The crux is then where we call get_provider_tree, and make sure what we get back has all the stuff in it. | 20:36 |
| fried_rice | BTW, the above story only talked about the first time we call into this stuff; the general case whenever we do update compute node (or whatever it's called in the periodic task) will be the opportunity for virt to tell us what, if anything, has changed in the provider world. It can shuffle inventories, aggregates, etc. | 20:37 |
| * cdent nods | 20:37 | |
| fried_rice | One would hope such changes to be rare to the point of almost-never, because virt doesn't manage allocations. | 20:37 |
| fried_rice | So we're talking about if someone hotplugs storage or CoDs a bank of CPUs or something. | 20:37 |
| fried_rice | anyway... | 20:37 |
| cdent | (if virt could manage allocations, that would be cooler, for hybrid environments) | 20:38 |
| fried_rice | As I was adding the code for that TODO at the bottom, I found that the traits were in fact not showing up for certain providers. | 20:38 |
| * cdent nods | 20:38 | |
| fried_rice | Which turns out to be because the traits PUT is 400ing. | 20:38 |
| *** Apoorva has joined #openstack-nova | 20:38 | |
| *** Apoorva has quit IRC | 20:38 | |
| cdent | but nobody can tell because of None? | 20:39 |
| fried_rice | No | 20:39 |
| * cdent shucks | 20:39 | |
| fried_rice | Because we don't have a report client method for creating traits. | 20:39 |
| *** Apoorva has joined #openstack-nova | 20:39 | |
| fried_rice | or aggregates | 20:39 |
| *** penick has joined #openstack-nova | 20:39 | |
| fried_rice | so I'm doing that directly in the test, and not checking the result. | 20:39 |
| * cdent nods | 20:39 | |
| fried_rice | So that's one (or maybe two) still-needed patch(es). | 20:39 |
| fried_rice | While we're talking about None, there's another patch (possibly more than one) needed, which could actually be done outside of this series, to make sure we're raising where we should. | 20:40 |
| fried_rice | Per discussion yesterday, we only need to do it for methods where we don't always check the return | 20:41 |
| fried_rice | I identified that at least the one that gets inventories qualifies. | 20:41 |
| fried_rice | There may be others. | 20:41 |
| *** penick has quit IRC | 20:42 | |
| * cdent nods | 20:42 | |
| *** penick has joined #openstack-nova | 20:42 | |
| fried_rice | Now, I think I identified a change that's needed to what's already in gerrit for the get_provider_tree method to make sure we're pulling the latest traits & aggs for all "associated" providers to populate the ProviderTree. But I won't know until I figure out & fix whatever that PUT /traits deal is. | 20:43 |
| fried_rice | (not PUT /traits -- PUT /resource_provider/%u/traits or whatever - you get the idea) | 20:43 |
| * cdent nods | 20:43 | |
| fried_rice | Okay, next to-do: The report client method that diffs what virt driver returns and pushes those changes back to placement. It'll go between the top two patches in the series, I think. Working title for that method is update_from_provider_tree, per https://review.openstack.org/#/c/520246/12/nova/compute/resource_tracker.py@839 | 20:45 |
| *** penick has quit IRC | 20:45 | |
| *** dgonzalez has left #openstack-nova | 20:46 | |
| fried_rice | The last major thing that affects the series is how we handle generations, which we started talking about some this morning. | 20:46 |
| *** yamamoto has joined #openstack-nova | 20:48 | |
| fried_rice | Honestly, I would need to relook at pretty much the whole report client with that in mind to figure out what we're gonna need to do about it. | 20:48 |
| fried_rice | Absent the proposed get-everything-about-a-provider API (https://review.openstack.org/#/c/521875/) we would have to do stuff like: | 20:49 |
| cdent | do you have a summary statement of the the “generations problem”? | 20:49 |
| fried_rice | when we GET a provider's traits, that response payload contains the RP generation as known to the server. If that doesn't match what we have in the local ProviderTree, we should force-refresh everything we know about that provider. | 20:50 |
| cdent | or more generally: any time we get a generation in a response, if it is not a match to us, refresh? | 20:51 |
| fried_rice | yeah, I think that sums it up at least to the 80/20 point :) | 20:52 |
| *** yamamoto has quit IRC | 20:52 | |
| fried_rice | At the moment, my code that retrieves traits & aggs is ignoring the rp generation field in the response. | 20:52 |
| *** nore_rabel has joined #openstack-nova | 20:53 | |
| * cdent nods | 20:53 | |
| fried_rice | In particular, what it's *not* doing is calling the set_*() method with the generation from that response, which would actually be the *wrong* thing to do I think. | 20:54 |
| fried_rice | Because that would be updating the generation based on thing A that changed without taking into account that thing B (whose content we still have at the old generation level) might have also changed. | 20:54 |
| * fried_rice takes notes for Dr. Seuss-esque book) | 20:55 | |
| *** itlinux has quit IRC | 20:55 | |
| cdent | quite | 20:55 |
| fried_rice | Beyond all that, while restacking, there are some nice review comments to address from finucannot, giblet, figleaf, takashin... | 20:56 |
| * cdent nods | 20:57 | |
| *** penick has joined #openstack-nova | 20:57 | |
| *** ttx has quit IRC | 20:57 | |
| *** ttx has joined #openstack-nova | 20:57 | |
| cdent | If you had to identify the scariest-here-be-dragons aspect, what would it be? | 20:57 |
| fried_rice | And writing up an executive summary, as you suggested in your email today, which could largely comprise text from the foregoing. | 20:57 |
| * figleaf doesn't feel his comments were all that nice | 20:57 | |
| fried_rice | Scariest as in potential for largest amount of rework of stuff that's not already marked WIP? The generation thing. | 20:58 |
| *** itlinux has joined #openstack-nova | 20:58 | |
| cdent | k | 20:59 |
| cdent | Let me see if I can sort of translate some things to do: | 21:00 |
| cdent | a) decode the last hour of the log to extract salient links, main points, action items | 21:01 |
| *** moshele has quit IRC | 21:01 | |
| cdent | b) attend to comments from reviews whilst restacking | 21:01 |
| *** tasker has quit IRC | 21:01 | |
| cdent | c) make the linked functional test, notably it’s ascii diagram “work” and make the code do that tree and allow that tree to change, and deal with generation changes | 21:01 |
| *** pchavva has quit IRC | 21:02 | |
| *** sbezverk has quit IRC | 21:02 | |
| cdent | eof | 21:03 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: mask InjectionInfo.admin_pass https://review.openstack.org/526772 | 21:03 |
| cdent | (since c) ended up being sort of d) and e) as well) | 21:03 |
| fried_rice | Pretty much. WRT (c), the only part of the diagram that the code doesn't "do" right is the traits on *some* providers. | 21:04 |
| fried_rice | Which may be solved by resolving that 400, but I actually think there's more to it. I'm leaving some code comments with more info. | 21:04 |
| * cdent nods | 21:04 | |
| *** gouthamr has quit IRC | 21:07 | |
| fried_rice | Done: https://review.openstack.org/#/c/521098/10 | 21:08 |
| cdent | cool | 21:08 |
| *** penick has quit IRC | 21:09 | |
| fried_rice | Okay, I gotta run get the kids. Realistically, I'll be working a few scattered hours over the course of the next week, so drop me an email if you want to set up some more discussion time. | 21:09 |
| cdent | I will endeavor to keep things moving. I suspect once I start breaking it it will make considerably more sense than now and happiness will abound and unicorns will leap and frolic etc | 21:09 |
| figleaf | Any word from HR? | 21:09 |
| figleaf | oops, wrong channel | 21:09 |
| fried_rice | Yeah, don't tell HR I'm working. | 21:09 |
| fried_rice | They're like, "you have HOW much vacation time left? Yeah, you gotta go." | 21:10 |
| figleaf | Don't tell them I'm not :) | 21:10 |
| leakypipes | figleaf: I've reported you to HR. | 21:10 |
| fried_rice | leakypipes Dunno if you followed any/all of that, but would be interested to know whether you concur on the high points. | 21:11 |
| figleaf | leakypipes: aw, c'mon - it was just a little hug | 21:11 |
| leakypipes | figleaf: :) | 21:11 |
| leakypipes | fried_rice: I was not paying attention, no... | 21:11 |
| fried_rice | leakypipes And likewise, shoot me an email if we need to get together for an hour or whatever. | 21:11 |
| cdent | leakypipes: the plan is to replace the scheduler, placement, the compute managers with etcd watches | 21:12 |
| *** fried_rice is now known as efried_cya_jan | 21:12 | |
| leakypipes | cdent: ah, cool. so k8s then. | 21:12 |
| figleaf | leakypipes: running on Cassandra, of course | 21:12 |
| cdent | before christmas | 21:12 |
| * efried_cya_jan waves | 21:12 | |
| figleaf | cya | 21:12 |
| *** penick has joined #openstack-nova | 21:12 | |
| leakypipes | efried_cya_jan: cya! | 21:12 |
| cdent | happy holidays or whathaveyou efried_cya_jan | 21:12 |
| efried_cya_jan | Y'all too | 21:12 |
| * leakypipes just finished up the PostgreSQL benchmark changes for that UUID vs. integer PK benchmark. | 21:13 | |
| cdent | i assume it is glorious? | 21:13 |
| cdent | actually, instead of etcd, we’ve got new technology that inverts cgroups to create global datacentre-wide share process spaces, and you can make global consensual RAM | 21:14 |
| leakypipes | cdent: very similar to Percona/MySQL | 21:14 |
| leakypipes | cdent: results-wise. | 21:14 |
| leakypipes | cdent: gonna run the full bench tonight while nothing else is running on my puter. | 21:14 |
| leakypipes | cdent: global consensual RAM sounds kinky. | 21:15 |
| cdent | does a bit | 21:16 |
| cdent | I will avoid a google search | 21:16 |
| cdent | my eyes are done | 21:19 |
| cdent | goodnight all, have good weekends | 21:19 |
| *** cdent has quit IRC | 21:19 | |
| *** david-lyle has joined #openstack-nova | 21:24 | |
| *** dave-mccowan has quit IRC | 21:25 | |
| *** smatzek has quit IRC | 21:27 | |
| *** david-lyle has quit IRC | 21:29 | |
| leakypipes | and I'm signing off too... ciao folks. | 21:33 |
| *** leakypipes has quit IRC | 21:33 | |
| *** hui has quit IRC | 21:37 | |
| *** nore_rabel has quit IRC | 21:38 | |
| *** Apoorva has quit IRC | 21:39 | |
| *** rcernin has joined #openstack-nova | 21:44 | |
| *** gbarros has joined #openstack-nova | 21:46 | |
| *** Apoorva has joined #openstack-nova | 21:48 | |
| *** yamamoto has joined #openstack-nova | 21:49 | |
| *** yamamoto has quit IRC | 21:54 | |
| *** edmondsw has joined #openstack-nova | 21:55 | |
| *** edmondsw has quit IRC | 21:56 | |
| *** smatzek has joined #openstack-nova | 21:57 | |
| *** tssurya has quit IRC | 21:57 | |
| *** ihrachys has joined #openstack-nova | 21:58 | |
| ihrachys | what could be the reason that I get this in placement-api (and compute service also spills something like that) | 21:59 |
| ihrachys | Placement API returning an error response: Unable to update inventory for resource provider 2c677702-34b2-4e93-9124-a512bb97230f: Invalid inventory for 'MEMORY_MB' on resource provider '2c677702-34b2-4e93-9124-a512bb97230f'. The reserved value is greater than or equal to total. | 21:59 |
| ihrachys | and the node can't boot instances, returning the usual No hosts found error. I suspect the placement api error is relevant. | 22:00 |
| mriedem | ihrachys: check the reserved_host_memory_mb config option | 22:00 |
| *** itlinux has quit IRC | 22:01 | |
| mriedem | the virt driver is saying there is x amount of total memory but the config is trying to reserve more than that for the host | 22:01 |
| *** damien_r has joined #openstack-nova | 22:01 | |
| *** damien_r has quit IRC | 22:01 | |
| mriedem | there is probably something in the debug logs before that failure in nova-compute where it dumps what it's thinking the values are | 22:01 |
| *** smatzek has quit IRC | 22:01 | |
| *** tidwellr has quit IRC | 22:03 | |
| *** rcernin has quit IRC | 22:03 | |
| *** rcernin has joined #openstack-nova | 22:03 | |
| ihrachys | mriedem, reserved_host_memory_mb = 4096 | 22:04 |
| *** tidwellr has joined #openstack-nova | 22:04 | |
| ihrachys | oh so it tries to lock all this memory? | 22:05 |
| ihrachys | I don't have it on the node, though I added some swap to make it running | 22:05 |
| ihrachys | maybe I should try to reduce that | 22:07 |
| melwitt | yeah it will make it unavailable https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.reserved_host_disk_mb | 22:07 |
| melwitt | sorry that was disk. memory is https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.reserved_host_memory_mb | 22:08 |
| *** tidwellr has quit IRC | 22:08 | |
| *** awaugama has quit IRC | 22:10 | |
| mriedem | ihrachys: this is the method that sets up the inventory data to send to placement https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L83 | 22:10 |
| *** felipemonteiro_ has quit IRC | 22:11 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add API and nova-manage tests that use the NoopQuotaDriver https://review.openstack.org/526270 | 22:11 |
| openstackgerrit | melanie witt proposed openstack/nova master: Follow up on removing old-style quotas code https://review.openstack.org/524234 | 22:11 |
| *** tbachman has quit IRC | 22:12 | |
| ihrachys | mriedem, reduced to 512, restarted, and now it seems to have started the instance as expected. I guess the issue was because swap is not considered for the locking | 22:12 |
| *** gouthamr has joined #openstack-nova | 22:12 | |
| ihrachys | thanks mriedem for the pointer to the option! | 22:12 |
| mriedem | yw | 22:14 |
| *** itlinux has joined #openstack-nova | 22:17 | |
| *** itlinux has quit IRC | 22:18 | |
| openstackgerrit | melanie witt proposed openstack/nova master: DNM: Test websocketproxy with TLS https://review.openstack.org/513160 | 22:21 |
| *** gmann_afk is now known as gmann | 22:21 | |
| *** lyan has quit IRC | 22:21 | |
| *** tssurya has joined #openstack-nova | 22:24 | |
| *** baoli has quit IRC | 22:33 | |
| *** BryanS68 has quit IRC | 22:33 | |
| melwitt | mriedem: I had to rebase the noop quotas test coverage bc of your quota force bug fix shenanigans https://review.openstack.org/#/c/526270 | 22:34 |
| *** lyan has joined #openstack-nova | 22:34 | |
| *** baoli has joined #openstack-nova | 22:34 | |
| *** baoli has quit IRC | 22:35 | |
| *** vladikr has quit IRC | 22:36 | |
| *** amodi has quit IRC | 22:39 | |
| mriedem | done | 22:41 |
| melwitt | thanks \o/ | 22:41 |
| *** lyan has quit IRC | 22:46 | |
| mriedem | superdan: were you going to update https://review.openstack.org/#/c/507638/ or save it for monday or are you ok with me updating it? | 22:49 |
| superdan | mriedem: yeah I started locally but likely won't finish until monday | 22:49 |
| superdan | that dependent one is in the gate atm too | 22:49 |
| mriedem | alright | 22:50 |
| *** yamamoto has joined #openstack-nova | 22:50 | |
| mriedem | well i guess i'll just quit and spend quality time with my family | 22:50 |
| superdan | good call | 22:51 |
| melwitt | superdan: were you gonna remove one of the dupe tests in this "unmap compute nodes" one or should I do it? https://review.openstack.org/#/c/524641 | 22:52 |
| superdan | melwitt: yes I was | 22:52 |
| superdan | so what you're both saying is, I suck, is that about right? | 22:52 |
| melwitt | okay. no, of course not! | 22:52 |
| *** figleaf is now known as edleafe | 22:53 | |
| *** yamamoto has quit IRC | 22:54 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Unmap compute nodes when deleting host mapping https://review.openstack.org/524641 | 22:55 |
| superdan | melwitt: ^ | 22:55 |
| melwitt | saweet | 22:56 |
| melwitt | mriedem: if you're still around ^ | 22:57 |
| superdan | I think you can slam it, he was +2 before right? | 22:59 |
| melwitt | yeah he was. I'm conservative about doing that so thought I'd ask in case he's still there | 23:00 |
| *** huanxie has quit IRC | 23:00 | |
| superdan | well, no pressure from me | 23:01 |
| superdan | I'm spent for today anyway | 23:01 |
| melwitt | ++ | 23:01 |
| *** burt has quit IRC | 23:08 | |
| *** esberglu has quit IRC | 23:10 | |
| *** baoli has joined #openstack-nova | 23:11 | |
| *** imacdonn has quit IRC | 23:14 | |
| *** imacdonn has joined #openstack-nova | 23:14 | |
| *** penick has quit IRC | 23:15 | |
| *** baoli has quit IRC | 23:16 | |
| *** tssurya has quit IRC | 23:21 | |
| *** purplerbot has quit IRC | 23:29 | |
| *** yamahata has joined #openstack-nova | 23:37 | |
| *** tbachman has joined #openstack-nova | 23:38 | |
| *** felipemonteiro has joined #openstack-nova | 23:39 | |
| openstackgerrit | Merged openstack/nova stable/pike: Add regression test for rebuild with new image doubling allocations https://review.openstack.org/523213 | 23:44 |
| *** markvoelker has quit IRC | 23:49 | |
| *** markvoelker has joined #openstack-nova | 23:50 | |
| *** yamamoto has joined #openstack-nova | 23:51 | |
| *** markvoelker has quit IRC | 23:54 | |
| *** yamamoto has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!