| *** kotra03_krk has joined #openstack-nova | 00:01 | |
| *** gyee has quit IRC | 00:02 | |
| *** markvoelker has quit IRC | 00:05 | |
| *** jistr has quit IRC | 00:07 | |
| *** hamzy has joined #openstack-nova | 00:09 | |
| *** jistr has joined #openstack-nova | 00:13 | |
| *** zhuli_ has joined #openstack-nova | 00:15 | |
| *** Nel1x has joined #openstack-nova | 00:17 | |
| *** fragatina has quit IRC | 00:18 | |
| *** liverpooler has joined #openstack-nova | 00:20 | |
| *** tbachman has joined #openstack-nova | 00:29 | |
| *** gouthamr has quit IRC | 00:30 | |
| *** hoangcx has joined #openstack-nova | 00:33 | |
| *** mingyu_ has joined #openstack-nova | 00:34 | |
| *** mingyu has quit IRC | 00:35 | |
| *** gouthamr has joined #openstack-nova | 00:38 | |
| *** suresh12 has joined #openstack-nova | 00:47 | |
| *** liverpooler has quit IRC | 00:52 | |
| *** edmondsw has joined #openstack-nova | 00:53 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add tests for alloc_cands with member_of https://review.openstack.org/561399 | 00:54 |
|---|---|---|
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Fix member_of with sharing providers https://review.openstack.org/561400 | 00:54 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Expand member_of functional test cases https://review.openstack.org/566011 | 00:54 |
| *** damien_r has joined #openstack-nova | 00:55 | |
| *** phuongnh has joined #openstack-nova | 00:58 | |
| *** gouthamr has quit IRC | 00:58 | |
| *** damien_r has quit IRC | 01:00 | |
| mrjazzercise | idlemind: you've got 2 cell1s, so probably drop the one with the wrong URL | 01:05 |
| *** edmondsw has quit IRC | 01:05 | |
| mrjazzercise | and update the cell0 one using nova-manage cell_v2 update_cell | 01:05 |
| *** markvoelker has joined #openstack-nova | 01:05 | |
| mrjazzercise | some services cache the cells so you'll have to restart some services, check the man page on update_cell | 01:06 |
| mrjazzercise | nova-api, nova-conductor and nova-scheduler i think | 01:06 |
| *** tiendc has joined #openstack-nova | 01:12 | |
| *** spsurya has joined #openstack-nova | 01:14 | |
| *** dpawlik has joined #openstack-nova | 01:15 | |
| *** annp has joined #openstack-nova | 01:16 | |
| *** fragatina has joined #openstack-nova | 01:17 | |
| *** edmondsw has joined #openstack-nova | 01:18 | |
| *** dpawlik has quit IRC | 01:20 | |
| *** gjayavelu has quit IRC | 01:21 | |
| *** zhuli_ is now known as zhuli | 01:21 | |
| *** fragatina has quit IRC | 01:21 | |
| *** fragatina has joined #openstack-nova | 01:22 | |
| *** gouthamr has joined #openstack-nova | 01:23 | |
| *** fragatina has quit IRC | 01:26 | |
| *** suresh12 has quit IRC | 01:30 | |
| *** yamahata has quit IRC | 01:30 | |
| *** gouthamr has quit IRC | 01:32 | |
| *** fragatina has joined #openstack-nova | 01:35 | |
| *** mrjazzercise has quit IRC | 01:36 | |
| *** gaoyan has joined #openstack-nova | 01:37 | |
| *** gaoyan has quit IRC | 01:37 | |
| *** markvoelker has quit IRC | 01:40 | |
| *** Tom-Tom has joined #openstack-nova | 01:44 | |
| *** kotra03_krk has quit IRC | 01:46 | |
| *** fragatina has quit IRC | 01:46 | |
| *** fragatina has joined #openstack-nova | 01:47 | |
| *** hongbin has joined #openstack-nova | 01:51 | |
| *** kotra03_krk has joined #openstack-nova | 01:53 | |
| *** gouthamr has joined #openstack-nova | 01:56 | |
| *** zhaochao has joined #openstack-nova | 01:59 | |
| *** LiveOne_ has joined #openstack-nova | 02:00 | |
| *** r-daneel has quit IRC | 02:06 | |
| *** suresh12 has joined #openstack-nova | 02:08 | |
| idlemind | mrjazzercise thx | 02:09 |
| *** suresh12 has quit IRC | 02:12 | |
| *** armaan has joined #openstack-nova | 02:13 | |
| *** armaan has quit IRC | 02:14 | |
| *** hongbin has quit IRC | 02:17 | |
| *** edmondsw has quit IRC | 02:18 | |
| *** lei-zh has joined #openstack-nova | 02:18 | |
| *** rcernin has quit IRC | 02:19 | |
| *** lei-zh has quit IRC | 02:20 | |
| *** rcernin has joined #openstack-nova | 02:20 | |
| *** lei-zh has joined #openstack-nova | 02:20 | |
| *** gouthamr has quit IRC | 02:22 | |
| *** mingyu_ has quit IRC | 02:22 | |
| *** mingyu has joined #openstack-nova | 02:22 | |
| *** gouthamr has joined #openstack-nova | 02:23 | |
| *** fragatina has quit IRC | 02:26 | |
| *** fragatina has joined #openstack-nova | 02:26 | |
| *** gouthamr has quit IRC | 02:27 | |
| *** gouthamr has joined #openstack-nova | 02:32 | |
| *** markvoelker has joined #openstack-nova | 02:37 | |
| *** gouthamr has quit IRC | 02:37 | |
| *** hongbin has joined #openstack-nova | 02:37 | |
| *** germs has quit IRC | 02:39 | |
| *** gjayavelu has joined #openstack-nova | 02:39 | |
| *** germs has joined #openstack-nova | 02:40 | |
| *** psachin has joined #openstack-nova | 02:41 | |
| *** esberglu has quit IRC | 02:47 | |
| *** yamamoto has quit IRC | 02:51 | |
| *** yamamoto has joined #openstack-nova | 02:51 | |
| *** gjayavelu has quit IRC | 02:51 | |
| *** kotra03_krk has quit IRC | 02:54 | |
| *** gjayavelu has joined #openstack-nova | 02:55 | |
| *** damien_r has joined #openstack-nova | 02:55 | |
| *** yamamoto has quit IRC | 02:58 | |
| *** damien_r has quit IRC | 03:00 | |
| *** Nel1x has quit IRC | 03:09 | |
| *** gjayavelu has quit IRC | 03:10 | |
| *** markvoelker has quit IRC | 03:10 | |
| *** dave-mccowan has quit IRC | 03:14 | |
| *** dpawlik has joined #openstack-nova | 03:16 | |
| *** dpawlik has quit IRC | 03:21 | |
| *** Tom-Tom has quit IRC | 03:26 | |
| *** fragatina has quit IRC | 03:29 | |
| *** Tom-Tom has joined #openstack-nova | 03:29 | |
| *** fragatina has joined #openstack-nova | 03:29 | |
| *** andreas_s has joined #openstack-nova | 03:30 | |
| *** andreas_s has quit IRC | 03:34 | |
| *** fragatina has quit IRC | 03:35 | |
| *** lei-zh has quit IRC | 03:36 | |
| *** felipemonteiro__ has joined #openstack-nova | 03:36 | |
| *** gjayavelu has joined #openstack-nova | 03:38 | |
| *** gouthamr has joined #openstack-nova | 03:38 | |
| *** felipemonteiro_ has joined #openstack-nova | 03:39 | |
| *** lpetrut has joined #openstack-nova | 03:41 | |
| *** felipemonteiro__ has quit IRC | 03:43 | |
| *** nicolasbock has quit IRC | 03:44 | |
| *** felipemonteiro_ has quit IRC | 03:45 | |
| *** mingyu has quit IRC | 03:46 | |
| *** kotra03_2 has joined #openstack-nova | 03:46 | |
| *** mingyu has joined #openstack-nova | 03:48 | |
| *** gjayavelu has quit IRC | 03:52 | |
| *** gouthamr has quit IRC | 03:52 | |
| *** gjayavelu has joined #openstack-nova | 03:53 | |
| *** yamamoto has joined #openstack-nova | 03:59 | |
| *** rcernin has quit IRC | 03:59 | |
| *** rcernin has joined #openstack-nova | 04:00 | |
| *** edmondsw has joined #openstack-nova | 04:00 | |
| *** germs has quit IRC | 04:02 | |
| *** hongbin has quit IRC | 04:02 | |
| *** germs has joined #openstack-nova | 04:02 | |
| *** germs has quit IRC | 04:02 | |
| *** germs has joined #openstack-nova | 04:02 | |
| *** gyankum has joined #openstack-nova | 04:04 | |
| *** edmondsw has quit IRC | 04:05 | |
| *** yamamoto has quit IRC | 04:06 | |
| *** markvoelker has joined #openstack-nova | 04:08 | |
| *** r-daneel has joined #openstack-nova | 04:10 | |
| *** slaweq has joined #openstack-nova | 04:11 | |
| *** suresh12 has joined #openstack-nova | 04:11 | |
| *** Tom-Tom_ has joined #openstack-nova | 04:12 | |
| *** slaweq has quit IRC | 04:15 | |
| *** Tom-Tom has quit IRC | 04:15 | |
| *** janki has joined #openstack-nova | 04:15 | |
| *** Tom-Tom has joined #openstack-nova | 04:16 | |
| *** Tom-Tom_ has quit IRC | 04:16 | |
| *** kotra03_2 has quit IRC | 04:19 | |
| *** lpetrut has quit IRC | 04:21 | |
| *** Tom-Tom has quit IRC | 04:23 | |
| *** lpetrut has joined #openstack-nova | 04:25 | |
| openstackgerrit | jichenjc proposed openstack/nova master: WIP:[doc]Move configuration to admin subfolder https://review.openstack.org/566212 | 04:31 |
| *** fragatina has joined #openstack-nova | 04:32 | |
| *** r-daneel_ has joined #openstack-nova | 04:35 | |
| *** r-daneel has quit IRC | 04:36 | |
| *** r-daneel_ is now known as r-daneel | 04:36 | |
| *** gouthamr has joined #openstack-nova | 04:40 | |
| *** markvoelker has quit IRC | 04:41 | |
| *** jaosorior has joined #openstack-nova | 04:42 | |
| *** bhagyashris has quit IRC | 04:43 | |
| *** bhagyashris has joined #openstack-nova | 04:43 | |
| *** lpetrut has quit IRC | 04:44 | |
| *** moshele has joined #openstack-nova | 04:45 | |
| *** gouthamr has quit IRC | 04:49 | |
| *** abhishekk has joined #openstack-nova | 04:49 | |
| *** gouthamr has joined #openstack-nova | 04:50 | |
| *** damien_r has joined #openstack-nova | 04:54 | |
| *** gouthamr has quit IRC | 04:56 | |
| *** damien_r has quit IRC | 04:58 | |
| *** gjayavelu has quit IRC | 05:01 | |
| *** ianw has quit IRC | 05:06 | |
| *** krtaylor has quit IRC | 05:09 | |
| *** NobodyCam has quit IRC | 05:15 | |
| *** cz2 has quit IRC | 05:15 | |
| *** patrickeast has quit IRC | 05:15 | |
| *** sdake has quit IRC | 05:15 | |
| *** TheJulia has quit IRC | 05:15 | |
| *** pas-ha has quit IRC | 05:15 | |
| *** hogepodge has quit IRC | 05:15 | |
| *** vdrok has quit IRC | 05:15 | |
| *** udesale has joined #openstack-nova | 05:15 | |
| *** sdake has joined #openstack-nova | 05:15 | |
| *** sdake has quit IRC | 05:15 | |
| *** sdake has joined #openstack-nova | 05:15 | |
| *** TheJulia has joined #openstack-nova | 05:15 | |
| *** hogepodge has joined #openstack-nova | 05:16 | |
| *** vdrok has joined #openstack-nova | 05:16 | |
| *** pas-ha has joined #openstack-nova | 05:16 | |
| *** NobodyCam has joined #openstack-nova | 05:16 | |
| *** cz2 has joined #openstack-nova | 05:16 | |
| *** dpawlik has joined #openstack-nova | 05:17 | |
| *** suresh12 has quit IRC | 05:20 | |
| *** dpawlik has quit IRC | 05:21 | |
| *** lpetrut has joined #openstack-nova | 05:22 | |
| *** moshele has quit IRC | 05:23 | |
| *** lpetrut has quit IRC | 05:29 | |
| *** suresh12 has joined #openstack-nova | 05:30 | |
| *** suresh12 has quit IRC | 05:34 | |
| *** Gorian has quit IRC | 05:34 | |
| *** markvoelker has joined #openstack-nova | 05:38 | |
| *** Gorian has joined #openstack-nova | 05:42 | |
| openstackgerrit | jichenjc proposed openstack/nova master: [doc]Move configuration to admin subfolder https://review.openstack.org/566212 | 05:43 |
| *** Tom-Tom has joined #openstack-nova | 05:44 | |
| *** nsingh has joined #openstack-nova | 05:45 | |
| nsingh | any command or way to confirm which services running on compute nodes??? | 05:45 |
| *** threestrands has quit IRC | 05:47 | |
| *** ratailor has joined #openstack-nova | 05:47 | |
| *** slaweq has joined #openstack-nova | 05:47 | |
| *** slaweq has quit IRC | 05:52 | |
| *** ianw has joined #openstack-nova | 05:55 | |
| *** sridharg has joined #openstack-nova | 05:58 | |
| *** hemna_ has quit IRC | 06:07 | |
| *** damien_r has joined #openstack-nova | 06:10 | |
| *** markvoelker has quit IRC | 06:11 | |
| *** evin has joined #openstack-nova | 06:12 | |
| *** trungnv has quit IRC | 06:20 | |
| *** damien_r has quit IRC | 06:20 | |
| *** lpetrut has joined #openstack-nova | 06:28 | |
| *** do3meli has joined #openstack-nova | 06:42 | |
| *** pcaruana has joined #openstack-nova | 06:43 | |
| *** do3meli has left #openstack-nova | 06:43 | |
| openstackgerrit | zhangyangyang proposed openstack/nova master: Remove the function get_back_port() https://review.openstack.org/566219 | 06:48 |
| *** r-daneel has quit IRC | 06:49 | |
| *** krtaylor has joined #openstack-nova | 06:50 | |
| *** damien_r has joined #openstack-nova | 06:50 | |
| *** alex_xu has quit IRC | 06:50 | |
| *** alex_xu has joined #openstack-nova | 06:51 | |
| *** damien_r has quit IRC | 06:55 | |
| *** damien_r has joined #openstack-nova | 06:55 | |
| *** rcernin has quit IRC | 07:05 | |
| *** slaweq has joined #openstack-nova | 07:05 | |
| *** slaweq has quit IRC | 07:06 | |
| *** slaweq has joined #openstack-nova | 07:06 | |
| *** markvoelker has joined #openstack-nova | 07:08 | |
| *** sahid has joined #openstack-nova | 07:09 | |
| *** dpawlik has joined #openstack-nova | 07:10 | |
| *** dpawlik has quit IRC | 07:16 | |
| *** brault has joined #openstack-nova | 07:18 | |
| *** dpawlik has joined #openstack-nova | 07:19 | |
| *** tesseract has joined #openstack-nova | 07:20 | |
| *** andreas_s has joined #openstack-nova | 07:25 | |
| *** jaosorior has quit IRC | 07:25 | |
| *** alexchadin has joined #openstack-nova | 07:32 | |
| *** amoralej|off is now known as amoralej | 07:34 | |
| *** wolverineav has joined #openstack-nova | 07:34 | |
| *** edmondsw has joined #openstack-nova | 07:36 | |
| bauzas | good morning Nova | 07:37 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova stable/ocata: libvirt: Make `cpu_model_extra_flags` case-insensitive for real https://review.openstack.org/565672 | 07:37 |
| bauzas | let me put a Friday swag | 07:37 |
| *** bauzas is now known as bauwser | 07:37 | |
| *** edmondsw has quit IRC | 07:41 | |
| *** markvoelker has quit IRC | 07:42 | |
| *** slunkad has quit IRC | 07:45 | |
| *** kukacz has quit IRC | 07:46 | |
| *** jpena|off is now known as jpena | 07:48 | |
| *** slunkad has joined #openstack-nova | 07:49 | |
| *** ccamacho has joined #openstack-nova | 07:51 | |
| *** kukacz_ has joined #openstack-nova | 07:55 | |
| *** yamahata has joined #openstack-nova | 07:57 | |
| *** wolverineav has quit IRC | 07:57 | |
| *** wolverineav has joined #openstack-nova | 07:57 | |
| *** janki has quit IRC | 07:57 | |
| *** kukacz_ has quit IRC | 08:01 | |
| *** kukacz_ has joined #openstack-nova | 08:01 | |
| *** wolverineav has quit IRC | 08:02 | |
| *** ttsiouts has quit IRC | 08:04 | |
| *** ttsiouts has joined #openstack-nova | 08:04 | |
| *** yassine has joined #openstack-nova | 08:04 | |
| *** yassine is now known as Guest38803 | 08:05 | |
| *** lucas-afk is now known as lucasagomes | 08:07 | |
| *** kukacz_ has quit IRC | 08:07 | |
| *** mgoddard has joined #openstack-nova | 08:08 | |
| *** zhaochao has quit IRC | 08:10 | |
| *** kukacz_ has joined #openstack-nova | 08:11 | |
| *** kukacz_ has quit IRC | 08:14 | |
| *** kukacz_ has joined #openstack-nova | 08:14 | |
| *** salv-orlando has joined #openstack-nova | 08:15 | |
| *** salv-orlando has quit IRC | 08:15 | |
| *** salv-orlando has joined #openstack-nova | 08:16 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: WIP new migration threads control https://review.openstack.org/563505 | 08:17 |
| *** zhaochao has joined #openstack-nova | 08:22 | |
| *** yamahata has quit IRC | 08:23 | |
| bauwser | andreykurilin: remind me, if I'm using novaclient python bindings for calling Nova, if I'm not providing a specific microversion request, then I get v2.1, right? | 08:24 |
| bauwser | contrary to when using the nova CLI, when we ask for nova.latest microversion, right? | 08:25 |
| *** damien_r has quit IRC | 08:25 | |
| bauwser | gibi: if you remind as well ^ | 08:25 |
| *** derekh has joined #openstack-nova | 08:25 | |
| bauwser | I mean, when I do a Client('2'), I should only get v2.1 | 08:29 |
| bauwser | actually, I should get /v2 which maps to /v2.1 to be precise | 08:29 |
| *** mdbooth has joined #openstack-nova | 08:29 | |
| *** stephenfin is now known as finucannot | 08:29 | |
| *** damien_r has joined #openstack-nova | 08:30 | |
| bauwser | yeah, confirmed https://github.com/openstack/python-novaclient/blob/master/novaclient/api_versions.py#L233-L234 | 08:31 |
| bauwser | do we have any novaclient specialists here ? | 08:31 |
| bauwser | just to confirm | 08:31 |
| *** lyarwood has quit IRC | 08:34 | |
| *** damien_r has quit IRC | 08:35 | |
| *** andreas_s has quit IRC | 08:38 | |
| *** andreas_s has joined #openstack-nova | 08:39 | |
| *** markvoelker has joined #openstack-nova | 08:39 | |
| *** damien_r has joined #openstack-nova | 08:41 | |
| *** germs has quit IRC | 08:43 | |
| *** germs has joined #openstack-nova | 08:44 | |
| *** germs has quit IRC | 08:44 | |
| *** germs has joined #openstack-nova | 08:44 | |
| *** Tom-Tom has quit IRC | 08:44 | |
| *** lbragstad has quit IRC | 08:45 | |
| *** lbragstad has joined #openstack-nova | 08:45 | |
| openstackgerrit | Nguyen Hai proposed openstack/nova-specs master: Follow the new PTI for document build https://review.openstack.org/551802 | 08:46 |
| *** Tom-Tom has joined #openstack-nova | 08:47 | |
| *** gibi is now known as giblet | 08:47 | |
| *** Tom-Tom has quit IRC | 08:47 | |
| giblet | bauwser: I don't know | 08:49 |
| bauwser | no worries, I think I have all that I want | 08:49 |
| giblet | OK | 08:49 |
| bauwser | we discover the versions with the shell | 08:50 |
| bauwser | but we don't with the python bindings directly | 08:50 |
| *** salv-orlando has quit IRC | 08:53 | |
| *** andreas_s has quit IRC | 09:01 | |
| *** andreas_s has joined #openstack-nova | 09:02 | |
| *** janki has joined #openstack-nova | 09:03 | |
| *** markvoelker has quit IRC | 09:12 | |
| *** tssurya has joined #openstack-nova | 09:25 | |
| *** udesale_ has joined #openstack-nova | 09:29 | |
| *** udesale__ has joined #openstack-nova | 09:31 | |
| *** udesale has quit IRC | 09:32 | |
| *** dtantsur|afk is now known as dtantsur | 09:33 | |
| *** udesale_ has quit IRC | 09:34 | |
| *** yamamoto has joined #openstack-nova | 09:49 | |
| *** rabel_ has joined #openstack-nova | 09:51 | |
| rabel_ | hi there. I just saw that add-floating-ip action is deprecated in compute api. but how is a floating ip associated to an instance then? | 09:51 |
| *** fragatina has quit IRC | 09:52 | |
| *** fragatina has joined #openstack-nova | 09:53 | |
| *** yamamoto has quit IRC | 09:53 | |
| *** kholkina has joined #openstack-nova | 09:54 | |
| *** rabel_ has left #openstack-nova | 09:55 | |
| *** rabel_ has joined #openstack-nova | 09:55 | |
| rabel_ | found it. network api https://developer.openstack.org/api-ref/network/v2/#floating-ips-floatingips | 09:55 |
| *** rabel_ has left #openstack-nova | 09:55 | |
| *** salv-orlando has joined #openstack-nova | 09:57 | |
| *** georgk has joined #openstack-nova | 09:59 | |
| *** georgk has left #openstack-nova | 09:59 | |
| *** armaan has joined #openstack-nova | 10:04 | |
| *** wolverineav has joined #openstack-nova | 10:07 | |
| *** hoangcx has quit IRC | 10:08 | |
| *** markvoelker has joined #openstack-nova | 10:08 | |
| *** wolverineav has quit IRC | 10:12 | |
| *** udesale_ has joined #openstack-nova | 10:13 | |
| *** udesale__ has quit IRC | 10:16 | |
| *** tiendc has quit IRC | 10:17 | |
| *** andreas_s has quit IRC | 10:25 | |
| *** andreas_s has joined #openstack-nova | 10:27 | |
| *** r-daneel has joined #openstack-nova | 10:29 | |
| *** mvk has quit IRC | 10:30 | |
| *** salv-orlando has quit IRC | 10:31 | |
| *** andreas_s has quit IRC | 10:35 | |
| *** lpetrut has quit IRC | 10:39 | |
| *** markvoelker has quit IRC | 10:42 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290 | 10:48 |
| finucannot | jaypipes, giblet: OK, I think that spec is good to go now. I've clarified a lot of the physnet/provider net stuff and fixed the diagrams https://review.openstack.org/541290 | 10:49 |
| * finucannot goes to catch up on bandwidth-aware scheduling | 10:49 | |
| *** phuongnh has quit IRC | 10:50 | |
| *** liuzz has quit IRC | 10:50 | |
| giblet | finucannot: looking | 10:55 |
| *** spsurya has quit IRC | 10:56 | |
| openstackgerrit | Surya Seetharaman proposed openstack/nova stable/queens: Make association_refresh configurable https://review.openstack.org/566288 | 11:01 |
| *** wolverineav has joined #openstack-nova | 11:02 | |
| *** andreas_s has joined #openstack-nova | 11:03 | |
| *** alexchadin has quit IRC | 11:04 | |
| *** salv-orlando has joined #openstack-nova | 11:07 | |
| giblet | finucannot: your spec looks good to me | 11:08 |
| *** tssurya is now known as sususuryashines | 11:09 | |
| *** lpetrut has joined #openstack-nova | 11:09 | |
| *** dave-mccowan has joined #openstack-nova | 11:14 | |
| *** abhishekk has quit IRC | 11:14 | |
| *** alexchadin has joined #openstack-nova | 11:17 | |
| *** lucasagomes is now known as lucas-pto | 11:17 | |
| *** udesale__ has joined #openstack-nova | 11:22 | |
| *** udesale__ has quit IRC | 11:22 | |
| *** udesale_ has quit IRC | 11:24 | |
| *** Eran_Kuris has quit IRC | 11:26 | |
| *** LiveOne_ has quit IRC | 11:26 | |
| *** pcaruana has quit IRC | 11:27 | |
| *** salv-orlando has quit IRC | 11:31 | |
| *** BobBall has quit IRC | 11:33 | |
| *** markvoelker has joined #openstack-nova | 11:39 | |
| *** pcaruana has joined #openstack-nova | 11:39 | |
| *** nicolasbock has joined #openstack-nova | 11:42 | |
| *** Eran_Kuris has joined #openstack-nova | 11:44 | |
| *** suresh12 has joined #openstack-nova | 11:46 | |
| *** Eran_Kuris_ has joined #openstack-nova | 11:47 | |
| *** Eran_Kuris has quit IRC | 11:49 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Placement: any traits in allocation_candidate query https://review.openstack.org/565730 | 11:50 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Placement: support mixing required traits with any traits https://review.openstack.org/565741 | 11:50 |
| *** linkmark has quit IRC | 11:51 | |
| *** suresh12 has quit IRC | 11:51 | |
| *** edmondsw has joined #openstack-nova | 11:53 | |
| *** Eran_Kuris_ has quit IRC | 11:54 | |
| *** kaisers1 has joined #openstack-nova | 11:55 | |
| *** kaisers has quit IRC | 11:56 | |
| *** annp has quit IRC | 11:57 | |
| *** pcaruana has quit IRC | 11:59 | |
| *** Eran_Kuris_ has joined #openstack-nova | 12:01 | |
| *** liverpooler has joined #openstack-nova | 12:01 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: Placement: support mixing required traits with any traits https://review.openstack.org/565741 | 12:02 |
| *** jpena is now known as jpena|lunch | 12:03 | |
| *** jmccarthy has joined #openstack-nova | 12:03 | |
| jmccarthy | Hmm I still have this issue where after a cold migration, a disk.info file shows up in instance dir and is left on source host | 12:06 |
| jmccarthy | I'm using kolla stable/queens - does this log seem right ? | 12:06 |
| jmccarthy | http://paste.openstack.org/show/720358/ | 12:06 |
| jmccarthy | It's when the resize/migration is confirmed that it shows up | 12:07 |
| jmccarthy | This part looks good CMD "rm -rf /var/lib/nova/instances/371e669b-0f15-49f2-9a84-bd1e89f34294_resize" returned: 0 | 12:09 |
| jmccarthy | But then a lock is acquired on disk.info *after that ? and it's left there .. | 12:10 |
| *** jistr is now known as jistr|mtg | 12:11 | |
| *** pcaruana has joined #openstack-nova | 12:12 | |
| *** markvoelker has quit IRC | 12:12 | |
| *** pchavva has joined #openstack-nova | 12:15 | |
| *** alexchadin has quit IRC | 12:15 | |
| *** dtantsur is now known as dtantsur|brb | 12:17 | |
| *** vladikr has joined #openstack-nova | 12:21 | |
| jmccarthy | After migrating an instance, is /var/lib/nova/instances/371e669b-0f15-49f2-9a84-bd1e89f34294/disk.info supposed to written out on the source host ? | 12:21 |
| jmccarthy | s/After/After cold-/ | 12:21 |
| *** markvoelker has joined #openstack-nova | 12:24 | |
| *** mchlumsky has joined #openstack-nova | 12:26 | |
| *** mchlumsky has quit IRC | 12:30 | |
| *** jistr|mtg is now known as jistr | 12:31 | |
| openstackgerrit | Merged openstack/nova master: libvirt: Drop BAD_LIBVIRT_CPU_POLICY_VERSIONS https://review.openstack.org/564012 | 12:31 |
| openstackgerrit | Merged openstack/nova master: libvirt: Drop MIN_QEMU_POSTCOPY_VERSION https://review.openstack.org/565724 | 12:31 |
| openstackgerrit | Merged openstack/nova master: libvirt: Drop MIN_LIBVIRT_REALTIME_VERSION https://review.openstack.org/565707 | 12:31 |
| *** mchlumsky has joined #openstack-nova | 12:32 | |
| *** vivsoni_ has joined #openstack-nova | 12:35 | |
| *** vivsoni has quit IRC | 12:35 | |
| finucannot | giblet: One thought I had on numa-aware-vswitches: how do we deal with changes in the networks attached to the guest when migrating? I think you touched on that here https://review.openstack.org/#/c/541290/8/specs/rocky/approved/numa-aware-vswitches.rst@243 | 12:37 |
| *** armaan has quit IRC | 12:38 | |
| giblet | finucannot: yes. even if the network itself is not changing the NUMA affinity of the devices providing access to the given network can be different on different hosts | 12:38 |
| *** armaan has joined #openstack-nova | 12:38 | |
| giblet | finucannot: and the actual physnet can change if we have multiprovider networks with mutliple segments having different physnets | 12:39 |
| giblet | finucannot: but the later does not supported today by nova anyhow | 12:39 |
| finucannot | OK, I don't think that's actually an issue. In that case, we'd be regenerating the guest NUMA topology for the new host and that would use the new host's NUMA-network mapping | 12:39 |
| finucannot | Also, I think we decided multi-segment is out-of-scope | 12:39 |
| giblet | finucannot: multi-segment is out of scope, I agree | 12:39 |
| finucannot | I'm more concerned about the actual networks attached. We've said that a user has to allocate networks at instance creation time so we can do NUMA affinity | 12:40 |
| finucannot | However, you can attach/detach networks to/from a running instance, right? | 12:40 |
| giblet | finucannot: could you be bit more specific how the network changes in your scenario | 12:40 |
| giblet | finucannot: ahh, yes | 12:41 |
| finucannot | So I boot an instance attached to network 'foo' and then, once it's running, detach from 'foo' and attach to 'bar' | 12:41 |
| giblet | finucannot: when you attach a new port / network that might affect the necessary affinity | 12:41 |
| finucannot | Yeah, exactly. We can't do anything about that while the instance is on the same host, but what about if we migrate/rebuild? | 12:42 |
| giblet | finucannot: you can actually check if the port being attached creates a contradiction with the existing affinity of the instance | 12:42 |
| giblet | finucannot: and you might reject the attach | 12:43 |
| giblet | finucannot: but I agree that when you migrate you have to take every port / network into account | 12:43 |
| giblet | finucannot: including those that was attached after the boot | 12:43 |
| finucannot | Yeah, we could do that. I guess that could/should be a configurable policy option down the line | 12:44 |
| finucannot | But yeah, I'm thinking I should regenerate the NetworkRequestList object attached to the instance/request spec when migrating to reflect the network topology pre-migration | 12:45 |
| finucannot | That might even happen already. I should check | 12:45 |
| *** slunkad has quit IRC | 12:45 | |
| *** pchavva has quit IRC | 12:45 | |
| *** pchavva has joined #openstack-nova | 12:46 | |
| giblet | finucannot: I think regenerating the information from Neutron is the safe solution | 12:46 |
| *** yamamoto has joined #openstack-nova | 12:46 | |
| *** Nel1x has joined #openstack-nova | 12:46 | |
| giblet | finucannot: I would even go that far that don't persist the NetworkRequestList but simply regenerate when it is needed | 12:47 |
| *** yamamoto has quit IRC | 12:48 | |
| *** yamamoto has joined #openstack-nova | 12:49 | |
| finucannot | You need to though so that you can use it during claiming. Without that we have no way to figure out what's necessary https://review.openstack.org/#/c/541290/8/specs/rocky/approved/numa-aware-vswitches.rst@243 | 12:52 |
| finucannot | whoops | 12:52 |
| finucannot | https://review.openstack.org/#/c/564449/1/nova/compute/claims.py | 12:52 |
| finucannot | It also needs to be stored in the RequestSpec object so that we don't need to query neutron from the filters (which we can't do because we don't have correct context) | 12:53 |
| *** yamamoto has quit IRC | 12:53 | |
| *** andreas_s has quit IRC | 12:54 | |
| *** jpena|lunch is now known as jpena | 12:54 | |
| *** udesale has joined #openstack-nova | 12:54 | |
| *** alexchadin has joined #openstack-nova | 12:55 | |
| giblet | finucannot: does the RequestSpec loaded from the db in the scheduler or passed via rpc? | 12:55 |
| *** felipemonteiro has joined #openstack-nova | 12:55 | |
| finucannot | The former, to the best of my knowledge | 12:55 |
| giblet | finucannot: then I agree that you have to store the NetworkREquestList to the db along with the RequestSpec | 12:56 |
| *** Nel1x has quit IRC | 12:56 | |
| *** andreas_s_ has joined #openstack-nova | 12:56 | |
| *** andreas_s_ has quit IRC | 12:56 | |
| *** andreas_s_ has joined #openstack-nova | 12:56 | |
| finucannot | Yeah, I mapped the whole thing here http://paste.openstack.org/show/720365/ | 12:56 |
| *** edleafe is now known as figleaf | 12:57 | |
| *** ratailor has quit IRC | 12:59 | |
| giblet | finucannot: I think the RequestSpec is passed to the scheduler via rpc https://github.com/openstack/nova/blob/5d97937c3c56a3e240a3350a7a9f0e3dcb954c52/nova/scheduler/rpcapi.py#L132 | 12:59 |
| *** yamamoto has joined #openstack-nova | 13:00 | |
| giblet | the spec_obj there is a RequestSpec obj | 13:00 |
| *** Guest38803 has quit IRC | 13:01 | |
| *** efried is now known as fried_rice | 13:02 | |
| giblet | finucannot: also the build_and_run_instance leading to the claim gets the instance object via rpc | 13:03 |
| finucannot | Yup, so if I wanted to, for example, pass an additional 'network_requests' parameter to 'claim()', I guess I'd have to bump the RPC version | 13:04 |
| finucannot | I'm pretty sure I looked at that though and it wasn't possible. Lemme look again | 13:05 |
| giblet | finucannot: you can still add the network_requests as a field to the Instance ovo and to the RequestSpec ovo, but you not necessary to persist the content of that fields to the db, as the user of that field always get the object via rpc and the sender can regenerate the content of the network_request | 13:06 |
| *** mriedem has joined #openstack-nova | 13:06 | |
| giblet | so in case of boot, the conductor generates the content of the network_request field and pass it down via the ReqestSpec of the Instance object | 13:07 |
| giblet | similarly in case of a VM move operation | 13:07 |
| *** awaugama has joined #openstack-nova | 13:07 | |
| *** dtantsur|brb is now known as dtantsur | 13:09 | |
| *** gyankum has quit IRC | 13:09 | |
| finucannot | Yeah, that shouldn't be an issue for RequestSpec as I don't think those are persisted. What about Instance though. Can you mark a field in a persistent object as non-persistent? | 13:09 |
| giblet | persistence is itching my mind because this data is already persisted in neutron so as soon as nova also persist it we will have two possible divergent copies | 13:09 |
| giblet | finucannot: there is a request_specs table in the api db so your technical problem how to not persist a field is valid for both object | 13:11 |
| *** amoralej is now known as amoralej|lunch | 13:12 | |
| giblet | finucannot: we should ask dansmith about this persistency issue but I have an idea | 13:13 |
| giblet | finucannot: make the new field lazy_loaded | 13:13 |
| *** psachin has quit IRC | 13:13 | |
| giblet | finucannot: and try to generate the value of the field in obj_load_attr | 13:13 |
| giblet | finucannot: or another idea. never set the field in _from_db_object but set it from the conductor when the data is available | 13:14 |
| giblet | finucannot: also never pass this field to the db method in save() | 13:15 |
| jmccarthy | @jaypipes You about ? | 13:16 |
| *** r-daneel has quit IRC | 13:16 | |
| *** andreas_s_ has quit IRC | 13:17 | |
| finucannot | giblet: They're all good ideas. Let me explore them and see what I can do | 13:18 |
| finucannot | Agreed on split brain issue too. | 13:18 |
| *** eharney has joined #openstack-nova | 13:18 | |
| giblet | finucannot: OK. I will have same issue with the bandwidth related resource requests coming from the neutron port and going to placement a_c query via nova-conductor and nova-scheduler | 13:18 |
| *** andreas_s_ has joined #openstack-nova | 13:19 | |
| giblet | finucannot: but you are further down the road as you already have code up | 13:20 |
| finucannot | Yeah, I found I needed to write that so I could actually reason about stuff properly, otherwise I was guessing a lot of it :D | 13:20 |
| finucannot | Turns out I didn't know the scheduler all that well, heh | 13:21 |
| giblet | finucannot: yeah devil is in the details and in this cases writing code means seeing the details | 13:24 |
| *** sahid has quit IRC | 13:24 | |
| * giblet have to start coding up some PoC for the bandwidth | 13:25 | |
| *** sahid has joined #openstack-nova | 13:25 | |
| mriedem | bauwser: can you get https://review.openstack.org/#/c/566161/ ? | 13:26 |
| *** mingyu has quit IRC | 13:26 | |
| *** jaypipes is now known as pipesinpain | 13:27 | |
| pipesinpain | jmccarthy: yes | 13:28 |
| *** esberglu has joined #openstack-nova | 13:28 | |
| pipesinpain | finucannot: cool, will review this morning. | 13:28 |
| *** gbarros has joined #openstack-nova | 13:28 | |
| bauwser | mriedem: ack, looking | 13:28 |
| jmccarthy | pipesinpain: Hiya ! Just wondering if you might have some insight into this bug here ? 1769131 It sounds a lot like 1666831 | 13:30 |
| *** gouthamr has joined #openstack-nova | 13:30 | |
| giblet | mriedem: both pipesinpain, fried_rice and mlavalle seems to be OK with the nova bandwidth spec. So if you are interested then this is a good time to look at it https://review.openstack.org/#/c/502306/ | 13:31 |
| fried_rice | pipesinpain: elbow? | 13:31 |
| pipesinpain | fried_rice: yeah :( | 13:32 |
| pipesinpain | jmccarthy: will take a look after caffeinating. | 13:33 |
| fried_rice | Did you golf or something? | 13:33 |
| pipesinpain | fried_rice: nope. didn't do anything. | 13:33 |
| fried_rice | boo. Your elbow and my knees. No idea what happened, just swelling and pain this morning. Hell to get old. | 13:33 |
| *** jmccarthy has quit IRC | 13:34 | |
| giblet | pipesinpain, fried_rice: I hope both of you get better soon | 13:35 |
| fried_rice | thanks giblet | 13:35 |
| *** andreas_s_ has quit IRC | 13:35 | |
| pipesinpain | giblet: thx man. | 13:35 |
| *** jmccarthy has joined #openstack-nova | 13:36 | |
| jmccarthy | Erg vpn blipped | 13:36 |
| *** andreas_s has joined #openstack-nova | 13:37 | |
| jmccarthy | I've been looking at driver.py but I may be misinterpreting some of the comments that are in there - this seems relevant https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1164 | 13:37 |
| mriedem | giblet: ok will do | 13:37 |
| giblet | mriedem: thanks | 13:37 |
| pipesinpain | jmccarthy: you are using volume-backed instances, yes? | 13:38 |
| pipesinpain | jmccarthy: i.e. boot-from-volume | 13:38 |
| *** esberglu has quit IRC | 13:39 | |
| *** brault has quit IRC | 13:39 | |
| jmccarthy | I have to check what horizon says, spawning cirros with cinder, so volume backed yep | 13:39 |
| *** esberglu has joined #openstack-nova | 13:39 | |
| pipesinpain | jmccarthy: right. so that is the behaviour by design. | 13:40 |
| *** andreas_s has quit IRC | 13:40 | |
| *** andreas_s has joined #openstack-nova | 13:40 | |
| pipesinpain | jmccarthy: maciej had added that code to ensure that the volume wasn't removed when resizing. | 13:40 |
| pipesinpain | jmccarthy: for boot-from-volume instances, you don't want to delete the original root volume, clearly :) | 13:40 |
| pipesinpain | jmccarthy: I'm wondering if this is because of some thin/sparse copy-on-write stuff that Docker is doing maybe.. | 13:41 |
| jmccarthy | pipesinpain: The volume itself is in cinder and seems safe enough in my case ? It's this disk.info that shows up on host that is throwing me off | 13:42 |
| pipesinpain | jmccarthy: but in all honesty, I'm pretty much the worst person to ask on this :) a) I have little knowledge of the block device layer, b) I don't use (or support) boot-from-volume, and c) I don't use resize ;) | 13:42 |
| pipesinpain | jmccarthy: good people to hit up are mriedem when he's online and maybe mdbooth | 13:42 |
| jmccarthy | pipesinpain: Oh ok I thought you might know, since my disk.info issue sounds a lot like a bug you fixed up | 13:43 |
| jmccarthy | pippesinpain: Ok I'll ask around - I'm just not sure why it would leave this file on the host afterwards, seems odd | 13:43 |
| pipesinpain | jmccarthy: that was more melwitt (soon to be jgwentworth) that fixed that bug :) | 13:43 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove support for monitoring Intel CMT `perf` events https://review.openstack.org/565242 | 13:43 |
| * kashyap wonders if the above requires a rel note | 13:43 | |
| jmccarthy | pipesinpain: Ok ! :) Sorry for the noise | 13:43 |
| pipesinpain | jmccarthy: no problem at all, man :) | 13:44 |
| jmccarthy | Any of those people sitting quielty by ? ;) | 13:44 |
| *** andreas_s has quit IRC | 13:44 | |
| *** yamamoto has quit IRC | 13:45 | |
| pipesinpain | jmccarthy: melwitt will be up in the next hour or so. mriedem will be by soon I imagine (PST and CST respectively) | 13:46 |
| jmccarthy | In horizon, if it boots from image and you create a new volume at the same time in persistent storage for it, that not boot-from-volume strictly speaking, correct ? | 13:46 |
| pipesinpain | jmccarthy: mdbooth is in GMT so should be around (he's likely seen "boot from volume" and run screaming for the hills) :P | 13:46 |
| *** andreas_s_ has joined #openstack-nova | 13:46 | |
| pipesinpain | jmccarthy: no, that's not boot-from-volume. | 13:47 |
| jmccarthy | pipesinpain: That is a volume backed instance, but booted off image - does that have a special name ? | 13:47 |
| pipesinpain | jmccarthy: boot-from-volume is when you supply a prebuilt bootable volume as your root disk (disk 0) for the instance | 13:47 |
| *** slunkad has joined #openstack-nova | 13:47 | |
| pipesinpain | jmccarthy: that's a normal instance, then | 13:47 |
| jmccarthy | pipesinpain: I am using volumes, but nope, not boot from vol, ok normal instances | 13:47 |
| pipesinpain | jmccarthy: you mean you just added a non-bootable data volume to the instance, right? | 13:48 |
| wolverineav | hi, while working on a neutron plugin, i ran into a weird situation where nova sends multiple requests for bind_port on different hosts, before neutron has had a chance to complete the first request. i’m guessing this would be by design, since I don’t see error or warning about the first request failing. in case anyone would know where the code path for this resides, that’d be great! | 13:48 |
| pipesinpain | jmccarthy: good man, yes, that's not boot-from-volume :) | 13:48 |
| jmccarthy | pipesinpain: Well not manually ? Horizon just does it .. "Instance source is the template used to create an instance. You can use an image, a snapshot of an instance (image snapshot), a volume or a volume snapshot (if enabled). You can also choose to use persistent storage by creating a new volume." | 13:48 |
| pipesinpain | wolverineav: ironic? | 13:49 |
| wolverineav | no, instance creation in overcloud | 13:49 |
| mriedem | jmccarthy: nova show the instance on the CLI, if the image_ref is "", it's volume-backed (boot from volume) | 13:49 |
| pipesinpain | jmccarthy: right. and you chose the "persistent storage by creating a new volume", yes? | 13:49 |
| wolverineav | pipesinpain: no, instance creation in overcloud | 13:49 |
| jmccarthy | pipesinpain: Mine just says 'create new volume' but yep | 13:50 |
| pipesinpain | wolverineav: multiple requests to bind the port to different hosts for the same instance? :( | 13:50 |
| jmccarthy | pipesinpain: Afaik this creates an instance the lives solely in the volume like, and subsequently will boot from there | 13:51 |
| jmccarthy | pipesinpain: 'pipesinpain' sounds painful | 13:52 |
| *** armaan has quit IRC | 13:52 | |
| *** r-daneel has joined #openstack-nova | 13:52 | |
| *** lyan has joined #openstack-nova | 13:53 | |
| *** amoralej|lunch is now known as amoralej | 13:53 | |
| *** lyan is now known as Guest36566 | 13:53 | |
| pipesinpain | jmccarthy: no. unless I'm mistaken (very much a possibility), "create a volume" just means the "You can also choose to use persistent storage by creating a new volume." option. In other words, it creates a data volume (non-bootable) for the instance to use. | 13:56 |
| wolverineav | pipesinpain: sorry about the phrasing - i'm looking at the problem completely from neutron logs - i (neutron) get port_update request with 2 different hosts in a span of less than 30secs. typically, there's about 3 port updates until its finally bound and in active state. i couldn't figure out where the aggresive timeout is, that forces a retry to another host. so i was checking in nova group if that was known :) | 13:56 |
| *** rajinir has joined #openstack-nova | 13:57 | |
| pipesinpain | wolverineav: but this is a port binding request for the *same* instance, yes? | 13:57 |
| wolverineav | pipesinpain: yes, correct. | 13:58 |
| *** ratailor has joined #openstack-nova | 13:58 | |
| pipesinpain | wolverineav: that is *seriously* odd... | 13:58 |
| pipesinpain | wolverineav: I can't understand why multiple hosts would be port-bound to the same instance at the same time | 13:59 |
| *** r-daneel has quit IRC | 13:59 | |
| pipesinpain | wolverineav: that would mean that the scheduler essentially picked multiple hosts for the same instance. | 13:59 |
| *** r-daneel has joined #openstack-nova | 13:59 | |
| pipesinpain | wolverineav: or that a retry is occurring nearly instantaneously. | 13:59 |
| pipesinpain | wolverineav: since the scheduler absolutely cannot simultaneously pick multiple hosts for the same instance, it must be related to the retry. | 14:00 |
| jmccarthy | pipesinpain: Hmm ok, what would that instance be booting from for future boots like after it's created ? I thought it did stuff with the image and used it to make the vol bootable like. I only see a /dev/sda and /dev/sda1 in the instance itself (which I thought was the new volume it created, made bootable) - I could be missing something .. | 14:00 |
| pipesinpain | wolverineav: can you please look in the compute logs for "Retry"? | 14:01 |
| pipesinpain | jmccarthy: it would be booting from the same original image you selected in horizon. | 14:01 |
| pipesinpain | jmccarthy: and would *attach* the persistent data volume after boot. | 14:01 |
| wolverineav | pipesinpain: i had to follow threads to figure out what was happening. it caused some inconsistency issue between neutron and DB and backend network controller when two requests got processed in different order in neutron vs calls made to controller. | 14:01 |
| wolverineav | pipesinpain: yes, let me check. I did not notice anything earlier. but i'll enable debug to be sure | 14:02 |
| jmccarthy | pipesinpain: Hmm ok that's still ok, I mean there is no other disk file for it in /var/lib/instances anywhere (at least that I could find) | 14:02 |
| pipesinpain | wolverineav: any chance you can upload the compute logs somewhere? or at least, a section of them around where this is happening? | 14:02 |
| jmccarthy | pipesinpain: (of whatever the path is, I'm sure I typed that wrong)# | 14:02 |
| wolverineav | pipesinpain: it doesn't always happen, so i don't have debug enabled logs (first happened at a customer site). i'll ping back here (/mailing list) once i have that reproduced. | 14:03 |
| pipesinpain | wolverineav: ++, thank you sir! :) | 14:04 |
| jmccarthy | pipesinpain: Sorry yes of course - I am confusing the preparation of volume to be used to boot from - nevermind :) I still think there is a bug or something strange going on :) Thanks for your help ! | 14:06 |
| *** armaan has joined #openstack-nova | 14:08 | |
| mriedem | giblet: weren't we at one point talking about the scheduler creating the allocations against the nic bw providers? now that gets passed down to neutron through port binding in the latest revision of the spec. | 14:10 |
| pipesinpain | jmccarthy: no problemo :) | 14:10 |
| mriedem | honestly i can't remember which one i was pushing for a month ago | 14:10 |
| *** mlavalle has joined #openstack-nova | 14:10 | |
| giblet | mriedem: scheduler does the claim, but nova sends to neutron what was claimed | 14:10 |
| mriedem | oh, ok, that was'nt clear to me yet | 14:11 |
| *** Cardoe has joined #openstack-nova | 14:12 | |
| *** evin has quit IRC | 14:12 | |
| *** jmccarthy has quit IRC | 14:12 | |
| giblet | mriedem: please leave a comment and I will clarify that in the spec | 14:13 |
| Cardoe | mriedem: You asked about rebooting rescued instances the other day at Rackspace here. I'm trying to find you an answer. | 14:13 |
| mriedem | Cardoe: that would be great, thanks - also posted to the openstack-dev ML | 14:13 |
| Cardoe | full disclosure, I'm fairly new here and I don't have any OpenStack experience/knowledge. | 14:14 |
| Cardoe | I'm purely a Xen hypervisor individual. But I hate hearing about downstream custom bits and fully advocate upstreaming all. | 14:14 |
| Cardoe | Hence why someone prodded me, but I'll do my best to get to the bottom of it. | 14:15 |
| *** melwitt is now known as jgwentworth | 14:15 | |
| jgwentworth | pipesinpain: just looked at those two bugs, agreed they look similar -- that is, the fix for the one should have avoided the problem in the other it seems. but I know not more than that, have to look deeper into it | 14:17 |
| *** salv-orlando has joined #openstack-nova | 14:18 | |
| *** salv-orlando has quit IRC | 14:18 | |
| *** alexchadin has quit IRC | 14:19 | |
| *** Shilpa has quit IRC | 14:20 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Base test module/class for functional placement db https://review.openstack.org/564590 | 14:24 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Use test_base symbols directly https://review.openstack.org/564592 | 14:24 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Use helpers in test_resource_provider (func) https://review.openstack.org/564638 | 14:24 |
| mriedem | pipesinpain: giblet: thinking about ironic and this nw bw qos thing - in the case of ironc, if the user wants guaranteed minimum bw, do they just request that using a specific ironic flavor with a custom resource class indicating that ironic node will have that min bw? | 14:25 |
| mriedem | or its still tied up in the port? | 14:25 |
| giblet | mriedem: I don't know. How does ironic request a port today? via neutron? | 14:27 |
| mriedem | jroll: ^ | 14:27 |
| giblet | mriedem, jroll: or maybe the real question is, who handles the compute side of the networking in case of ironic | 14:28 |
| giblet | mriedem, jroll: as the current bandwidth spec only handles the bandwidth resource on the device that is in the compute node | 14:29 |
| *** artom_ has joined #openstack-nova | 14:29 | |
| mriedem | giblet: the reason i ask is because i was reading the "Finding the compute RP" section | 14:29 |
| *** artom has quit IRC | 14:29 | |
| mriedem | and thinking about how >1 ironic node will have the same compute 'host' | 14:29 |
| kashyap | mriedem: When you get a sec, does this require a release note? Remove support for monitoring Intel CMT `perf` events https://review.openstack.org/565242 | 14:30 |
| mriedem | if we just can't support ironic, fine, i just want to make sure i understand it's a limitation | 14:30 |
| mriedem | kashyap: yes | 14:30 |
| mriedem | at least as an fyi | 14:30 |
| mriedem | kashyap: just put in an 'other' release note | 14:30 |
| kashyap | mriedem: Yeah, thought so. Let me make it right away | 14:30 |
| mriedem | or 'update' | 14:30 |
| kashyap | Thanks | 14:30 |
| mriedem | *'upgrade' | 14:30 |
| kashyap | Yep, the 'upgrade' tag seems more applicable | 14:31 |
| *** jmccarthy has joined #openstack-nova | 14:31 | |
| giblet | mriedem: the name field of the ResourceProvider has a unique constraint in the DB so the unique compute node RP name is enforced for ironic case as well | 14:31 |
| *** hemna_ has joined #openstack-nova | 14:31 | |
| giblet | mriedem: I guess ironic uses the node name instead of the host name for the compute RP | 14:32 |
| *** lpetrut has quit IRC | 14:34 | |
| Cardoe | mriedem: So vm_states.py doesn't allow rebooting from rescue either. | 14:35 |
| jmccarthy | mriedem: Hiya, just wondering if you have any ideas maybe about this bug ? https://bugs.launchpad.net/nova/+bug/1769131 It seems like another bug that has come up before (1666831) but I'm not sure what the story is | 14:36 |
| openstack | jmccarthy: Error: Could not gather data from Launchpad for bug #1769131 (https://launchpad.net/bugs/1769131). The error has been logged | 14:36 |
| mriedem | Cardoe: correct. the upstream nova api doesn't allow rebooting a rescued vm | 14:36 |
| Cardoe | mriedem: I'm talking about the rax patched version | 14:36 |
| mriedem | Cardoe: oh, heh | 14:36 |
| Cardoe | I had to figure out how to get into a compute node over here. | 14:36 |
| mriedem | is it maybe hard-coded into the nova/compute/api.py code? | 14:37 |
| mriedem | there was some refactoring done in there in the last couple of releases, and i don't think rax has updated code to match upstream in a long time | 14:37 |
| Cardoe | checked that too. the @check_instance_state is the same as upstream | 14:37 |
| mriedem | so don't check vm_states.py, check nova/compute/api.py:API.reboot() | 14:38 |
| jroll | mriedem: giblet: for ironic, nova creates the port and ironic updates it later in the provisioning process | 14:38 |
| mriedem | hmm | 14:38 |
| jroll | I would like to see qos be a traits thing | 14:38 |
| Cardoe | well not the same because yeah I see the results of a big refactor but it doesn't seem to be allowed in there. | 14:38 |
| jroll | for ironic, it's just a property of the machine, not anything we can control | 14:38 |
| * jroll has to step away for a bit | 14:39 | |
| mriedem | Cardoe: i don't suppose anyone still has a line to Matthew Sherborne huh | 14:39 |
| giblet | jroll: the current spec defines bandwidth as a resource on provided by the physical device on the compute https://review.openstack.org/#/c/502306/ | 14:39 |
| mriedem | because https://bugs.launchpad.net/nova/+bug/1170237 seems totally bogus | 14:39 |
| openstack | Launchpad bug 1170237 in OpenStack Compute (nova) "cannot reboot instances when in rescue mode" [Medium,Fix released] - Assigned to Matthew Sherborne (msherborne+openstack) | 14:39 |
| jroll | hrm | 14:39 |
| jroll | giblet: I'll visit it when I'm back from errands | 14:39 |
| mriedem | unless he just took a bunch of nova people on a wild goose chase 5 years ago | 14:39 |
| *** germs has quit IRC | 14:39 | |
| giblet | jroll: thanks | 14:39 |
| Cardoe | mriedem: I'm lighting up some folks. I'll get you a better answer soon. | 14:40 |
| *** germs has joined #openstack-nova | 14:40 | |
| *** germs has quit IRC | 14:40 | |
| *** germs has joined #openstack-nova | 14:40 | |
| *** andreas_s_ has quit IRC | 14:41 | |
| *** eharney has quit IRC | 14:42 | |
| *** hongbin has joined #openstack-nova | 14:43 | |
| *** armaan has quit IRC | 14:44 | |
| giblet | mriedem: as far as I understand neutron only configures TOR switches for ironic and ironic does the compute side physical device handling | 14:45 |
| *** armaan has joined #openstack-nova | 14:45 | |
| giblet | mriedem: this means that neutron does not know what capabilities the physical device has | 14:45 |
| *** yamamoto has joined #openstack-nova | 14:45 | |
| *** r-daneel has quit IRC | 14:45 | |
| giblet | mriedem: so I think if ironic needs bandwidth handling the ironic virtdriver can create RPs with traits/resources and the user request that via flavor extra_spec | 14:45 |
| openstackgerrit | Eric Fried proposed openstack/nova master: placement: Granular GET /allocation_candidates https://review.openstack.org/517757 | 14:47 |
| giblet | mriedem: when (if) the bandwidth handling support is extened to TOR switches then baremetal neutron ports will have a resource request describing what the resource the port needs on the TOR switch | 14:47 |
| *** jistr is now known as jistr|tpb | 14:49 | |
| kashyap | "Lighting up some folks" brings a very strong image to the brain... | 14:50 |
| *** yamamoto has quit IRC | 14:52 | |
| *** r-daneel has joined #openstack-nova | 14:52 | |
| *** andreas_s has joined #openstack-nova | 14:52 | |
| *** dpawlik has quit IRC | 14:52 | |
| mriedem | jgwentworth: am i dreaming this up, or did we agree at the dublin ptg to add a type column to the consumers table in placement to be able to distinguish instance from migration consumers? | 14:57 |
| *** gyankum has joined #openstack-nova | 14:59 | |
| jgwentworth | mriedem: no, there wasn't agreement on that. but I realized what I needed it for (quotas) could be achieved with instance_mappings + queued_for_delete column + user_id column | 15:00 |
| *** andreas_s has quit IRC | 15:00 | |
| mriedem | oh i was just looking for it in your "count quotas using placement" spec | 15:00 |
| mriedem | ah i found it here https://etherpad.openstack.org/p/nova-ptg-rocky | 15:01 |
| mriedem | "In Sydney we talked about tracking a 'type' in the placement allocations/consumers tab"... | 15:01 |
| jgwentworth | yeah, that spec has been languishing and I really needed to update it after tssurya proposed her spec for adding queued_for_delete since mine would depend on that | 15:01 |
| mriedem | giblet: i dont know what "If QoS aware and non QoS aware ports are mixed on the same physical port" means | 15:02 |
| *** armaan has quit IRC | 15:02 | |
| *** armaan has joined #openstack-nova | 15:02 | |
| mriedem | that's possible today? to have a port in neutron that is both qos aware and not at the same time? | 15:02 |
| jgwentworth | yeah, we discussed the 'type' idea a bit at the ptg but things got pretty complicated, it would not be straightforward. we may need it someday for other reasons but nothing right now really needs it. I had thought I needed it for quotas but realized I didn't if we could have two more instance_mappings columns | 15:03 |
| mriedem | jgwentworth: ack; it's the new bdm.uuid column - something we'll always think we need every 4 months | 15:03 |
| giblet | mriedem: assume there is a compute node with an SRIOV PF that provide VFs for neutron ports. | 15:03 |
| jgwentworth | mriedem: heh yeah | 15:04 |
| giblet | mriedem: after our spec that PF will also provide bandwidth as well | 15:04 |
| jgwentworth | (I think we do have the bdm.uuid column now tho) | 15:04 |
| giblet | mriedem: a neutron port might have a QoS policy rule attached | 15:04 |
| mriedem | jgwentworth: yes we do | 15:05 |
| giblet | mriedem: when we place a neutron port with QoS policy rule attached to the above PF we will consume some bandwidth as well | 15:05 |
| *** jistr|tpb is now known as jistr | 15:05 | |
| mriedem | for mdbooth's local device serial series | 15:05 |
| jgwentworth | need=TRUE | 15:05 |
| jgwentworth | ah right | 15:05 |
| jgwentworth | persistent cereal numbers | 15:05 |
| giblet | mriedem: but if there is two neutron port one with QoS another without QoS ends up using VFs from the same PF then the minimum bandwidth rule cannot be garanteed | 15:05 |
| mriedem | jgwentworth: ala https://images-na.ssl-images-amazon.com/images/I/51hDzZDOhdL.jpg ? | 15:06 |
| *** r-daneel has quit IRC | 15:06 | |
| mriedem | giblet: ok | 15:06 |
| mriedem | over my head nfv isms, but ok | 15:06 |
| jgwentworth | hah, yup | 15:06 |
| *** r-daneel has joined #openstack-nova | 15:06 | |
| *** eharney has joined #openstack-nova | 15:07 | |
| giblet | mriedem: :) | 15:08 |
| *** artom_ has quit IRC | 15:08 | |
| *** ratailor has quit IRC | 15:08 | |
| dansmith | jgwentworth: just to be clear, avoiding the type column with mappings doesn't require joining any placement and nova-api tables right? | 15:10 |
| jgwentworth | dansmith: no, I was thinking a mappings query for count of instance_mappings that would be filter on project_id and user_id where queued_for_delete=0 | 15:12 |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Remove support for monitoring Intel CMT `perf` events https://review.openstack.org/565242 | 15:12 |
| dansmith | jgwentworth: okay that gets you quotas for instance count, but not the cores,ram,disk -- is that what you mean? | 15:12 |
| jgwentworth | dansmith: right. cores/ram would come from placement /usages query | 15:12 |
| dansmith | which still requires something like a type yeah? | 15:13 |
| *** pcaruana has quit IRC | 15:13 | |
| dansmith | like if a tenant has some allocations for VCPU that isn't nova-related | 15:13 |
| jgwentworth | I was thinking it didn't. unless we're suggesting VCPU and RAM can come from things that are not instances | 15:13 |
| jgwentworth | that was not my understanding thus far | 15:14 |
| idlemind | so cisco's ftd (firepower threat defense - read new asa) virtual in openstack ... boot from image (w/o create a volume) works but when i create a volume it fails to boot ... what impact does this have? if i "upgrade" a ftd virtual machine does it write back to the image or emphereal storage? do i need to create an "image" for each unique ftdv? | 15:14 |
| dansmith | jgwentworth: we brought that up in dublin, if something else started using placement for those resource types you'd be hosed.. that was the real reason we needed type I thought | 15:14 |
| dansmith | jgwentworth: so for example if bifrost or something like the container service, etc allocated those resources as well | 15:15 |
| dansmith | jgwentworth: and non-instance DISK_GB allocations for just generic volumes | 15:15 |
| jgwentworth | dansmith: yeah, if that's the case, then yeah type would be an inevitable need. or at least need a way for a subsystem to ask, "what's the resource usage for the resources that **I** put there?" so some identifier that describes the source or the type | 15:15 |
| *** Eran_Kuris_ has quit IRC | 15:16 | |
| dansmith | jgwentworth: yup, either type or "creator" or something | 15:16 |
| jgwentworth | yeah | 15:16 |
| kashyap | Any "stickler for words", appreciate a quick once-over: https://review.openstack.org/#/c/565242/4/releasenotes/notes/Remove-support-for-Intel-CMT-events-017fbb890b631d70.yaml | 15:18 |
| *** dave-mccowan has quit IRC | 15:20 | |
| *** pooja_jadhav has quit IRC | 15:24 | |
| jmccarthy | mriedman: totally missed your comment earlier ! I don't see image_ref, but 'image' has no value listed | 15:25 |
| *** nicolasbock has quit IRC | 15:26 | |
| openstackgerrit | sahid proposed openstack/nova master: pci: don't consider case when match tags specs https://review.openstack.org/565808 | 15:27 |
| openstackgerrit | sahid proposed openstack/nova master: network: update pci request spec to handle trusted tags https://review.openstack.org/458820 | 15:27 |
| openstackgerrit | sahid proposed openstack/nova master: libvirt: configure trust mode for vfs https://review.openstack.org/458514 | 15:27 |
| openstackgerrit | sahid proposed openstack/nova master: libvirt: add vf_trusted field for network metadata https://review.openstack.org/566343 | 15:27 |
| openstackgerrit | sahid proposed openstack/nova master: metadata: add vf_trusted field to device metadata https://review.openstack.org/566344 | 15:27 |
| jgwentworth | idlemind: boot from image without specifying a block device mapping will just use local storage on the compute host (hypervisor). boot from volume will use the volume's storage. what volume backend are you using? | 15:27 |
| *** dave-mccowan has joined #openstack-nova | 15:27 | |
| *** suresh12 has joined #openstack-nova | 15:27 | |
| jgwentworth | and if your local storage is shared/persistent, boot from image will use that (if you're using ceph, for example) | 15:28 |
| *** Eran_Kuris_ has joined #openstack-nova | 15:28 | |
| sahid | giblet: there is a serie that jay already started to review, i think you were also interested about it so if you have a moment to have look https://review.openstack.org/#/c/561912/ | 15:28 |
| giblet | sahid: I've put it on my review list | 15:29 |
| sahid | giblet: thanks | 15:29 |
| jmccarthy | Hiya melwitt: or mriedem: Any thought on this bug 1769131 ? I am getting this disk.info file created on the original compute after migrating the instance | 15:30 |
| openstack | bug 1769131 in OpenStack Compute (nova) "After cold-migration, disk.info file leftover on source host" [Undecided,New] https://launchpad.net/bugs/1769131 | 15:30 |
| *** sususuryashines has quit IRC | 15:30 | |
| jgwentworth | jmccarthy: hey, I looked at that earlier and agreed it looks like the same other similar bug you linked earlier. so I'm not sure what's going on there, how it's happening if someone already fixed the same problem | 15:31 |
| jmccarthy | jgwentworth: I dunno !? I'm not sure how it's happening for me, but it happens everytime | 15:31 |
| jmccarthy | jgwentworth: I have two setups doing it actually, but they both are cinder/lvm for volumes | 15:32 |
| jgwentworth | and it sounds like you're doing boot from volume? when you create the instance, are you specifying a volume or a block device mapping like source=image dest=volume? | 15:32 |
| jmccarthy | I'm just using horizon :| I give it an image name to start with and ask for it to create a new volume for it | 15:33 |
| jgwentworth | okay, that is going to be boot from volume I think | 15:33 |
| smcginnis | I believe so. | 15:33 |
| jgwentworth | that probably is why you're seeing something different than was described in the other similar bug | 15:33 |
| jmccarthy | Well the instance is definitely volume backed, but whether it boots from that as such I'm unclear now | 15:34 |
| jgwentworth | boot from volume is kind of a separate appendage in most of the code | 15:35 |
| jmccarthy | The only file associated with under instances, is it's console.log, that is until I cold migrate it and this disk.info show up | 15:37 |
| jgwentworth | yeah, volume backed == boot from volume. it just means the operating system, disk etc is stored on the volume | 15:38 |
| *** wwriverrat_ has joined #openstack-nova | 15:38 | |
| *** nicolasbock has joined #openstack-nova | 15:40 | |
| *** wwriverrat has quit IRC | 15:40 | |
| *** wwriverrat_ is now known as wwriverrat | 15:40 | |
| mriedem | giblet: a virtual cornucopia of comments and questions in https://review.openstack.org/#/c/502306/ | 15:41 |
| mriedem | giblet: good news is i think most of it is just asking for clarification | 15:41 |
| jgwentworth | jmccarthy: you see disk.info on the source or destination? | 15:42 |
| openstackgerrit | Eric Fried proposed openstack/nova master: placement: Object changes for granular https://review.openstack.org/564351 | 15:42 |
| openstackgerrit | Eric Fried proposed openstack/nova master: placement: Granular GET /allocation_candidates https://review.openstack.org/517757 | 15:42 |
| giblet | mriedem: thanks for the review. | 15:42 |
| jmccarthy | jgwentworth: It's no where in sight, until I confirm the resize/migrate - and then it appears on the host (the cold migrate completes fine) | 15:42 |
| *** bnemec is now known as beekneemech | 15:42 | |
| jmccarthy | *source host | 15:43 |
| jgwentworth | ok | 15:43 |
| jmccarthy | I'm going to check - I just remembered I looked at the file earlier - have to recheck what is actually in it somethingsomething qcow2 it said in the file | 15:44 |
| giblet | mriedem: I will go through you comments on Monday | 15:44 |
| jgwentworth | yeah, I can never remember what all of those are. I have to re-look at it every time in the code | 15:44 |
| mriedem | dansmith: (1) where is superdan and (2) i've got a request spec modeling comment in https://review.openstack.org/#/c/502306/26/specs/rocky/approved/bandwidth-resource-provider.rst@172 that could use your input | 15:45 |
| mriedem | giblet: ack | 15:45 |
| *** dansmith is now known as superdan | 15:45 | |
| superdan | mriedem: tab queued | 15:45 |
| *** armaan has quit IRC | 15:49 | |
| *** armaan has joined #openstack-nova | 15:49 | |
| jgwentworth | jmccarthy: what's your CONF.libvirt.images_type set to? | 15:50 |
| jmccarthy | one sec | 15:51 |
| mriedem | umm https://review.openstack.org/#/c/516395/ | 15:51 |
| jgwentworth | hah, nice | 15:52 |
| *** artom_ has joined #openstack-nova | 15:52 | |
| mriedem | also added a tempest test for that https://review.openstack.org/#/c/516396/ | 15:52 |
| jgwentworth | so maybe ... supposed to remove the disk.info that imagebackend creates but don't remove the whole instance directory? | 15:53 |
| jmccarthy | Hmm it's images_type = rbd in nova.conf | 15:53 |
| jgwentworth | k, thanks | 15:53 |
| mriedem | yeah so that's exactly what that patch was meant to fix, | 15:53 |
| mriedem | and from the tempest patch commit message, it's about shared storage backends | 15:53 |
| mriedem | so in our ci, that's ceph and nfs | 15:53 |
| *** artom_ has quit IRC | 15:53 | |
| mriedem | the problem was you would resize the instance and then try to get the console log which was deleted | 15:54 |
| *** artom_ has joined #openstack-nova | 15:54 | |
| mriedem | https://bugs.launchpad.net/nova/+bug/1728603 | 15:54 |
| openstack | Launchpad bug 1728603 in OpenStack Compute (nova) pike "Resize a boot-from-volume instance with NFS destroys instance" [High,Fix committed] - Assigned to Matt Riedemann (mriedem) | 15:54 |
| jgwentworth | right, that's what I'm trying to say is that it's true we shouldn't nuke the entire instance dir | 15:54 |
| jgwentworth | but, we should delete the disk.info by itself | 15:54 |
| jgwentworth | the original intent of https://review.openstack.org/#/c/437356/3/nova/virt/libvirt/driver.py was to delete the disk.info that imagebackend sometimes creates | 15:55 |
| *** damien_r has quit IRC | 15:55 | |
| jgwentworth | but it went ahead and took out the entire instance directory not realizing it would lose the console.log | 15:56 |
| jmccarthy | Yea I was reading those comments in driver.py - but the rm commands seem to only involve the dir (which I agree you can remove i.e. nfs) | 15:56 |
| jmccarthy | *can't | 15:56 |
| Cardoe | mriedem: ok so a bit more info. Modifying vm_states.py to allow rescued to be rebooted then it works. | 15:56 |
| *** dpawlik has joined #openstack-nova | 15:57 | |
| mriedem | Cardoe: sure, when using the xenapi driver right? | 15:57 |
| Cardoe | yes | 15:57 |
| jmccarthy | jgwentworth: But there must be logic to know when it's safe to remove that dir on the source host ? | 15:57 |
| jgwentworth | jmccarthy: did you see the patch mriedem linked though? it's bypassing all of that if volume-backed https://review.openstack.org/#/c/516395/2/nova/virt/libvirt/driver.py because it was trying to fix the case where the console.log was being deleted | 15:57 |
| mriedem | i have no idea if libvirt/hyperv/vmware support that | 15:57 |
| jmccarthy | 1728603 ? Let me look | 15:57 |
| Cardoe | mriedem: I've been told that I do not believe that they do. | 15:58 |
| jgwentworth | no, https://review.openstack.org/#/c/516395/2/nova/virt/libvirt/driver.py | 15:58 |
| Cardoe | s/I/rax OpenStack folks/ | 15:58 |
| jmccarthy | jswentworth: I think the driver.py that I have now looks like that one ? | 15:59 |
| jgwentworth | jmccarthy: so because you are volume-backed, it's not removing anything at all. I think it needs to be adjusted to only delete disk.info if volume-backed, or something like that | 16:00 |
| mriedem | jmccarthy: because you're on queens and that change was made in queens | 16:00 |
| mriedem | and backported to pike and ocata because it was fixing a regression which was also backported to pike and ocata :) | 16:00 |
| jmccarthy | jgwentworth: Ok so it's not just me - yes some logic is missing ? | 16:00 |
| mriedem | it's bugs all the way down | 16:00 |
| jmccarthy | jgwentworth: Leaving the dir there causes a problem for live migration too | 16:00 |
| *** gyee has joined #openstack-nova | 16:00 | |
| jgwentworth | yeah, there is definitely a bug. we were just trying to find what it was | 16:00 |
| mriedem | jmccarthy: the dir or just the disk.info file itself? | 16:01 |
| jmccarthy | jgwentworth: the dir | 16:01 |
| jmccarthy | mriedman: the dir | 16:01 |
| jmccarthy | Sorry I'm eventually getting there lol thanks for the help all ! :) | 16:01 |
| jmccarthy | One sec | 16:01 |
| jgwentworth | so what about the console.log then? where is it supposed to go | 16:01 |
| openstackgerrit | Eric Fried proposed openstack/nova master: placement: Object changes for granular https://review.openstack.org/564351 | 16:01 |
| openstackgerrit | Eric Fried proposed openstack/nova master: placement: Granular GET /allocation_candidates https://review.openstack.org/517757 | 16:01 |
| jgwentworth | oh, wait, supposed to remove the dir from the source | 16:02 |
| *** dpawlik has quit IRC | 16:02 | |
| mriedem | jgwentworth: but if you're on shared local storage, deleting the dir on the source will also remove it from the dest | 16:02 |
| jmccarthy | mriedman: Actually I have it at the end of the bug - https://bugs.launchpad.net/nova/+bug/1769131 about 'already exists, it is expected not to exist.' | 16:02 |
| openstack | Launchpad bug 1769131 in OpenStack Compute (nova) "After cold-migration of a volume-backed instance, disk.info file leftover on source host" [Undecided,Triaged] | 16:02 |
| jgwentworth | mriedem: yeah, right. okay. | 16:02 |
| mriedem | jmccarthy: yeah DestinationDiskExists only happens if compute says you're not on shared storage, | 16:02 |
| mriedem | but since you're using image_type=rbd i'd think your computes would be using shared storage | 16:03 |
| jgwentworth | yeah so it sounds like just need to delete disk.info, the presence of the dir should be okay right? | 16:03 |
| jmccarthy | Should it ? | 16:03 |
| jmccarthy | I only have 2 computes | 16:03 |
| jmccarthy | if I try to live migrate back I get the complaint about the dir being there | 16:03 |
| mriedem | jgwentworth: i don't think so https://github.com/openstack/nova/blob/4b0d0ea9f18139d58103a520a6a4e9119e19a4de/nova/virt/libvirt/driver.py#L7457 | 16:03 |
| jgwentworth | oh :\ | 16:04 |
| mriedem | jmccarthy: i guess you can use image type rbd for local storage and not having it be shared, but that seems weird | 16:04 |
| jmccarthy | 1) cold migrate - it works, but disk.info is left on source | 16:04 |
| mriedem | apparently people do it, but i'm not sure why you'd be using rbd and not sharing it | 16:04 |
| jmccarthy | 2) try to live back fails due to dir :/ | 16:04 |
| jgwentworth | then how is this supposed to work with shared storage | 16:04 |
| jmccarthy | mriedman: Not sure I follow, I'm using cinder for storage, not local ? | 16:04 |
| mriedem | the console.log is stored on local disk for the compute host | 16:05 |
| mriedem | not in cinder | 16:05 |
| jmccarthy | Well after the live fails then it does: Cleaning up deleted instances with incomplete migration | 16:05 |
| jmccarthy | _cleanup_incomplete_migrations | 16:05 |
| jmccarthy | and the next attempt to live migrate works | 16:05 |
| jgwentworth | okay yeah I see now, you are using rbd but not shared storage ... only does DestinationDiskExists if not shared | 16:05 |
| mriedem | so is it just me, or does https://review.openstack.org/#/c/437356/3/nova/virt/libvirt/driver.py@1136 seem like kind of a hack? | 16:06 |
| jmccarthy | mreidman: I'm confused, I do have a console log file there, but I am getting volumes also .. | 16:06 |
| Cardoe | mriedem: what would you like to see happen with the rescue stuff? We can try and submit a patch to allow it for xenapi? | 16:06 |
| mriedem | "self.image_backend.image for some backends recreates instance directory and image disk.info - remove it here if exists" | 16:06 |
| jgwentworth | mriedem: yeah, it does. when I read it I was like, "why is imagebackend sometimes recreating the dir" | 16:06 |
| mriedem | Cardoe: idk yet, that's why i posted to the ML | 16:06 |
| jmccarthy | s/getting/using | 16:06 |
| mriedem | those details are in https://bugs.launchpad.net/nova/+bug/1666831 | 16:07 |
| openstack | Launchpad bug 1666831 in OpenStack Compute (nova) ocata "Nova recreates instance directory after migration/resize" [Low,Fix committed] - Assigned to Lee Yarwood (lyarwood) | 16:07 |
| *** germs has quit IRC | 16:08 | |
| mriedem | apparently this recreates the disk.info on the source host https://github.com/openstack/nova/blob/4b0d0ea9f18139d58103a520a6a4e9119e19a4de/nova/virt/libvirt/driver.py#L1152 | 16:08 |
| mriedem | and then https://github.com/openstack/nova/blob/4b0d0ea9f18139d58103a520a6a4e9119e19a4de/nova/virt/libvirt/driver.py#L1170 was added to delete it | 16:08 |
| *** kholkina has quit IRC | 16:08 | |
| Cardoe | mriedem: I've been told that the reason why we didn't upstream the change is that there was no easy way to allow it only for xenapi at the time. | 16:09 |
| jmccarthy | For me this happens, and it's ok | 16:09 |
| jmccarthy | CMD "rm -rf /var/lib/nova/instances/371e669b-0f15-49f2-9a84-bd1e89f34294_resize" returned: 0 | 16:09 |
| jmccarthy | except then | 16:09 |
| *** EmilienM is now known as EvilienM | 16:09 | |
| jmccarthy | Lock "/var/lib/nova/instances/371e669b-0f15-49f2-9a84-bd1e89f34294/disk.info" acquired | 16:09 |
| jmccarthy | and Lock "/var/lib/nova/instances/371e669b-0f15-49f2-9a84-bd1e89f34294/disk.info" released by "nova.virt.libvirt.imagebackend.write_to_disk_info_file" | 16:10 |
| jmccarthy | recreates it from what I can tell ? | 16:10 |
| jgwentworth | yeah, I think so | 16:10 |
| *** fragatina has quit IRC | 16:10 | |
| *** armaan has quit IRC | 16:11 | |
| jgwentworth | Cardoe: is rebooting a rescued instance only safe or something that makes sense for xenapi? | 16:11 |
| mriedem | right, that's what bug 1666831 is saying, | 16:11 |
| openstack | bug 1666831 in OpenStack Compute (nova) ocata "Nova recreates instance directory after migration/resize" [Low,Fix committed] https://launchpad.net/bugs/1666831 - Assigned to Lee Yarwood (lyarwood) | 16:11 |
| mriedem | and was trying to fix | 16:11 |
| mriedem | to summarize, the libvirt driver and the imagebackend are totally inbred and shitty | 16:11 |
| mriedem | mdbooth: ^? | 16:11 |
| *** armaan has joined #openstack-nova | 16:11 | |
| mriedem | the driver needs the image backend for rbd to remove snapshots, | 16:11 |
| mriedem | but for qcow2 imagebackend it recreates the disk.info, | 16:12 |
| mriedem | so then the libvirt driver deletes the directory that imagebackend.by_name() for qcow2 created | 16:12 |
| mriedem | the control flow logic in the driver _cleanup_resize method is totally tightly coupled based on the image backend being used | 16:12 |
| jgwentworth | yeah ... sucks | 16:13 |
| *** fried_rice is now known as fried_rolls | 16:13 | |
| Cardoe | jgwentworth: No. It'd make sense for others to get it. How its been explained to me is that when someone screws up their grub or whatever bootloader and they can't boot their instance it can go into a rescue mode where their disk is attached to a read-only Linux instance where they can fix it themselves. Then when its rebooted it will go back to their original configuration, but hopefully this time with a fixed boot loader. | 16:13 |
| *** r-daneel has quit IRC | 16:13 | |
| *** r-daneel has joined #openstack-nova | 16:13 | |
| mriedem | Cardoe: so this isn't a case where the user literally calls the rescue API on the instance and then reboots it? | 16:14 |
| mriedem | it's server create -> fails -> xen rescue mode shenanigans -> user reboots the instance | 16:15 |
| Cardoe | create or software update | 16:15 |
| mriedem | "software update" being something that happens within the xen guest? | 16:15 |
| mriedem | that's not a nova term | 16:15 |
| mriedem | anyway, none of that sounds like it puts the instance vm_state into RESCUED | 16:16 |
| mriedem | which is what the compute manager code is checking | 16:16 |
| Cardoe | When the instance fails to boot, the rescue API can be called to put it into rescued. | 16:16 |
| jgwentworth | looks like cfriesen might have highlighted why reboot is rejected in his reply to the ML, reboot could make it impossible to unrescue the instance? | 16:16 |
| jgwentworth | "loss of original instance state" | 16:17 |
| mriedem | jgwentworth: jmccarthy: so i have an idea about how to handle this disk.info issue, | 16:17 |
| mriedem | let me wip something up here and we can take a gander | 16:17 |
| jmccarthy | mriedman: Ok cool :) | 16:17 |
| jgwentworth | mriedem: sounds good | 16:18 |
| mriedem | Cardoe: if you're subscribed to the openstack-dev ML, it would be cool if you could reply with details on the xenapi background in that thread | 16:19 |
| mriedem | i've got some code to break | 16:19 |
| jmccarthy | mriedman: I did try running the tox tests in this area, either I ran the wrong tests, or this setup isn't quite covered | 16:20 |
| Cardoe | I'm not. I certainly can as I get more info. sorry its a bad game of telephone with me. If you guys wanted to talk assembly and hypercalls I'm you're guy. APIs for orchestrating VMs not so much. | 16:20 |
| *** derekh has quit IRC | 16:21 | |
| *** sahid has quit IRC | 16:24 | |
| mriedem | i haven't wanted to talk assembly in 19 years | 16:25 |
| jmccarthy | Is it a misconfiguration on my part to have images_type = rbd if I'm not using ceph ? | 16:25 |
| mriedem | jmccarthy: well what local ephemeral storage on your compute hosts are you using for non-volume-backed instances? | 16:26 |
| jmccarthy | I don't have anything setup specifically, local disk on computes I guess ? | 16:26 |
| jgwentworth | I would think so, rbd == ceph | 16:26 |
| superdan | mriedem: do we not store network_request(s) today? | 16:26 |
| mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.images_type | 16:27 |
| mriedem | superdan: not in the request spec no | 16:27 |
| mriedem | nor bdms | 16:27 |
| superdan | mriedem: I mean at all | 16:27 |
| mriedem | correct | 16:27 |
| superdan | mriedem: don't we need that on rebuild/evac? or do we get it from neutron or something? | 16:27 |
| *** suresh12 has quit IRC | 16:27 | |
| mriedem | we have the network info cache / neutron | 16:27 |
| superdan | surely not just the info_cache at that point...? | 16:27 |
| mriedem | you can't request new ports on rebuild/evac | 16:27 |
| superdan | hrm | 16:27 |
| *** zhaochao has quit IRC | 16:28 | |
| superdan | mriedem: so, storing it in the reqspec.. that may potentially include port uuids that you booted with right? | 16:28 |
| mriedem | the original network request list woudn't have things attached to the instance after it was created either | 16:28 |
| superdan | mriedem: what happens if/when I attach a new port and detach the old one.. now the reqspec has a stale from-two-years-ago port uuid in it right? | 16:28 |
| mriedem | superdan: port id or network id yeah | 16:28 |
| mriedem | superdan: yeah, same could be said about the security groups right? | 16:29 |
| superdan | yes, although that seems less problematic to me, | 16:29 |
| mriedem | the spec says they aren't going to persist this anyway, just include it in the request spec is a middleman to get from the api to the scheduler | 16:29 |
| superdan | probably because port_uuids are somewhat ephemeral | 16:29 |
| *** fragatina has joined #openstack-nova | 16:29 | |
| mriedem | i'm fine with not persisting it | 16:29 |
| superdan | mriedem: okay in general I think it's confusing to put things in DB-persisted objects that we don't store in the DB | 16:30 |
| mriedem | but i was thinking for modeling, NetworkRequest object gains a RequestGroup field, and then RequestSpec gains a (non-persisted) NetworkRequestList field | 16:30 |
| superdan | someone sets something there and calls save(), no error, so expects it's saved now, but isn't | 16:30 |
| superdan | what is requestgroup? | 16:30 |
| mriedem | ok, gibi has an alternative in the spec that it doesn't go into the request spec at all, and it's passed as a separate param to select_destinations() | 16:30 |
| mriedem | it's in the placement lib - contains things about a granular request group | 16:31 |
| jmccarthy | Ah sorry ! I was looking at a template, not the final rendered file | 16:31 |
| superdan | oh, that right | 16:31 |
| mriedem | so they'd have one of those per port | 16:31 |
| jmccarthy | For my deployed nova.conf there is no entry there for images_type at all (so I guess default?) | 16:31 |
| superdan | mriedem: well, a new rpc argument to the scheduler just for network stuff is also kinda odd, which I guess was your point | 16:31 |
| mriedem | if it doesn't go in the request spec because we don't persist it, then it's a new param to select_destinations() | 16:31 |
| mriedem | jgwentworth: so i don't think my idea will work..drats | 16:32 |
| *** dtantsur is now known as dtantsur|afk | 16:32 | |
| mriedem | jgwentworth: my idea was just to check https://github.com/openstack/nova/blob/4b0d0ea9f18139d58103a520a6a4e9119e19a4de/nova/virt/libvirt/imagebackend.py#L57 in driver._cleanup_resize and if True, do the snapshot removal stuff that is rbd-specific | 16:32 |
| jgwentworth | *sad trombone* | 16:33 |
| mriedem | but to get that value, we have to init the imagebackend object which is the thing that recreates the gd disk.info file | 16:33 |
| *** artom_ is now known as notartom | 16:33 | |
| mriedem | so, we could make a module-level dict of image types that support clone and use that, | 16:33 |
| mriedem | or pass a flag to imagebackend init to tell it to not touch the filesystem | 16:33 |
| mriedem | former seems easier | 16:34 |
| mriedem | or hell, just: if CONF.libvirt.images_type == 'rbd': in _cleanup_resize | 16:34 |
| mriedem | that's essentially what we'd be doing if we check SUPPORTS_CLONE | 16:34 |
| jgwentworth | do you mean change the check for is_volume_backed to essentially "is_shared_storage"? | 16:37 |
| *** eharney has quit IRC | 16:38 | |
| mriedem | that entire block goes away | 16:38 |
| jmccarthy | <back asap - afk> | 16:39 |
| mriedem | i.e. basically undo this https://review.openstack.org/#/c/437356/3/nova/virt/libvirt/driver.py | 16:39 |
| mriedem | and fix the logic | 16:39 |
| jgwentworth | oh, I see, yeah | 16:39 |
| jgwentworth | instead of using root_disk.exists() | 16:39 |
| mriedem | "if os.path.exists(inst_base) and not root_disk.exists():" was added for qcow2/flat/ploop because those will recreate the instance dir and disk.info on init, | 16:39 |
| mriedem | when all we needed the imagebackend for was the remove_snap call, | 16:40 |
| mriedem | which is rbd-specific | 16:40 |
| jgwentworth | right | 16:40 |
| jgwentworth | gotcha | 16:40 |
| mriedem | so if we just don't create the imagebackend object in the first place, we avoid the init et al | 16:40 |
| mriedem | \o/ | 16:40 |
| jgwentworth | yeah, seriously | 16:40 |
| mriedem | i of course will require jmccarthy to test the patch in his setup | 16:40 |
| jgwentworth | oh, because you have to use it to do the remove_snap | 16:41 |
| mriedem | yup and that's pointed out in https://bugs.launchpad.net/nova/+bug/1666831 | 16:41 |
| openstack | Launchpad bug 1666831 in OpenStack Compute (nova) ocata "Nova recreates instance directory after migration/resize" [Low,Fix committed] - Assigned to Lee Yarwood (lyarwood) | 16:41 |
| mriedem | "root_disk is used to remove rdb snapshots, but during execution of self.image_backend.by_name() nova recreates instance directory." | 16:41 |
| jgwentworth | so how can we remove_snap without the imagebackend object? | 16:41 |
| mriedem | so in his case, he wasn't even using rbd, he was using qcow2 | 16:41 |
| mriedem | we can, we'll get it if the backend supports clone | 16:42 |
| mriedem | remove_snap is only implemented for image backends that support clone | 16:42 |
| jgwentworth | ohhhh k | 16:42 |
| mriedem | and that's only rb | 16:42 |
| mriedem | *rbd | 16:42 |
| *** sridharg has quit IRC | 16:42 | |
| *** AlexeyAbashkin has joined #openstack-nova | 16:43 | |
| jgwentworth | yeah, makes sense | 16:43 |
| *** moshele has joined #openstack-nova | 16:43 | |
| openstackgerrit | Merged openstack/nova master: Convert websocketproxy to use db for token validation https://review.openstack.org/333990 | 16:44 |
| *** armaan has quit IRC | 16:45 | |
| mriedem | and because i added that tempest test, we should be testing a resize of a volume-backed instance on shared local storage since we have the NFS job in nova's experimental queue | 16:45 |
| mriedem | plus CEPH | 16:45 |
| jgwentworth | *mind blown* | 16:48 |
| *** tbachman has quit IRC | 16:48 | |
| *** udesale has quit IRC | 16:50 | |
| *** tbachman has joined #openstack-nova | 16:50 | |
| *** tbachman has quit IRC | 16:51 | |
| *** moshele has quit IRC | 16:52 | |
| jmccarthy | back | 16:54 |
| *** armaan has joined #openstack-nova | 16:55 | |
| *** jpena is now known as jpena|off | 16:55 | |
| *** mdbooth has quit IRC | 16:56 | |
| *** mvk has joined #openstack-nova | 16:58 | |
| *** mgoddard has quit IRC | 17:04 | |
| jmccarthy | mriedem: I can test any stuff no probs ! It might take a little while, but hopefully not too long | 17:07 |
| *** tesseract has quit IRC | 17:09 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: check image type before removing snapshots in _cleanup_resize https://review.openstack.org/566367 | 17:12 |
| mriedem | jmccarthy: jgwentworth: ^ | 17:12 |
| jmccarthy | mriedman: Nice ! | 17:12 |
| jmccarthy | mriedman: Let me see about working that in | 17:12 |
| *** itlinux has joined #openstack-nova | 17:14 | |
| *** esberglu has quit IRC | 17:17 | |
| *** germs has joined #openstack-nova | 17:18 | |
| *** germs has quit IRC | 17:18 | |
| *** germs has joined #openstack-nova | 17:18 | |
| *** tbachman has joined #openstack-nova | 17:19 | |
| *** germs has quit IRC | 17:19 | |
| *** germs has joined #openstack-nova | 17:19 | |
| openstackgerrit | Merged openstack/nova master: Add multi-cell negative test for cold migration with target host https://review.openstack.org/524027 | 17:21 |
| openstackgerrit | Merged openstack/nova master: Update layout docs for running console proxies https://review.openstack.org/557489 | 17:21 |
| *** sq4ind has quit IRC | 17:23 | |
| *** sq4ind has joined #openstack-nova | 17:27 | |
| *** mgoddard has joined #openstack-nova | 17:36 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: remove old rbd snapshot removal error handling https://review.openstack.org/566369 | 17:37 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: remove old rbd snapshot removal error handling https://review.openstack.org/566369 | 17:38 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: rework how we pass candidate request information https://review.openstack.org/566166 | 17:39 |
| idlemind | jgwentworth thx i'm using lvm via iscsi (simple stuff for now) | 17:39 |
| *** suresh12 has joined #openstack-nova | 17:40 | |
| *** suresh12_ has joined #openstack-nova | 17:48 | |
| jmccarthy | mriedman: Ok I thought this might happen - I'm going to need a while longer, unfortunately it's a long weekend here - I'll update the bug asap but it may not be until Tuesday | 17:49 |
| *** yamamoto has joined #openstack-nova | 17:50 | |
| jmccarthy | mriedman: I appreciate your quick efforts ! I'm going to keep at it another while | 17:51 |
| *** suresh12 has quit IRC | 17:51 | |
| *** evin has joined #openstack-nova | 17:52 | |
| *** pcaruana has joined #openstack-nova | 17:54 | |
| *** gjayavelu has joined #openstack-nova | 17:54 | |
| *** yamamoto has quit IRC | 17:54 | |
| *** eharney has joined #openstack-nova | 17:56 | |
| *** dpawlik has joined #openstack-nova | 17:58 | |
| *** dpawlik has quit IRC | 18:03 | |
| mriedem | jmccarthy: sure np | 18:15 |
| pipesinpain | mriedem, jgwentworth, johnthetubaguy, alex_xu: your eyeballs on https://review.openstack.org/#/c/565565/ would be appreciated. | 18:15 |
| *** fragatina has quit IRC | 18:18 | |
| *** janki has quit IRC | 18:19 | |
| arvindn05 | mriedem: can you look over the spec amendment? https://review.openstack.org/#/c/560718/ | 18:21 |
| arvindn05 | once its approved, hoping the scheduler patch can be upstreamed | 18:21 |
| *** AlexeyAbashkin has quit IRC | 18:30 | |
| *** owalsh is now known as owalsh-afk | 18:32 | |
| *** mgoddard has quit IRC | 18:38 | |
| *** gyankum has quit IRC | 18:49 | |
| *** fried_rolls is now known as fried_rice | 18:49 | |
| jgwentworth | pipesinpain: ack, it's on my list | 18:53 |
| *** imacdonn has quit IRC | 18:54 | |
| *** imacdonn has joined #openstack-nova | 18:55 | |
| * jgwentworth will bbl | 18:57 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766 | 18:59 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Isolate placement database config https://review.openstack.org/541435 | 18:59 |
| openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Ensure that os-traits sync is attempted only at start of process https://review.openstack.org/553857 | 18:59 |
| openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Add PLACEMENT_DB_ENABLED=True to the nova-next job https://review.openstack.org/564067 | 19:00 |
| *** liverpooler has quit IRC | 19:00 | |
| *** sq4ind has quit IRC | 19:04 | |
| pipesinpain | jgwentworth: thx Melanie. | 19:06 |
| openstackgerrit | Merged openstack/nova master: Base test module/class for functional placement db https://review.openstack.org/564590 | 19:07 |
| *** sq4ind has joined #openstack-nova | 19:08 | |
| *** pchavva has quit IRC | 19:09 | |
| *** pcaruana has quit IRC | 19:10 | |
| *** sq4ind has quit IRC | 19:10 | |
| *** sq4ind has joined #openstack-nova | 19:14 | |
| *** tbachman has quit IRC | 19:14 | |
| eandersson | mriedem, I was gonna do a quick pull request to change the log "Successfully synced instances from host '%s'." to DEBUG, but noticed that it's still doing log translation | 19:19 |
| eandersson | Can I remove the log translation in the same commit? | 19:20 |
| *** jmccarthy has left #openstack-nova | 19:20 | |
| *** mgoddard has joined #openstack-nova | 19:22 | |
| mriedem | eandersson: yes | 19:24 |
| mriedem | we don't translate logs anymore | 19:24 |
| eandersson | Can I remove the log translation for all the entries in that file? :D or will that make it too difficult to see what changed | 19:24 |
| eandersson | I can also follow up with a new pull request for that :D | 19:24 |
| mriedem | i'd keep those separate | 19:25 |
| openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/nova master: Changing scheduler sync event from INFO to DEBUG https://review.openstack.org/566392 | 19:27 |
| eandersson | I don't even want to know how much disk space is wasted on that one log line :D | 19:29 |
| *** mgoddard has quit IRC | 19:29 | |
| mriedem | easiest -1 ever | 19:31 |
| eandersson | haha | 19:31 |
| eandersson | omg | 19:31 |
| eandersson | was too fixated on the translation | 19:31 |
| eandersson | good, get to fix that missing e in computes | 19:31 |
| kashyap | mriedem: This is largely code deletion, should be easy for you: https://review.openstack.org/#/c/565242/ | 19:32 |
| openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/nova master: Changing scheduler sync event from INFO to DEBUG https://review.openstack.org/566392 | 19:32 |
| * kashyap expects perhaps one more iteration from potential feedback from Matt | 19:32 | |
| mriedem | eandersson: off the top of your head, what's the average number of cpus in your compute hosts? | 19:33 |
| mriedem | 16? 32? | 19:34 |
| eandersson | 32 probably | 19:35 |
| mriedem | ok. so you wouldn't run something like 132 concurrent live migrations on a compute host like that would you. | 19:35 |
| mriedem | 32 * 5 | 19:35 |
| eandersson | Yea unlikely | 19:35 |
| mriedem | cool, fyi https://docs.python.org/3.5/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor | 19:36 |
| mriedem | your logging change is sane btw, check out a 24 hour CI run for that log message http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Successfully%20synced%20instances%20from%20host%5C%22%20AND%20tags%3A%5C%22screen-n-sch.txt%5C%22&from=1d | 19:39 |
| mriedem | ~16K hits | 19:39 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: Add columns for generation to consumers https://review.openstack.org/557958 | 19:55 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: placement: add Project, User and Consumer objects https://review.openstack.org/565403 | 19:55 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: Add create() methods to Project, User and Consumer https://review.openstack.org/565404 | 19:55 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: move consumer ensure to API layer https://review.openstack.org/565405 | 19:55 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: remove Allocation.project_id & Allocation.user_id https://review.openstack.org/565406 | 19:55 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: rework allocation handler _allocations_dict() https://review.openstack.org/565407 | 19:55 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: Add a microversion for consumer generation support https://review.openstack.org/565604 | 19:55 |
| *** dpawlik has joined #openstack-nova | 19:59 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: Add InstanceNUMATopology to LibvirtLiveMigrateData https://review.openstack.org/566398 | 20:01 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: Add InstanceNUMATopology to LibvirtLiveMigrateData https://review.openstack.org/566398 | 20:03 |
| *** dpawlik has quit IRC | 20:04 | |
| *** brault has joined #openstack-nova | 20:07 | |
| *** notartom has quit IRC | 20:09 | |
| *** brault has quit IRC | 20:11 | |
| arvindn05 | mriedem: question on tests cases within conductor | 20:15 |
| arvindn05 | why are we running few tests multiple times? ConductorTaskAPITestCase,ConductorTaskRPCAPITestCase extend from _BaseTaskTestCase and test_compute.BaseTestCase | 20:17 |
| arvindn05 | 20:17 | |
| arvindn05 | they both run test_unshelve_instance_on_host for example...which is defined under the BaseTaskTestCase. i dont see any differences in the test setup either... | 20:18 |
| mriedem | one hits the rpcapi and one doesn't | 20:19 |
| *** armaan has quit IRC | 20:23 | |
| arvindn05 | ahh...noticed the difference in setups now...the difference is very subtle...will add a line comment on top of self.conductor so it draws the difference for future contributors | 20:23 |
| *** armaan has joined #openstack-nova | 20:23 | |
| arvindn05 | btw i investigated further on code changes...wanted to confirm something with you | 20:25 |
| arvindn05 | mriedem: i would need to add an else condition here as well correct? to handle rebuild with image remaining the samehttps://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L913 | 20:26 |
| *** pchavva has joined #openstack-nova | 20:26 | |
| arvindn05 | see http://paste.openstack.org/show/720407/ for how i think the code would need to look....let me know if this is not consistent with your idea and i can rework as needed | 20:29 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Use ThreadPoolExecutor for max_concurrent_live_migrations https://review.openstack.org/563505 | 20:31 |
| mriedem | superdan: huzzah ^ that's not too bad | 20:31 |
| mriedem | arvindn05: i said else after the elif here https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L943 | 20:31 |
| mriedem | arvindn05: idk, do we need to re-validate the host/image for rebuild if the image doesn't change? | 20:31 |
| mriedem | i guess that's just to catch a case that the traits on the compute node resource provider changed even though the image didn't? | 20:32 |
| arvindn05 | yup...melwitt suggested we use the same approach as imageproperties filter....which i beleive has the same behavior | 20:32 |
| arvindn05 | we run even though image hasnt changed | 20:32 |
| arvindn05 | i added the else as well @943 see http://paste.openstack.org/show/720407 - line58 | 20:34 |
| *** fishbone_ has quit IRC | 20:34 | |
| arvindn05 | briefly tested the approach...it works...adding more unit tests | 20:35 |
| superdan | mriedem: looks weird to index by migration uuid instead of instance, is that because of how the cancel migration call works? | 20:36 |
| *** gbarros has quit IRC | 20:37 | |
| mriedem | i think it could go either way, live_migration_abort gets the instance and migration | 20:37 |
| mriedem | kevin just doesn't have that patch up yet | 20:38 |
| superdan | okay I'll make a half-assed comment about it | 20:39 |
| mriedem | quarter cheek friday please | 20:40 |
| *** mchlumsky has quit IRC | 20:42 | |
| *** fragatina has joined #openstack-nova | 20:43 | |
| *** fragatina has quit IRC | 20:43 | |
| mriedem | arvindn05: that's not correct, | 20:44 |
| mriedem | for rebuild, if the image doesn't change, we don't run through the scheduler, so we don't run the ImagePropertiesFilter | 20:44 |
| mriedem | see https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L898 | 20:44 |
| mriedem | if host != instance.host: will be False in that block for rebuild | 20:44 |
| *** fragatina has joined #openstack-nova | 20:45 | |
| mriedem | so, idk, that's new validation in the rebuild + same image case, i don't know if we need to revalidate the image in that case, we didn't before, but i also don't really care all that much | 20:45 |
| *** fragatina has quit IRC | 20:46 | |
| arvindn05 | ok...i guess it can be handled as part of CR | 20:46 |
| *** fragatina has joined #openstack-nova | 20:46 | |
| mriedem | arvindn05: +2 on the spec amendment | 20:50 |
| mriedem | time for you to pop a bottle | 20:50 |
| mriedem | of pills | 20:50 |
| arvindn05 | awesome...thanks! :) | 20:50 |
| arvindn05 | mriedem: see here on melwitt comments http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2018-05-03.log.html#t2018-05-03T20:48:55 | 20:54 |
| arvindn05 | i think if we are not running imageproperties filter now when image does not change we might want to be consistent...i can raise a bug to state that we dont run image properties filter if image hasnt changed(even though image properties on that image could have changed) | 20:55 |
| arvindn05 | and when its fixed...they can fix to handle traits as well....another way is if i just need to call schedule instances for running the image properties filter...i can do that as part of this change | 20:57 |
| mriedem | please no | 21:04 |
| mriedem | i think if we don't call the scheduler (ImagePropertiesFilter) today, we should just continue to not do the validation | 21:05 |
| mriedem | i.e. image doesn't change | 21:05 |
| arvindn05 | sounds like a plan....although a bug might be warranted at that point.... | 21:07 |
| mriedem | i don't know that it's a bug | 21:07 |
| mriedem | but it's 4pm on a friday, so i also don't care to think about this right now | 21:07 |
| arvindn05 | in case of rebuild with a trait like Trusted host...if the host is no longer trusted we should prevent rebuild...might be a security issue | 21:07 |
| arvindn05 | but like you said...its friday...we can figure it out in the CR :) | 21:08 |
| mriedem | is trusted host a standard trait? | 21:08 |
| arvindn05 | any plans for the weekend? | 21:08 |
| mriedem | yardwork, mexican food and avoiding little girls | 21:08 |
| *** burt has quit IRC | 21:09 | |
| mriedem | i.e. my daughters annoying friends, not pervy stuff | 21:09 |
| arvindn05 | nope....its a custom trait...but traits could be used for security things as well | 21:09 |
| arvindn05 | lol....good job with the quick clarification :P | 21:09 |
| openstackgerrit | Jay Pipes proposed openstack/nova master: rework how we pass candidate request information https://review.openstack.org/566166 | 21:09 |
| mriedem | if an external service says this host is trusted or not and there are instances using an image that requires a trusted host, then i'd expect that external service to migrate those instances before saying it's no longer trusted | 21:10 |
| mriedem | not waiting for the user to rebuild and find out | 21:10 |
| mriedem | the instance user is not an admin so they can't see what the traits are on the node RP anyway | 21:10 |
| arvindn05 | i guess there are 2 models...there is just a trust attestation piece which does not do any active managment...but more for reporting...another could be with active management like you mentioned | 21:11 |
| arvindn05 | anyway...will leave you to enjoy your friday and the weekend :) | 21:12 |
| *** arvindn05 is now known as arvindn05_away | 21:12 | |
| mriedem | i would personally prefer that nova doesn't grow a bunch of side effect code to handle random traits-related what-if kind of conditionals because of the ability of external services to monkey with traits on compute node providers | 21:12 |
| mriedem | and with that, i'm out also, ttyl | 21:13 |
| *** mriedem is now known as mriedem_yard | 21:13 | |
| arvindn05_away | ttyl | 21:13 |
| *** notartom has joined #openstack-nova | 21:19 | |
| *** dave-mccowan has quit IRC | 21:19 | |
| *** felipemonteiro has quit IRC | 21:20 | |
| *** pchavva has quit IRC | 21:29 | |
| *** awaugama has quit IRC | 21:29 | |
| *** edmondsw has quit IRC | 21:47 | |
| *** itlinux has quit IRC | 21:47 | |
| *** edmondsw has joined #openstack-nova | 21:47 | |
| *** brad[] has quit IRC | 21:50 | |
| *** hamzy_ has joined #openstack-nova | 21:51 | |
| *** edmondsw has quit IRC | 21:52 | |
| *** damien_r has joined #openstack-nova | 21:53 | |
| *** liverpooler has joined #openstack-nova | 21:54 | |
| *** hamzy has quit IRC | 21:54 | |
| *** wolverineav has quit IRC | 21:54 | |
| *** figleaf is now known as edleafe | 21:54 | |
| *** wolverineav has joined #openstack-nova | 21:55 | |
| *** Guest36566 has quit IRC | 21:57 | |
| *** moshele has joined #openstack-nova | 21:58 | |
| *** wolverineav has quit IRC | 21:59 | |
| *** fragatina has quit IRC | 22:00 | |
| *** slaweq has quit IRC | 22:03 | |
| *** moshele has quit IRC | 22:03 | |
| *** moshele has joined #openstack-nova | 22:04 | |
| *** abalutoiu_ has joined #openstack-nova | 22:08 | |
| *** ccamacho has quit IRC | 22:10 | |
| *** abalutoiu__ has quit IRC | 22:11 | |
| *** ccamacho has joined #openstack-nova | 22:12 | |
| *** moshele has quit IRC | 22:14 | |
| *** damien_r has quit IRC | 22:15 | |
| *** evin has quit IRC | 22:17 | |
| *** r-daneel has quit IRC | 22:21 | |
| *** fragatina has joined #openstack-nova | 22:24 | |
| *** markvoelker has quit IRC | 22:29 | |
| *** brad[] has joined #openstack-nova | 22:34 | |
| *** fragatina has quit IRC | 22:34 | |
| *** fragatina has joined #openstack-nova | 22:35 | |
| *** pchavva has joined #openstack-nova | 22:35 | |
| *** liverpooler has quit IRC | 22:40 | |
| *** wolverineav has joined #openstack-nova | 22:41 | |
| *** imacdonn has quit IRC | 22:42 | |
| *** fragatina has quit IRC | 22:45 | |
| *** fragatina has joined #openstack-nova | 22:45 | |
| *** wolverineav has quit IRC | 22:45 | |
| *** wolverineav has joined #openstack-nova | 22:49 | |
| *** linkmark has joined #openstack-nova | 22:52 | |
| *** wolverineav has quit IRC | 22:53 | |
| *** fried_rice is now known as efried | 22:57 | |
| *** hongbin has quit IRC | 23:01 | |
| *** wolverineav has joined #openstack-nova | 23:03 | |
| *** wolverineav has quit IRC | 23:05 | |
| *** wolverineav has joined #openstack-nova | 23:06 | |
| *** wolverineav has quit IRC | 23:10 | |
| *** slaweq has joined #openstack-nova | 23:11 | |
| *** krtaylor has quit IRC | 23:13 | |
| *** fragatina has quit IRC | 23:14 | |
| *** slaweq has quit IRC | 23:15 | |
| *** yamamoto has joined #openstack-nova | 23:21 | |
| *** pchavva has quit IRC | 23:22 | |
| *** wolverineav has joined #openstack-nova | 23:26 | |
| *** yamamoto has quit IRC | 23:26 | |
| *** gjayavelu has quit IRC | 23:31 | |
| *** wolverineav has quit IRC | 23:31 | |
| *** wolverineav has joined #openstack-nova | 23:34 | |
| *** wolverineav has quit IRC | 23:39 | |
| *** wolverineav has joined #openstack-nova | 23:40 | |
| *** wolverineav has quit IRC | 23:40 | |
| *** wolverineav has joined #openstack-nova | 23:40 | |
| *** wolverin_ has joined #openstack-nova | 23:43 | |
| *** wolverineav has quit IRC | 23:45 | |
| *** tbachman has joined #openstack-nova | 23:45 | |
| *** wolverin_ has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
