| *** markvoelker has quit IRC | 00:09 | |
| *** liuyulong has quit IRC | 00:31 | |
| *** bhagyashris has joined #openstack-nova | 01:03 | |
| *** markvoelker has joined #openstack-nova | 01:05 | |
| *** lbragstad has quit IRC | 01:10 | |
| *** lifeless has joined #openstack-nova | 01:14 | |
| *** _erlon_ has quit IRC | 01:17 | |
| *** markvoelker has quit IRC | 01:39 | |
| *** boxiang has joined #openstack-nova | 01:45 | |
| *** boxiang_ has quit IRC | 01:47 | |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Introduces SDK to IronicDriver and uses for node.get https://review.opendev.org/642899 | 02:00 |
|---|---|---|
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK instead of ironicclient for node.list https://review.opendev.org/656027 | 02:00 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Use SDK instead of ironicclient for validating instance and node https://review.opendev.org/656028 | 02:00 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Use SDK instead of ironicclient for setting instance id https://review.opendev.org/659690 | 02:00 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Use SDK instead of ironicclient for add/remove instance info from node https://review.opendev.org/659691 | 02:00 |
| *** gyee has quit IRC | 02:11 | |
| *** licanwei has joined #openstack-nova | 02:13 | |
| *** whoami-rajat has joined #openstack-nova | 02:14 | |
| *** brinzhang has joined #openstack-nova | 02:17 | |
| *** BjoernT_ has quit IRC | 02:24 | |
| *** markvoelker has joined #openstack-nova | 02:36 | |
| *** guozijn has joined #openstack-nova | 02:38 | |
| *** lifeless has quit IRC | 02:38 | |
| *** boxiang has quit IRC | 02:39 | |
| *** boxiang has joined #openstack-nova | 02:39 | |
| *** lifeless has joined #openstack-nova | 02:44 | |
| *** cfriesen has quit IRC | 03:00 | |
| *** guozijn has quit IRC | 03:02 | |
| *** BjoernT has joined #openstack-nova | 03:06 | |
| *** JamesBenson has joined #openstack-nova | 03:09 | |
| *** markvoelker has quit IRC | 03:09 | |
| *** BjoernT has quit IRC | 03:35 | |
| *** ricolin has joined #openstack-nova | 03:50 | |
| *** dave-mccowan has quit IRC | 03:54 | |
| *** boxiang has quit IRC | 04:00 | |
| *** kaisers has quit IRC | 04:06 | |
| *** markvoelker has joined #openstack-nova | 04:06 | |
| *** igordc has joined #openstack-nova | 04:07 | |
| *** kaisers has joined #openstack-nova | 04:21 | |
| *** whoami-rajat has quit IRC | 04:23 | |
| *** JamesBenson has quit IRC | 04:24 | |
| *** markvoelker has quit IRC | 04:40 | |
| *** udesale has joined #openstack-nova | 04:41 | |
| *** janki has joined #openstack-nova | 04:52 | |
| *** ratailor has joined #openstack-nova | 05:00 | |
| *** pcaruana|afk| has joined #openstack-nova | 05:01 | |
| *** pcaruana|afk| has quit IRC | 05:04 | |
| *** pcaruana has joined #openstack-nova | 05:04 | |
| *** maciejjozefczyk has joined #openstack-nova | 05:10 | |
| *** ivve has joined #openstack-nova | 05:10 | |
| *** Luzi has joined #openstack-nova | 05:20 | |
| *** guozijn has joined #openstack-nova | 05:27 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Make RequestContext(instance_lock_checked) fail https://review.opendev.org/665003 | 05:35 |
| *** ricolin has quit IRC | 05:36 | |
| *** markvoelker has joined #openstack-nova | 05:37 | |
| *** takamatsu has joined #openstack-nova | 05:42 | |
| *** luksky has joined #openstack-nova | 05:45 | |
| *** jaypipes has quit IRC | 05:47 | |
| *** guozijn has quit IRC | 05:49 | |
| *** guozijn has joined #openstack-nova | 05:52 | |
| openstackgerrit | ya.wang proposed openstack/nova-specs master: Add spec for expose-auto-converge-post-copy https://review.opendev.org/651681 | 05:53 |
| openstackgerrit | Merged openstack/nova master: Remove cells v1 parameter from 'ComputeTaskAPI.resize_instance' https://review.opendev.org/651311 | 05:56 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix a warning about flags in an expression string https://review.opendev.org/665065 | 06:03 |
| openstackgerrit | Merged openstack/nova master: libvirt: Use SATA bus for cdrom devices when using Q35 machine type https://review.opendev.org/663011 | 06:04 |
| *** guozijn has quit IRC | 06:07 | |
| *** spsurya has joined #openstack-nova | 06:07 | |
| *** markvoelker has quit IRC | 06:10 | |
| *** zbr|flow is now known as zbr|ooo | 06:11 | |
| *** jaypipes has joined #openstack-nova | 06:11 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (19) https://review.opendev.org/576676 | 06:20 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (20) https://review.opendev.org/576689 | 06:21 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (21) https://review.opendev.org/576709 | 06:21 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (22) https://review.opendev.org/576712 | 06:21 |
| *** brinzh has joined #openstack-nova | 06:23 | |
| *** brinzhang has quit IRC | 06:26 | |
| *** guozijn has joined #openstack-nova | 06:27 | |
| *** hamdyk has joined #openstack-nova | 06:28 | |
| *** vsk_17 has joined #openstack-nova | 06:49 | |
| *** xek_ has joined #openstack-nova | 06:53 | |
| *** rpittau|afk is now known as rpittau | 07:00 | |
| *** markvoelker has joined #openstack-nova | 07:08 | |
| *** dtantsur|afk is now known as dtantsur | 07:10 | |
| *** slaweq has joined #openstack-nova | 07:12 | |
| *** markvoelker has quit IRC | 07:13 | |
| *** tssurya has joined #openstack-nova | 07:13 | |
| *** igordc has quit IRC | 07:15 | |
| *** ccamacho has quit IRC | 07:15 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Validate requested host/node during servers create https://review.opendev.org/661237 | 07:18 |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Add host and hypervisor_hostname flag to create server https://review.opendev.org/645520 | 07:18 |
| *** tesseract has joined #openstack-nova | 07:21 | |
| bhagyashris | sean-k-mooney: Hi, | 07:22 |
| *** boxiang has joined #openstack-nova | 07:26 | |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Make evacuation respects anti-affinity rule https://review.opendev.org/649963 | 07:31 |
| *** jangutter has joined #openstack-nova | 07:31 | |
| *** helenafm has joined #openstack-nova | 07:31 | |
| *** trident has quit IRC | 07:31 | |
| *** takamatsu has quit IRC | 07:32 | |
| *** trident has joined #openstack-nova | 07:34 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: libvirt: Use SATA bus for cdrom devices when using Q35 machine type https://review.opendev.org/663677 | 07:35 |
| openstackgerrit | Boxiang Zhu proposed openstack/nova master: Fix live migration break group policy simultaneously https://review.opendev.org/651969 | 07:36 |
| *** mkrai has joined #openstack-nova | 07:40 | |
| *** guozijn has quit IRC | 07:40 | |
| *** mkrai_ has joined #openstack-nova | 07:41 | |
| mkrai_ | Hi Nova Developers, I request you to provide your input on https://etherpad.openstack.org/p/ironic-nova-reset-configuration | 07:41 |
| *** awalende has joined #openstack-nova | 07:43 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: WIP: Specify availability_zone to unshelve https://review.opendev.org/663851 | 07:44 |
| yaawang | sean-k-mooney: johnthetubaguy Hi, I've updated this spec, please take a look if you have time. https://review.opendev.org/#/c/651681 | 07:55 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: WIP: Specify availability_zone to unshelve https://review.opendev.org/663851 | 07:59 |
| *** guozijn has joined #openstack-nova | 08:09 | |
| *** ttsiouts has joined #openstack-nova | 08:09 | |
| *** markvoelker has joined #openstack-nova | 08:10 | |
| *** whoami-rajat has joined #openstack-nova | 08:11 | |
| *** sridharg has joined #openstack-nova | 08:13 | |
| *** markvoelker has quit IRC | 08:14 | |
| *** ralonsoh has joined #openstack-nova | 08:15 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix cleaning up console tokens https://review.opendev.org/637716 | 08:17 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add a live migration regression test https://review.opendev.org/641200 | 08:17 |
| *** takamatsu has joined #openstack-nova | 08:19 | |
| *** mdbooth has joined #openstack-nova | 08:20 | |
| *** ttsiouts has quit IRC | 08:20 | |
| *** ttsiouts has joined #openstack-nova | 08:21 | |
| *** rcernin has quit IRC | 08:23 | |
| *** jcosmao has left #openstack-nova | 08:24 | |
| *** jcosmao has joined #openstack-nova | 08:24 | |
| *** ttsiouts has quit IRC | 08:25 | |
| openstackgerrit | ya.wang proposed openstack/nova-specs master: Re-propose the spec to allow specifying a list of CPU models https://review.opendev.org/642030 | 08:26 |
| *** ccamacho has joined #openstack-nova | 08:27 | |
| *** priteau has joined #openstack-nova | 08:28 | |
| *** cdent has joined #openstack-nova | 08:28 | |
| *** jaosorior has joined #openstack-nova | 08:29 | |
| *** ociuhandu has joined #openstack-nova | 08:31 | |
| *** ccamacho has quit IRC | 08:32 | |
| *** ccamacho has joined #openstack-nova | 08:33 | |
| *** ttsiouts has joined #openstack-nova | 08:34 | |
| *** ociuhandu has quit IRC | 08:36 | |
| *** imacdonn has quit IRC | 08:38 | |
| *** imacdonn has joined #openstack-nova | 08:39 | |
| *** luksky has quit IRC | 08:39 | |
| *** dpawlik has joined #openstack-nova | 08:40 | |
| *** ociuhandu has joined #openstack-nova | 08:42 | |
| *** mdbooth has quit IRC | 08:47 | |
| *** mdbooth has joined #openstack-nova | 08:48 | |
| *** mdbooth_ has joined #openstack-nova | 08:48 | |
| *** tkajinam has quit IRC | 08:51 | |
| *** mdbooth has quit IRC | 08:53 | |
| hjensas | Hi Nova, anyone who can review (my first?) ever nova change - https://review.opendev.org/664559 | 08:55 |
| *** brinzhang has joined #openstack-nova | 09:04 | |
| *** brinzh has quit IRC | 09:06 | |
| *** mdbooth has joined #openstack-nova | 09:06 | |
| *** helenafm has quit IRC | 09:07 | |
| *** mdbooth has quit IRC | 09:09 | |
| *** boxiang has quit IRC | 09:10 | |
| *** markvoelker has joined #openstack-nova | 09:10 | |
| *** markvoelker has quit IRC | 09:15 | |
| *** edleafe has quit IRC | 09:22 | |
| *** luksky has joined #openstack-nova | 09:22 | |
| *** mdbooth has joined #openstack-nova | 09:25 | |
| *** mdbooth_ has quit IRC | 09:27 | |
| *** derekh has joined #openstack-nova | 09:31 | |
| *** takashin has left #openstack-nova | 09:33 | |
| *** gmann has quit IRC | 09:33 | |
| *** mkrai_ has quit IRC | 09:37 | |
| *** slaweq has quit IRC | 09:45 | |
| *** slaweq has joined #openstack-nova | 09:46 | |
| *** helenafm has joined #openstack-nova | 09:49 | |
| *** udesale has quit IRC | 09:59 | |
| *** udesale has joined #openstack-nova | 09:59 | |
| *** ttsiouts has quit IRC | 10:00 | |
| *** ttsiouts has joined #openstack-nova | 10:00 | |
| *** maciejjozefczyk has quit IRC | 10:00 | |
| *** maciejjozefczyk has joined #openstack-nova | 10:01 | |
| *** bhagyashris has quit IRC | 10:04 | |
| *** ttsiouts has quit IRC | 10:05 | |
| *** hamdyk has quit IRC | 10:05 | |
| *** hamdyk has joined #openstack-nova | 10:05 | |
| *** markvoelker has joined #openstack-nova | 10:11 | |
| *** markvoelker has quit IRC | 10:15 | |
| *** mdbooth has quit IRC | 10:25 | |
| gibi | hjensas: I'm on it. thanks for working on it | 10:31 |
| *** maciejjozefczyk has quit IRC | 10:33 | |
| *** tssurya has quit IRC | 10:37 | |
| *** ttsiouts has joined #openstack-nova | 10:42 | |
| *** guozijn has quit IRC | 10:44 | |
| *** bbowen has quit IRC | 10:49 | |
| *** awalende has quit IRC | 10:49 | |
| *** awalende has joined #openstack-nova | 10:50 | |
| *** whoami-rajat has quit IRC | 10:50 | |
| *** davidsha has joined #openstack-nova | 10:51 | |
| *** priteau has quit IRC | 10:54 | |
| *** _erlon_ has joined #openstack-nova | 11:05 | |
| *** maciejjozefczyk has joined #openstack-nova | 11:06 | |
| *** gmann has joined #openstack-nova | 11:09 | |
| *** maciejjozefczyk has quit IRC | 11:09 | |
| *** mdbooth has joined #openstack-nova | 11:10 | |
| *** mdbooth_ has joined #openstack-nova | 11:11 | |
| gibi | hjensas: I've replied to you in https://review.opendev.org/664559 | 11:12 |
| *** mdbooth has quit IRC | 11:15 | |
| *** nicolasbock has joined #openstack-nova | 11:21 | |
| *** guozijn has joined #openstack-nova | 11:26 | |
| hjensas | gibi: thanks, I'll take look. | 11:38 |
| *** cdent has quit IRC | 11:41 | |
| *** bhagyashris has joined #openstack-nova | 11:49 | |
| *** bbowen has joined #openstack-nova | 11:53 | |
| jangutter | jaypipes: Did you know about this? If PEP 594 gets adopted you won't be able to do 'import pipes' in Python? THEY'RE REMOVING YOU! | 11:55 |
| jangutter | https://lwn.net/ml/python-dev/bf0bf5b8-1ef0-556b-632c-eaebfb325496@python.org/ | 11:55 |
| jaypipes | jangutter: what the actual f!? | 11:55 |
| jaypipes | :) | 11:55 |
| jaypipes | it's a conspiracy! | 11:55 |
| jangutter | jaypipes: I guess they'll have to adopt the much superior 'import gutter' module. | 11:56 |
| jaypipes | lol | 11:57 |
| *** ohwhyosa has joined #openstack-nova | 11:57 | |
| *** ttsiouts has quit IRC | 11:57 | |
| kashyap | Hehe | 11:58 |
| frickler | woa, being called a dead battery sounds really harsh to me | 11:58 |
| *** ttsiouts has joined #openstack-nova | 11:58 | |
| *** awalende has quit IRC | 12:02 | |
| *** ttsiouts has quit IRC | 12:02 | |
| *** awalende has joined #openstack-nova | 12:03 | |
| openstackgerrit | Harald Jensås proposed openstack/nova master: cell_mapping - wrap IPv6 address in format_url https://review.opendev.org/664559 | 12:04 |
| ohwhyosa | Hiya there! | 12:06 |
| ohwhyosa | I was advised to come here to ask about an issue, I'll try to summarize it as well as I can! | 12:06 |
| ohwhyosa | So, I have a 3-nodes openstack-ansible deployment | 12:07 |
| *** awalende has quit IRC | 12:07 | |
| ohwhyosa | infra1 (10.0.0.11) compute1(10.0.0.9) and storage1(10.0.0.10) | 12:07 |
| ohwhyosa | It was kinda working on the first deployment, using spice (though it worked horribly) and without multidomain suport | 12:08 |
| ohwhyosa | So I nuked the lxc containers, deployed again with novnc and multidomain support | 12:08 |
| ohwhyosa | I nuked them using lxc-containers-destroy.yml | 12:08 |
| ohwhyosa | when it was all up again, I couldn't create instances (it complained no hosts were available, though that was a lie, there were plenty resources) | 12:10 |
| ohwhyosa | Looking at the logs, placement said it couldn't assign resources to that node beacsue "hostname" already existed | 12:11 |
| ohwhyosa | I changed the hostname, rebooted everything, and now I can create instances | 12:11 |
| sean-k-mooney | ohwhyosa: so i think i know what the issue is | 12:11 |
| sean-k-mooney | let me see if i can find the bug | 12:12 |
| ohwhyosa | But when I try to access the console, I get "Invalid token, expired or invalid token"... I've checked the logs, and even took the liberty of modifying the python file to provide more logging information...and the token string is empty (not when it hashes it, though, because it does still hash an empty string) | 12:12 |
| sean-k-mooney | basically im guessing teh lxv-contaiern-destroy destored the compute node containers but not placment right? | 12:12 |
| *** markvoelker has joined #openstack-nova | 12:13 | |
| ohwhyosa | It should have destroyed everything, I think I only have containers on infra1 | 12:13 |
| ohwhyosa | but could be, that would explain why the hostname was already registered | 12:13 |
| sean-k-mooney | is infra1 where your contol plain services run | 12:13 |
| ohwhyosa | Yup | 12:13 |
| sean-k-mooney | ok | 12:14 |
| sean-k-mooney | what i think happend is that we destroyted the compute agent and the compute service entry for that agent | 12:14 |
| sean-k-mooney | but im assuming we could not destroy the plamcenet resouce providers because there were allcoation against them | 12:14 |
| ohwhyosa | I'm checking, and all the console_auth_tokens are on the nova database, cell0 has that table empty, in case that is of importance | 12:14 |
| sean-k-mooney | so when you redeployed a new compute service was created | 12:15 |
| ohwhyosa | Hmmm makes sense as a protective measure | 12:15 |
| sean-k-mooney | and that had a new uuid and when it tried to create a new inventory in placment it failed because the old RP still existd wiht a differnet uuid but the same name | 12:15 |
| ohwhyosa | Yup, it did complain about uuid (though that problem "vanishged" after changing hostname...which I now realize might not have been a great idea) | 12:16 |
| ohwhyosa | But didn't lxc-containers-destroy nuke the DB as well? | 12:16 |
| *** awalende has joined #openstack-nova | 12:16 | |
| sean-k-mooney | ohwhyosa: if you check placement for a resouce provider with the onld hostname it should have allocation against it | 12:16 |
| sean-k-mooney | if that is the case its the same issue we are currently trying to fix | 12:17 |
| *** tssurya has joined #openstack-nova | 12:17 | |
| *** markvoelker has quit IRC | 12:17 | |
| *** spsurya has quit IRC | 12:18 | |
| ohwhyosa | Command would be openstack allocation candidate list? | 12:18 |
| sean-k-mooney | am im not sure if we can do this vai the openstack cli but no i dont think so | 12:19 |
| ohwhyosa | sean-k-mooney: How should I check? | 12:20 |
| sean-k-mooney | you need to hit this endpoint https://developer.openstack.org/api-ref/placement/?expanded=list-resource-provider-allocations-detail#list-resource-provider-allocations | 12:20 |
| sean-k-mooney | but ill see if there is a cli you can use one second | 12:20 |
| ohwhyosa | Thanks! brb, will check | 12:20 |
| sean-k-mooney | ohwhyosa: this is the mailing list post related to this issue by the way http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007097.html | 12:21 |
| sean-k-mooney | ohwhyosa: openstack resource provider allocation show | 12:22 |
| *** ratailor has quit IRC | 12:22 | |
| sean-k-mooney | actully no that is not per RP its per consumer which is not what you want | 12:23 |
| ohwhyosa | sean-k-mooney: that command returns "is not an openstack command" | 12:24 |
| ohwhyosa | sean-k-mooney: It recommends resource member list, service provider list and service list | 12:24 |
| sean-k-mooney | you would need a new enough version of osc-placmenet for it to be there | 12:24 |
| sean-k-mooney | but it also is incorrect for what you want to do | 12:25 |
| *** takashin has joined #openstack-nova | 12:25 | |
| *** janki has quit IRC | 12:26 | |
| ohwhyosa | Hmmm according to openstack endpoints list tells me that placement api is at 10.0.0.23:8780 | 12:28 |
| ohwhyosa | But nmap tells me that port is closed and thus curl connectionr efused | 12:28 |
| sean-k-mooney | that is strange | 12:28 |
| sean-k-mooney | do you need to run the command within one of the lxc continers | 12:29 |
| ohwhyosa | the openstack one? | 12:29 |
| sean-k-mooney | yes | 12:30 |
| sean-k-mooney | im not sure how osa setup your networking | 12:30 |
| ohwhyosa | Yup, I use it from the utility container | 12:30 |
| ohwhyosa | Okay, now I have access to the api | 12:31 |
| ohwhyosa | I was using the external_lb_vip u.u | 12:31 |
| sean-k-mooney | ah | 12:31 |
| sean-k-mooney | well i would normally suggest doing | 12:31 |
| sean-k-mooney | curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://10.0.0.23:8780/resource_providers/${uuid}/allocations" | python -mjson.tool | 12:32 |
| sean-k-mooney | assuming you have a token and know the uuid of the RP with the old host name | 12:32 |
| sean-k-mooney | you can get a token with "OS_TOKEN=$(openstack token issue -c id -f value)" | 12:33 |
| sean-k-mooney | you can get teh uuid with "uuid=$(resource provider list --name ${old_hostname} -c uuid -f value)" | 12:35 |
| *** edleafe has joined #openstack-nova | 12:36 | |
| sean-k-mooney | sorry "uuid=$(openstack resource provider list --name ${old_hostname} -c uuid -f value)" | 12:37 |
| *** lbragstad has joined #openstack-nova | 12:37 | |
| *** tssurya has quit IRC | 12:38 | |
| ohwhyosa | resource member or service provider? | 12:39 |
| ohwhyosa | resource provider complains it doesn't exists | 12:39 |
| ohwhyosa | and thanks a lot, I'm learning a ton | 12:39 |
| sean-k-mooney | do you have osc-placement installed? and if so what version | 12:40 |
| ohwhyosa | I believe I don't, I don't seem to be able to use any of its commands | 12:41 |
| sean-k-mooney | ok you could install it but since we are using curl anyway this will fined the RP for you | 12:42 |
| sean-k-mooney | curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://192.168.100.253/placement/resource_providers/?name=${old_hostname}" | python -mjson.tool | 12:42 |
| sean-k-mooney | you need to replace 192.168.100.253/placement with your placemetn endpoint | 12:43 |
| ohwhyosa | Not found (neither with the new nor the old hostname) | 12:45 |
| sean-k-mooney | ya i had a typo | 12:45 |
| sean-k-mooney | curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://192.168.100.253/placement/resource_providers?name=${old_hostname}" | python -mjson.tool | 12:45 |
| sean-k-mooney | you do not want the / at the end of the url i think that was the error | 12:45 |
| ohwhyosa | Yup! | 12:46 |
| ohwhyosa | Indeed I have both the old name and the new one | 12:46 |
| sean-k-mooney | cool so if you grab the uuid for the old name and append <uuid>/allocations to that url | 12:47 |
| sean-k-mooney | then it should not be empty on the old host name correct? | 12:47 |
| sean-k-mooney | e.g. curl -s -H "X-Auth-Token: ${OS_TOKEN}" "http://192.168.100.253/placement/resource_providers/${uuid}/allocations" | python -mjson.tool | 12:47 |
| ohwhyosa | { | 12:48 |
| ohwhyosa | "resource_provider_generation": 2, | 12:48 |
| ohwhyosa | "allocations": {} | 12:48 |
| ohwhyosa | } | 12:48 |
| sean-k-mooney | huh i was expecting that to be non empty | 12:48 |
| sean-k-mooney | well the issue is that that RP was not deleted | 12:48 |
| *** mkrai_ has joined #openstack-nova | 12:49 | |
| sean-k-mooney | i guess the lxc container destory playbook may not have actully deleted it properly | 12:49 |
| ohwhyosa | It has self link, inventories link and usages link when getting the uuid | 12:49 |
| ohwhyosa | Hmmm could be, should I just delete it? | 12:50 |
| ohwhyosa | And should that fix the novnc console also? | 12:51 |
| sean-k-mooney | you have changed the hostname so it should not be an issue. | 12:51 |
| sean-k-mooney | i think the novnc console issue is seperate | 12:51 |
| sean-k-mooney | the fact the RP exsited is why you could not originally recreate the invenoties when the compute agent started up | 12:52 |
| ohwhyosa | It does have a populated resource inventory though | 12:52 |
| ohwhyosa | Oooooh get it! | 12:52 |
| ohwhyosa | Thanks a ton | 12:52 |
| sean-k-mooney | ya the invetory and RP are just left over form the last time the compute node ran update_provider_tree before you redeploy | 12:53 |
| sean-k-mooney | you should clean up the RPs so that the schduler does not try to select them but the nova filters shoudl eliminate them anyway | 12:54 |
| sean-k-mooney | before you try to delete them can you first check if there is a compute service with the old name still | 12:54 |
| ohwhyosa | sorry, i deleted it before reading this | 12:54 |
| ohwhyosa | How can I check if there is a service with the old name? | 12:55 |
| sean-k-mooney | openstack compute service list --host ${old_hostname} --service nova-compute | 12:56 |
| ohwhyosa | Yep, there is one! enabled and down | 12:56 |
| sean-k-mooney | ya its down because it has not recived a heartbeat since you redeployed. | 12:56 |
| ohwhyosa | I should disable and delete it right? | 12:57 |
| ohwhyosa | So, only two issues left with nova for now hahaha the novnc proxy and the err 32 broken pipe | 12:57 |
| sean-k-mooney | if you do openstack compute service delete <service uuid> it will clean up the old placmenet RP too | 12:57 |
| ohwhyosa | Oh! So it does already what I did via the api! | 12:58 |
| sean-k-mooney | ya | 12:58 |
| sean-k-mooney | the playbook you ran jsut nuked the containers | 12:58 |
| sean-k-mooney | it did not also clean up the compute resouces in nova or placmenet | 12:58 |
| sean-k-mooney | normally if it reused the same host name it should have reused the same compute service but im guessing somthing went wrong | 12:59 |
| *** awalende has quit IRC | 12:59 | |
| *** awalende has joined #openstack-nova | 13:00 | |
| ohwhyosa | Good, good, note to self then, delete all the resources before nuking containers | 13:00 |
| *** maciejjozefczyk has joined #openstack-nova | 13:00 | |
| *** pcaruana has quit IRC | 13:01 | |
| sean-k-mooney | well i would check with the openstack-ansible people on how they expect it to be done | 13:01 |
| sean-k-mooney | e.g. im sure they have a playbook or docs for reporviioning node somewhere | 13:01 |
| sean-k-mooney | or they will be able to tell you what the workflow shoud be with there tooling | 13:02 |
| sean-k-mooney | if they dont you can file a bug and im sure they would be happy to add some | 13:02 |
| *** awalende_ has joined #openstack-nova | 13:02 | |
| ohwhyosa | Yup! I'll ask them, to be fair the whole #openstack-* ircs are full of helpful people, thanks a couple tons | 13:03 |
| sean-k-mooney | ohwhyosa: mnaser is really approachable although he is also quite busy but im sure he and the other osa folk will be able to help too | 13:03 |
| mnaser | yeah i don't think we have a 'off-board compute' playbook | 13:03 |
| mnaser | whichgoes and cleans up placement and what not | 13:03 |
| sean-k-mooney | mnaser: do you have a playbook that will reconfigure exisitng nodes? | 13:04 |
| *** awalende has quit IRC | 13:04 | |
| ohwhyosa | Thanks! mnaser so you recommend deleting the service via cli? | 13:04 |
| mnaser | i mean usually the idea is that we run things and they automatically converge again.. in theory | 13:05 |
| *** awalende_ has quit IRC | 13:07 | |
| *** mriedem has joined #openstack-nova | 13:10 | |
| *** jaosorior has quit IRC | 13:11 | |
| ohwhyosa | mnaser: Hmmm converge as in work again right? So, even if I hadn't changed the hostnamed, should it have started working again? | 13:13 |
| *** markvoelker has joined #openstack-nova | 13:14 | |
| ohwhyosa | Also, https://pastebin.com/vnmSEKDW --> This is the broken pipe error, in case you want to take a look, and this | 13:15 |
| ohwhyosa | And this https://pastebin.com/A3VrPvy9 is the noVNC issue | 13:18 |
| *** markvoelker has quit IRC | 13:18 | |
| ohwhyosa | The problem, though, is that the "token" var is an empty string | 13:18 |
| mriedem | amorin: do you plan on backporting this? https://review.opendev.org/#/c/660761/ | 13:18 |
| ohwhyosa | I don't really know where is it querying from, it does mention something in the docs about the token being in the cookies | 13:19 |
| ohwhyosa | But it is inthe url | 13:19 |
| mriedem | need some stable cores on these +2ed rocky changes https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:stable/rocky+label:Code-Review=2 | 13:19 |
| lyarwood | ack, I can look shortly | 13:20 |
| lyarwood | mriedem: https://review.opendev.org/#/c/663677/ would you mind looking at that in return | 13:20 |
| mriedem | sure | 13:21 |
| amorin | mriedem: yes | 13:22 |
| *** udesale has quit IRC | 13:22 | |
| *** udesale has joined #openstack-nova | 13:24 | |
| amorin | mriedem: however, I think I will abandon this one: https://review.opendev.org/#/c/660982/ | 13:25 |
| *** mdbooth_ is now known as mdbooth | 13:25 | |
| amorin | we dont need to force refresh, nova DB is always correct in my tests | 13:25 |
| amorin | we dont need to ask neutron | 13:25 |
| amorin | what do you think? | 13:25 |
| mriedem | i'd say we don't need it until we do :) | 13:26 |
| mriedem | the force refresh is more for when the cache gets corrupted | 13:26 |
| *** udesale has quit IRC | 13:29 | |
| openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.74: Support Specifying AZ to unshelve https://review.opendev.org/665136 | 13:29 |
| *** awalende has joined #openstack-nova | 13:30 | |
| openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.74: Support Specifying AZ to unshelve https://review.opendev.org/665136 | 13:31 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Specify availability_zone to unshelve https://review.opendev.org/663851 | 13:32 |
| openstackgerrit | Leehom Li proposed openstack/nova master: Make sure update resource usage when revert_resize https://review.opendev.org/665138 | 13:35 |
| *** awalende has quit IRC | 13:35 | |
| *** BjoernT has joined #openstack-nova | 13:37 | |
| *** brinzhang has quit IRC | 13:39 | |
| mnaser | ohwhyosa: better to move the convo over to #openstack-ansible to avoid causing noise here :) | 13:40 |
| *** dave-mccowan has joined #openstack-nova | 13:40 | |
| openstackgerrit | Merged openstack/nova-specs master: Spec to pre-filter disabled computes with placement https://review.opendev.org/657884 | 13:42 |
| ohwhyosa | mnaser:Nice! Going there, I though since it was nova related here would be better! | 13:43 |
| sean-k-mooney | ohwhyosa: it depends on the issue. it sounds liek this might be more related to the confgiration of the service rather then the code | 13:44 |
| sean-k-mooney | ohwhyosa: so #openstack-ansible might be able to help more | 13:44 |
| *** ricolin has joined #openstack-nova | 13:45 | |
| ohwhyosa | Great! And thanks for all the help, really, I feel like an openstack jedi already (not even a padawan really, but the feeling is there) | 13:47 |
| openstackgerrit | sean mooney proposed openstack/nova-specs master: Libvirt: add vPMU spec for train https://review.opendev.org/651269 | 13:53 |
| sean-k-mooney | now ^ in the correct folder and some typos fixed :) | 13:53 |
| *** hamdyk has quit IRC | 13:55 | |
| *** ttsiouts has joined #openstack-nova | 13:55 | |
| *** artom has joined #openstack-nova | 13:57 | |
| sean-k-mooney | stephenfin: can you take a look at these quickly https://review.opendev.org/#/q/topic:bug/1832652+(status:open+OR+status:merged) | 13:58 |
| stephenfin | ;sure | 13:58 |
| efried | nova meeting now in #openstack-meeting | 14:00 |
| bhagyashris | stephenfin, sean-k-mooney: Hi, I just want your opinion about http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007084.html http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007109.html | 14:00 |
| sean-k-mooney | bhagyashris i replied to both this morning | 14:01 |
| *** Luzi has quit IRC | 14:02 | |
| *** BjoernT_ has joined #openstack-nova | 14:02 | |
| sean-k-mooney | bhagyashris: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007118.html and http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007119.html | 14:02 |
| bhagyashris | sean-k-mooney; Thank you! i would also like more opinions about the same so that i can proceed | 14:02 |
| *** BjoernT has quit IRC | 14:03 | |
| sean-k-mooney | the important thing to remember is vcpu_pin_set does not and has never ment that the host was use for pinnned instances | 14:03 |
| sean-k-mooney | its related but it not a direct mapping | 14:04 |
| *** igordc has joined #openstack-nova | 14:05 | |
| *** szaher has quit IRC | 14:05 | |
| openstackgerrit | Arnaud Morin proposed openstack/nova stable/stein: Refresh instance network info on deletion https://review.opendev.org/665143 | 14:07 |
| *** JamesBenson has joined #openstack-nova | 14:08 | |
| amorin | mriedem and others see ^, cherry pick for stein | 14:08 |
| mriedem | ack | 14:08 |
| openstackgerrit | Arnaud Morin proposed openstack/nova stable/rocky: Refresh instance network info on deletion https://review.opendev.org/665144 | 14:08 |
| openstackgerrit | Arnaud Morin proposed openstack/nova stable/queens: Refresh instance network info on deletion https://review.opendev.org/665145 | 14:09 |
| amorin | mriedem: are pike and ocata still cherry-pickable also? | 14:09 |
| amorin | or are they freezed? | 14:10 |
| mriedem | they're open, we just don't release them anymore | 14:10 |
| mriedem | they are in extended maintenance mode | 14:10 |
| mriedem | https://docs.openstack.org/project-team-guide/stable-branches.html | 14:10 |
| *** mkrai_ has quit IRC | 14:11 | |
| *** mkrai__ has joined #openstack-nova | 14:11 | |
| *** guozijn has quit IRC | 14:12 | |
| *** mlavalle has joined #openstack-nova | 14:12 | |
| *** JamesBenson has quit IRC | 14:12 | |
| *** xek_ has quit IRC | 14:13 | |
| *** markvoelker has joined #openstack-nova | 14:14 | |
| *** amodi has quit IRC | 14:16 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Follow-up for I6a777b4b7a5729488f939df8c40e49bd40aec3dd https://review.opendev.org/664967 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'MultiattachSupportNotYetAvailable' exception https://review.opendev.org/651315 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: conductor: Remove cells v1 workarounds https://review.opendev.org/651316 | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: Start rewriting all these tests https://review.opendev.org/665146 | 14:16 |
| *** szaher has joined #openstack-nova | 14:19 | |
| *** markvoelker has quit IRC | 14:19 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/stein: Fix double word hacking test https://review.opendev.org/665147 | 14:21 |
| *** ricolin has quit IRC | 14:21 | |
| stephenfin | mriedem: As promised. lyarwood might want to take a look too ^ | 14:21 |
| lyarwood | stephenfin: ah cool, I don't think the change to disable this has landed in stein yet | 14:22 |
| stephenfin | nope, just left a comment on that backport of yours | 14:22 |
| lyarwood | thanks | 14:22 |
| openstackgerrit | John Garbutt proposed openstack/nova-specs master: Add Unified Limits Spec https://review.opendev.org/602201 | 14:26 |
| *** luksky has quit IRC | 14:29 | |
| jangutter | Could I ask someone more familiar with tox and multiple python versions to check sean-k-mooney's one-liner here: https://review.opendev.org/#/c/665034 | 14:31 |
| stephenfin | The 'build_instances' function is so fricking complex. It's almost impossible figure out what most of these tests are supposed to be testing :( https://review.opendev.org/665146 | 14:33 |
| stephenfin | jangutter: I've already hit it. It's correct | 14:33 |
| * stephenfin added that feature to tox | 14:34 | |
| sean-k-mooney | stephenfin: yep thats why i pinged you to review :) | 14:34 |
| sean-k-mooney | and you know its os-vif | 14:34 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Recalculate 'RequestSpec.numa_topology' on resize https://review.opendev.org/662522 | 14:34 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Cleanup of '_test_resize' helper test https://review.opendev.org/664245 | 14:34 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add '_setup_compute_services' helper https://review.opendev.org/663102 | 14:34 |
| jangutter | stephenfin: thanks, I suspected as much, but if it summons Cthulhu, I'm pointing him to you. | 14:34 |
| *** mkrai__ has quit IRC | 14:36 | |
| stephenfin | #success Cells v1 has finally be removed from nova. Next up, nova-network | 14:36 |
| openstackstatus | stephenfin: Added success to Success page (https://wiki.openstack.org/wiki/Successes) | 14:36 |
| *** JamesBenson has joined #openstack-nova | 14:36 | |
| stephenfin | s/be/been/ Damn it | 14:36 |
| *** jaosorior has joined #openstack-nova | 14:37 | |
| *** JamesBenson has quit IRC | 14:41 | |
| *** igordc has quit IRC | 14:42 | |
| *** jaosorior has quit IRC | 14:45 | |
| *** bhagyashris has quit IRC | 14:48 | |
| *** cfriesen has joined #openstack-nova | 14:55 | |
| *** artom has quit IRC | 15:00 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve E731 (do not assign a lambda expression, use a def) https://review.opendev.org/651554 | 15:00 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve W503 (line break occurred before a binary operator) https://review.opendev.org/651555 | 15:00 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve E741 (ambiguous variable name) https://review.opendev.org/652103 | 15:00 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: hacking: Resolve W605 (invalid escape sequence) https://review.opendev.org/652104 | 15:00 |
| *** artom has joined #openstack-nova | 15:01 | |
| *** ratailor has joined #openstack-nova | 15:01 | |
| *** BjoernT has joined #openstack-nova | 15:02 | |
| sean-k-mooney | before i forget my alternitive proposal for the trait -> aggrage mapping would be as follows. when we set tratis in nova host_aggrages we create a resouceless shareing RP in placmeent and add the traits to that | 15:02 |
| mriedem | mgoddard: you might want to read through https://review.opendev.org/#/c/449155/ first | 15:02 |
| sean-k-mooney | and dont modify the host at all | 15:02 |
| sean-k-mooney | then the sync is simple | 15:02 |
| sean-k-mooney | that would require resouceless RPs for it to work however | 15:02 |
| mgoddard | mriedem: I'll add it to my reading list | 15:03 |
| sean-k-mooney | efried: ^ was for you and i guess anyone that is interested | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Remove references to nova-consoleauth https://review.opendev.org/652965 | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Stop starting consoleauth in functional tests https://review.opendev.org/652966 | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: xvp: Start using consoleauth tokens https://review.opendev.org/652967 | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-status: Remove consoleauth workaround check https://review.opendev.org/652968 | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove nova-consoleauth https://review.opendev.org/652969 | 15:03 |
| efried | sean-k-mooney: That could work, but sounds like a pretty big hammer. | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove ConsoleAuthToken.to_dict https://review.opendev.org/652970 | 15:03 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Rework nova console diagram https://review.opendev.org/660147 | 15:03 |
| sean-k-mooney | efried: it might be but it makes the nova and placmenet logic a lot simpler | 15:03 |
| sean-k-mooney | but it also intoduce a depency on resouceles providers | 15:04 |
| *** takashin has quit IRC | 15:04 | |
| mriedem | mgoddard: the things you'd have to be aware of is the allow_resize_to_same_host config option in the compute api - you'd have to ignore that for baremetal instances, and figuring out if the instance is baremetal to begin with to change how scheduling is going to work etc - and then it's implementing some virt driver methods | 15:04 |
| sean-k-mooney | maybe something to consider after train? | 15:04 |
| mriedem | sean-k-mooney: or, or, we just say nova doesn't orchestrate that and if you want it, do it external to nova | 15:04 |
| *** BjoernT_ has quit IRC | 15:04 | |
| mriedem | b/c christ almighty we're already bad at orchestrating external resources, | 15:04 |
| sean-k-mooney | mriedem: yes which is definetly what i think we shoudl do for train | 15:04 |
| mriedem | case in point, see my ML thread about deleting a compute service | 15:05 |
| sean-k-mooney | :) fair point | 15:05 |
| sean-k-mooney | we are also terrible at math | 15:05 |
| efried | mriedem, mgoddard: ironic resize: Is the consensus that someone should produce a spec (either revive the one linked above or write a new one)? But generally speaking there's no major objection to doing some form of "same host resize" for ironic? | 15:05 |
| sean-k-mooney | efried: you would proably need a prefilter or somthing that could understand its a baremetal instance and force the same host | 15:06 |
| efried | yes, mriedem mentioned that above | 15:06 |
| sean-k-mooney | the allow same hsot resize jsut allows it rather then forces it | 15:06 |
| efried | yes, mriedem mentioned that above | 15:06 |
| sean-k-mooney | yep | 15:07 |
| sean-k-mooney | the question i have however is do we really need to force that | 15:07 |
| *** takashin has joined #openstack-nova | 15:07 | |
| sean-k-mooney | i mean ironic could support resize in general | 15:07 |
| efried | are you suggesting allowing ironic resize to a new node? | 15:07 |
| sean-k-mooney | yes | 15:07 |
| efried | baby steps, sean-k-mooney. Baby steps. | 15:07 |
| aspiers | sean-k-mooney: turns out meetbot will recognise anyone issuing an #action | 15:08 |
| sean-k-mooney | for bfv guest there is nothitng ot copy and for non-bfv the ironic agent could be extended to dd the data form one to anohter | 15:08 |
| aspiers | which matches my memory | 15:08 |
| aspiers | http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-06-13-14.00.html | 15:08 |
| sean-k-mooney | aspiers: cool good to know | 15:08 |
| efried | sean-k-mooney: Yes, but let's address the case we know we want to solve first, before enabling the general case and all the test surface that goes with it. | 15:08 |
| sean-k-mooney | efried: sure although im pretty sure the general case had a spec form a few cylces ago which is why i brought it up | 15:09 |
| artom | mriedem, efried, https://wiki.openstack.org/wiki/Nova/Mentoring#Work_Items_for_New_Contributors and https://etherpad.openstack.org/p/nova-reviews-looking-for-owner | 15:10 |
| sean-k-mooney | efried: but yes i think that the simpler case of same host could be a stepping stone | 15:10 |
| efried | thanks artom | 15:11 |
| *** takashin has left #openstack-nova | 15:19 | |
| mriedem | artom: i'll update 522084 underneath the scheduler hints change | 15:21 |
| artom | mriedem, ah, thanks, didn't notice it was a 2-patch series | 15:22 |
| *** igordc has joined #openstack-nova | 15:23 | |
| openstackgerrit | Merged openstack/os-vif master: Fix mock of built in "open" function in unit tests https://review.opendev.org/664583 | 15:25 |
| openstackgerrit | Merged openstack/os-vif master: OVS DPDK port representors support https://review.opendev.org/658786 | 15:25 |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/queens: Fail to live migration if instance has a NUMA topology https://review.opendev.org/629597 | 15:28 |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/queens: fix up numa-topology live migration hypervisor check https://review.opendev.org/648939 | 15:28 |
| stephenfin | lyarwood: Could you take a look at https://review.opendev.org/#/c/629597 when you have a chance? | 15:29 |
| *** dpawlik has quit IRC | 15:35 | |
| *** gyee has joined #openstack-nova | 15:35 | |
| lyarwood | stephenfin: queued | 15:35 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Make RequestContext(instance_lock_checked) fail https://review.opendev.org/665003 | 15:36 |
| *** ttsiouts has quit IRC | 15:37 | |
| *** ttsiouts has joined #openstack-nova | 15:37 | |
| *** ivve has quit IRC | 15:40 | |
| *** ttsiouts has quit IRC | 15:42 | |
| *** sridharg has quit IRC | 15:42 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Api-guide: Add Block Device Mapping https://review.opendev.org/522084 | 15:43 |
| mriedem | artom: rebase on ^ - stephenfin - easy docs change ^ | 15:43 |
| artom | mriedem, ack | 15:43 |
| *** mkrai_ has joined #openstack-nova | 15:47 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: allow getting resource request of every bound ports of an instance https://review.opendev.org/655110 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pass network API to the conducor's MigrationTask https://review.opendev.org/655111 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add request_spec to server move RPC calls https://review.opendev.org/655721 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: re-calculate provider mapping during migration https://review.opendev.org/655112 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: update allocation in binding profile during migrate https://review.opendev.org/656422 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Extend NeutronFixture to handle migrations https://review.opendev.org/655114 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth https://review.opendev.org/655109 | 15:49 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: func test for migrate server with ports having resource request https://review.opendev.org/655113 | 15:49 |
| stephenfin | mriedem: Could you or artom s/.. code::/.. code-block:: json/ in that? +W if so https://review.opendev.org/#/c/522084/ | 15:49 |
| mriedem | stephenfin: nope | 15:52 |
| mriedem | you can't b/c of the ... | 15:52 |
| *** helenafm has quit IRC | 15:52 | |
| stephenfin | I noted that. Can we drop it like we did elsewhere in the file? | 15:52 |
| mriedem | drop what? the ... ? | 15:52 |
| stephenfin | aye | 15:52 |
| mriedem | i don't really want to put a full server create request in there | 15:52 |
| mriedem | since those are big | 15:52 |
| stephenfin | see line 810 in there | 15:52 |
| stephenfin | I know, but I imagine people can guess it's not a full response | 15:53 |
| stephenfin | as with 810, 829, etc. | 15:53 |
| *** jangutter has quit IRC | 15:54 | |
| mriedem | ffs zuul restart | 15:54 |
| openstackgerrit | Harald Jensås proposed openstack/nova master: cell_mapping - wrap IPv6 address in format_url https://review.opendev.org/664559 | 15:56 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: libvirt: Use SATA bus for cdrom devices when using Q35 machine type https://review.opendev.org/663677 | 15:56 |
| mriedem | lyarwood: why did you need to rebase ^? | 15:57 |
| mriedem | oh i see | 15:57 |
| lyarwood | mriedem: yeeeeeeah, I didn't notice the zuul restart, I wonder if that's going to get picked up. | 15:58 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Api-guide: Add Block Device Mapping https://review.opendev.org/522084 | 15:58 |
| mriedem | stephenfin: done | 15:58 |
| mriedem | lyarwood: yeah idk, was just rechecking a bunch of other things too | 15:59 |
| lyarwood | ah, so that change was picked up | 15:59 |
| stephenfin | mriedem: +2. Happy for you to +W or I can | 15:59 |
| stephenfin | thanks :) | 15:59 |
| mriedem | i got it | 16:01 |
| *** damien_r has quit IRC | 16:02 | |
| *** mkrai_ has quit IRC | 16:02 | |
| *** mkrai_ has joined #openstack-nova | 16:03 | |
| mriedem | looks like all my rechecks were dropped | 16:03 |
| mriedem | weeeee | 16:03 |
| *** ccamacho has quit IRC | 16:08 | |
| *** ratailor has quit IRC | 16:10 | |
| *** mkrai_ has quit IRC | 16:13 | |
| *** rpittau is now known as rpittau|afk | 16:13 | |
| *** markvoelker has joined #openstack-nova | 16:16 | |
| sean-k-mooney | artom: i noticed you updated your numa migration stuff yesterday | 16:19 |
| sean-k-mooney | artom: have you more changes pending or shoudl i plan to review it/test it next week | 16:20 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Rework all things metadata'y https://review.opendev.org/640730 | 16:20 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: conf: Rename 'configuration drive' to 'config drive' https://review.opendev.org/641723 | 16:20 |
| *** markvoelker has quit IRC | 16:20 | |
| stephenfin | efried: Think I asked you this before, but what's the chances of getting https://review.opendev.org/640730 blindly approved? | 16:21 |
| stephenfin | It's docs only and I think it's a huge improvement, but I've no idea how to break it up more than I have without rewriting the whole darn thing | 16:21 |
| mriedem | stephenfin: maybe ask dansmith to review it again first :) | 16:22 |
| stephenfin | I think I've done that a few times too :( | 16:22 |
| mriedem | since he did the most thorough review | 16:22 |
| *** dtantsur is now known as dtantsur|afk | 16:22 | |
| dansmith | stephenfin: done what a few times? | 16:23 |
| stephenfin | dansmith: Have a look at that ridiculously big metadata doc rework patch of mine | 16:23 |
| stephenfin | (https://review.opendev.org/640730) | 16:23 |
| artom | sean-k-mooney, just a rebase for now, bottomg patch is WIP | 16:23 |
| artom | To indicate that the whole series isn't ready yet | 16:23 |
| dansmith | stephenfin: just wanted to make sure you weren't implying you had asked me to revisit a few times | 16:24 |
| sean-k-mooney | artom: ok so saving the updated xml after the migration is still pending | 16:24 |
| sean-k-mooney | artom: that was the main thing that needed to be fixed and the rest is cleanup right | 16:24 |
| stephenfin | I think I may have and I get why you haven't, if so. Not intending to point fingers at all though. I'd just like to be done with it | 16:26 |
| artom | sean-k-mooney, the XML is fine, IIRC, it's the database numa_topology that wasn't saved | 16:28 |
| dansmith | stephenfin: point being, I'm quite sure I didn't hear anything after that review, but I'm looking at it right now | 16:28 |
| sean-k-mooney | artom: maybe i just recall we didnt save somthing | 16:28 |
| stephenfin | dansmith: Ah, sorry. That's my mistake so | 16:29 |
| stephenfin | And thank you | 16:29 |
| *** READ10 has joined #openstack-nova | 16:29 | |
| *** mdbooth_ has joined #openstack-nova | 16:31 | |
| *** mdbooth has quit IRC | 16:34 | |
| dansmith | mriedem: for the tenth and last time, can you hit this? https://review.opendev.org/#/c/640730/7 | 16:36 |
| sean-k-mooney | stephenfin: i personlay prefer the ... then assuming people will guess its not a full respocne but its proably fine eitehr way | 16:36 |
| stephenfin | sean-k-mooney: Too late. mriedem already did the deed | 16:36 |
| stephenfin | mwahaha | 16:37 |
| sean-k-mooney | stephenfin: that was regarding mriedem change but he has updated it already | 16:37 |
| sean-k-mooney | yep i know | 16:37 |
| stephenfin | No take backsies | 16:37 |
| mriedem | dansmith: lucky for y'all i just got a fresh pack of red markers | 16:37 |
| sean-k-mooney | stephenfin: damn i have not heard that in a while | 16:37 |
| *** mdbooth_ has quit IRC | 16:38 | |
| *** gibi has quit IRC | 16:41 | |
| efried | stephenfin: looking now. | 16:43 |
| aspiers | efried: oslo.log guidelines feedback addressed, when you are looking for some light relief | 16:46 |
| efried | ack | 16:46 |
| *** cdent has joined #openstack-nova | 16:56 | |
| *** JamesBenson has joined #openstack-nova | 16:57 | |
| *** JamesBenson has quit IRC | 17:00 | |
| *** cdent has quit IRC | 17:01 | |
| *** derekh has quit IRC | 17:03 | |
| *** mdbooth has joined #openstack-nova | 17:04 | |
| *** davidsha has quit IRC | 17:05 | |
| *** mdbooth has quit IRC | 17:07 | |
| *** mdbooth has joined #openstack-nova | 17:07 | |
| *** ohwhyosa has quit IRC | 17:08 | |
| *** pcaruana has joined #openstack-nova | 17:09 | |
| *** nicolasbock has quit IRC | 17:10 | |
| *** cdent has joined #openstack-nova | 17:11 | |
| *** ociuhandu has quit IRC | 17:15 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Warn for duplicate host mappings during discover_hosts https://review.opendev.org/651947 | 17:18 |
| melwitt | dansmith: ^ updated | 17:19 |
| dansmith | gdi | 17:21 |
| melwitt | what | 17:22 |
| dansmith | you're supposed to wait a month between all updates | 17:23 |
| melwitt | :) | 17:23 |
| *** panda has quit IRC | 17:24 | |
| efried | stephenfin: I'm going to wait until that sucker renders so I can at least feel I've done due diligence on the links and stuff. | 17:24 |
| *** ralonsoh has quit IRC | 17:26 | |
| *** panda has joined #openstack-nova | 17:27 | |
| melwitt | \o\ | 17:30 |
| *** whoami-rajat has joined #openstack-nova | 17:38 | |
| *** JamesBenson has joined #openstack-nova | 17:41 | |
| *** hongbin has joined #openstack-nova | 17:43 | |
| *** JamesBenson has quit IRC | 17:45 | |
| *** JamesBenson has joined #openstack-nova | 17:50 | |
| artom | mriedem, so https://review.opendev.org/#/c/522106/ can be abandoned | 17:50 |
| *** damien_r has joined #openstack-nova | 17:52 | |
| mriedem | artom: not really | 17:53 |
| mriedem | https://review.opendev.org/#/c/581410/ isn't really describing scheduler hints in detail, | 17:53 |
| mriedem | it's just describing scheduler hints vs flavor extra specs | 17:53 |
| mriedem | which is more of an admin/operator thing than end user | 17:53 |
| artom | mriedem, are we reading the same first paragraph of https://docs.openstack.org/nova/latest/reference/scheduler-hints-vs-flavor-extra-specs.html#scheduler-hints? | 17:53 |
| mriedem | i agree the in-tree hints are described now in the api-ref | 17:53 |
| mriedem | artom: i wrote https://docs.openstack.org/nova/latest/reference/scheduler-hints-vs-flavor-extra-specs.html#scheduler-hints? | 17:54 |
| *** JamesBenson has quit IRC | 17:54 | |
| artom | mriedem, I know - it's almost word for word as what's in https://review.opendev.org/#/c/522106/8/api-guide/source/server_concepts.rst, no? | 17:54 |
| artom | OK, maybe not word for word | 17:55 |
| mriedem | i think the api-guide would benefit from (1) a short description of what are scheduler hints, (2) an example api request, and then (3) links to the thing above (hints vs extra specs) and the server create api reference for the description of the available in-tree hints | 17:55 |
| mriedem | our api-ref doesn't have a sample with a scheduler hint in | 17:56 |
| *** ivve has joined #openstack-nova | 17:56 | |
| mriedem | *it | 17:56 |
| mriedem | so every time i write a test that uses one, i have to look this up | 17:56 |
| mriedem | b/c 1. there are 2 param names accepted for hints and 2. it's at the top level of the request rather than within the server object | 17:56 |
| openstackgerrit | Merged openstack/os-vif master: set ignore_basepython_conflict = True in tox.ini https://review.opendev.org/665034 | 17:57 |
| artom | Well, I signed up for this, might as well finish it | 17:57 |
| artom | mriedem, same patch? Smells like a new change | 17:57 |
| artom | Can I achieve peak arrogance by trading you reviews on https://review.opendev.org/#/c/644881/? | 17:58 |
| artom | I guess same change to preserve review history | 17:59 |
| mriedem | i would do the same patch for review history yes | 18:01 |
| mriedem | add yourself as co-author | 18:01 |
| mriedem | and because you asked for https://review.opendev.org/#/c/644881/ yet again i'm going to say i'll never review it again now | 18:01 |
| mriedem | you're welcome | 18:01 |
| *** JamesBenson has joined #openstack-nova | 18:02 | |
| openstackgerrit | sean mooney proposed openstack/os-vif stable/stein: set ignore_basepython_conflict = True in tox.ini https://review.opendev.org/665195 | 18:02 |
| *** gyee has quit IRC | 18:04 | |
| openstackgerrit | sean mooney proposed openstack/os-vif stable/stein: Fix mock of built in "open" function in unit tests https://review.opendev.org/665196 | 18:07 |
| mriedem | artom: before you raise HR alarms downstream, i was joking | 18:08 |
| melwitt | mriedem: this is a bit old and has a +2 from dansmith now, you're probably the best person to review since it's about nova-manage cells stuff https://review.opendev.org/651947 | 18:08 |
| artom | mriedem, you haven't touched me inappropriately (yet?), so no HR alarms | 18:08 |
| artom | ;) | 18:09 |
| mriedem | dansmith: you know how we have this warning about stopping nova-compute before deleting it? https://developer.openstack.org/api-ref/compute/?expanded=delete-compute-service-detail#delete-compute-service otherwise dragons? it strikes me now that the delete compute service API should just be checking if the service is up and if so, fail the request with a 409 | 18:09 |
| sean-k-mooney | artom: a +1 from zuul, how did you apise the ci gods to make that happen | 18:10 |
| dansmith | mriedem: maybe, unless there's a reason to delete and have it recreate or something | 18:10 |
| artom | sean-k-mooney, which patch? | 18:10 |
| dansmith | mriedem: the other thing would be that unless you also force down, you can't stop and delete inside of a minute or whatever the service timeout is | 18:10 |
| sean-k-mooney | artom: you have more then one with a +1 :P the hybrid_plug events one | 18:11 |
| mriedem | yeah you'd have to force down | 18:11 |
| mriedem | dansmith: the problem with the recreate is the new provider it will create in placement with no allocations for the instances on that host, which will f up scheduling | 18:11 |
| artom | sean-k-mooney, except for the double words thing (that was fixed) I'm not aware of any Zuul issues | 18:11 |
| mriedem | you can heal that with the heal_allocations CLI of course... | 18:11 |
| mriedem | well, for non-nested / sharing allocations | 18:12 |
| *** cdent has quit IRC | 18:12 | |
| dansmith | mriedem: sure | 18:12 |
| sean-k-mooney | artom: it was more the fact its taken a few hours for zuul to notice and run on your patch for the last few days then it being flaky | 18:12 |
| artom | sean-k-mooney, oh, yeah, it's slow | 18:13 |
| mriedem | i'm just down a hole on how we're orphaning providers today and how best to shore that up | 18:13 |
| sean-k-mooney | im following the thread on the ml bug trying to stay out of it to a degree to leave ops speak up | 18:16 |
| sean-k-mooney | although it looks like the last few mails are back and forth between you erric and chris | 18:16 |
| *** markvoelker has joined #openstack-nova | 18:17 | |
| *** gyee has joined #openstack-nova | 18:19 | |
| *** markvoelker has quit IRC | 18:22 | |
| mriedem | i think likely the best thing to do is a phased conservative approach, | 18:24 |
| mriedem | 1. land my change that fixes it to delete all providers, not just the first (the ironic case), | 18:24 |
| mriedem | 2. if deleting any provider fails, don't delete the compute service and bail (option 1 in the email), | 18:24 |
| mriedem | 3. check for and cleanup allocations for evacuations - something we can be pretty sure we can get rid of safely | 18:25 |
| mriedem | that leaves in-progress or unconfirmed migrations, and in that case i think we likely just should be saying "you can't delete the compute service until these migrations are done" | 18:25 |
| sean-k-mooney | ya that seam sane. the issue that im not sure how to solve/if we shoul solv eis if you had in progeress migration and the host failed/went offline | 18:26 |
| sean-k-mooney | can you evaucate an instance that was migrating | 18:27 |
| sean-k-mooney | when the host went off line | 18:27 |
| bnemec | aspiers: +2. Thanks for taking care of this! | 18:27 |
| aspiers | bnemec: yw | 18:28 |
| mriedem | sean-k-mooney: don't think so | 18:28 |
| mriedem | unless you reset the instance state to ERROR | 18:28 |
| sean-k-mooney | im fine to leave that up to the operator to clean up too, but im just wondering about the case where the reason their are pending migration or evacuation is because of a host failure | 18:28 |
| sean-k-mooney | or reset it ot active | 18:28 |
| sean-k-mooney | but ok | 18:28 |
| sean-k-mooney | if it was in resize_verify or whatever the state is when we havent confirmed yet but was successafaly migrated we would proably want to reset it to active | 18:29 |
| mriedem | if you're in the middle of a migration and the source host went offline, i don't see why you'd delete the service - just bring the service back up if possible | 18:30 |
| sean-k-mooney | ya im somewhat unclear why the compute service is being deleted with outstanding migrations too | 18:31 |
| sean-k-mooney | also reset state to active instead of confirming might not be right as it likely would not have updated the db correctly for the new host | 18:32 |
| sean-k-mooney | so you would want to confirm. let it error out and then rest to active/hardreboot to fix it right | 18:32 |
| mriedem | well the good news is a contracted google recruiter has contacted me about opportunities in the bay area of new york so if i want to avoid this problem i can just follow that lead and up-root my entire life | 18:33 |
| mriedem | delete compute service problems be damned! | 18:34 |
| sean-k-mooney | haha well that is one way to solve it yes then you can work on kubernetes problems | 18:34 |
| sean-k-mooney | its really a seperate issue but should we allow you to confim a resize/migration if the souce host is down without erroring out? | 18:35 |
| mriedem | then after 6 months you get there and they're like, "ok we have this resource leak issue when you delete a pod" | 18:35 |
| mriedem | sean-k-mooney: i'm not sure what happens in that case, i haven't tried it, but i'd think the cast to the source compute would blow up and you'd get a 500 response from the confirmResize API | 18:36 |
| sean-k-mooney | well if its a cast it a fire and forget if its a call then yes | 18:37 |
| sean-k-mooney | do we start the confrim form the souce node so the api will jsut time out the request | 18:37 |
| sean-k-mooney | e.g. because notheing handels it. | 18:37 |
| melwitt | efried: do you want a follow up patch for your comments on https://review.opendev.org/651947 | 18:38 |
| efried | melwitt: meh, they're so minor. | 18:38 |
| melwitt | ack | 18:38 |
| efried | melwitt: If you want to spin a new PS I can fast approve | 18:38 |
| sean-k-mooney | i need to get a better undestand of how to create nova functional test to test this kind fo stuff without actully depolying devstack | 18:38 |
| efried | or we can just ignore | 18:38 |
| efried | mriedem: Was about to start composing the email whining about bug queue. Unless you want to do it... <bats eyes> | 18:40 |
| melwitt | efried: ok. I can respin, that's easy enough. mostly I'm thinking of the doc rendering where literal would help | 18:40 |
| efried | melwitt: There's probably opportunity to add literals throughout that doc - I didn't check. If so, maybe we could scoop it all up in a fup | 18:40 |
| efried | anyway, I'm around for at least another 3-4h to push whatever | 18:41 |
| melwitt | ok. yeah, I see most seem literal'd but some others aren't so could be done all together | 18:42 |
| mriedem | efried: i do not, writing a reply to this compute service delete thread | 18:43 |
| efried | mriedem: ack. Okay if I run it by you before sending? | 18:44 |
| sean-k-mooney | mriedem: i might steal your test code form https://review.opendev.org/#/c/663737/1/nova/tests/functional/wsgi/test_services.py and try and create a similar test for that migrate confim case while im waiting for the pizza to arrive | 18:47 |
| dansmith | mmm pizza | 18:48 |
| * dansmith waits for it | 18:48 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: api-guide: better explain scheduler hints. https://review.opendev.org/522106 | 18:48 |
| sean-k-mooney | hehe one of irelands suckiest inventions is slow "fast food" | 18:48 |
| sean-k-mooney | all the quality of fast food with the slowness of going to the store and cooking it your self. | 18:49 |
| mriedem | COOKIES | 18:49 |
| sean-k-mooney | given i havent ordered it yet i might actully do that | 18:49 |
| melwitt | did somebody say pizza | 18:51 |
| mriedem | dansmith: i was just thinking about that yesterday for some reason, probably b/c i made a pizza for lunch | 18:52 |
| dansmith | heh | 18:53 |
| *** dpawlik has joined #openstack-nova | 18:54 | |
| *** hoonetorg has quit IRC | 18:55 | |
| *** hoonetorg has joined #openstack-nova | 19:00 | |
| dansmith | mriedem: I don't even remember what the gag was.. trying to explain to sdague how to use punctuation or something | 19:03 |
| openstackgerrit | melanie witt proposed openstack/nova master: Literalize CLI options in docs https://review.opendev.org/665217 | 19:04 |
| mriedem | i think it was related to commit message titles | 19:05 |
| mriedem | something like that | 19:05 |
| mriedem | Something like that. | 19:05 |
| mriedem | one of the annual flare ups in the ML | 19:05 |
| mriedem | the good old days | 19:05 |
| *** bnemec has quit IRC | 19:06 | |
| mriedem | i think we were shitting on his commit message which was all lower case with sparse amount of periods | 19:06 |
| dansmith | sounds right | 19:06 |
| mriedem | mostly to poke the bear and air grievances | 19:06 |
| mriedem | it wasn't even festivus | 19:06 |
| *** damien_r has quit IRC | 19:06 | |
| mriedem | and now look at him, in front of congress and all | 19:07 |
| mriedem | we really showed him | 19:07 |
| dansmith | because congress has such high standads | 19:07 |
| dansmith | *standards | 19:07 |
| mriedem | like garth brooks, sean has friends in low places | 19:08 |
| dansmith | heh | 19:08 |
| melwitt | remember chris gaines | 19:09 |
| mriedem | how could i forget | 19:10 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Delete resource providers for all nodes when deleting compute service https://review.opendev.org/657016 | 19:10 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Avoid unnecessary joins in delete_resource_provider https://review.opendev.org/657021 | 19:10 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Optimize SchedulerReportClient.delete_resource_provider https://review.opendev.org/657070 | 19:10 |
| mriedem | remember...THIS?! https://www.youtube.com/watch?v=oGpFcHTxjZs | 19:11 |
| dansmith | man.. bad hair day | 19:12 |
| melwitt | the intro made it look like the panpipe sound was coming from the tugboat lol | 19:12 |
| mriedem | maya got her first exposure to that in the car last night, with me singing along to it | 19:12 |
| melwitt | peruvian panpipes | 19:12 |
| mriedem | and laura telling me to shut up | 19:13 |
| melwitt | did you dance too? I remember you said that's a favorite for car trips with the family | 19:13 |
| mriedem | i did the shoulder shake sure | 19:14 |
| mriedem | it cannot be stopped | 19:14 |
| melwitt | yeah, that's what I meant. shoulder shake was the thing they are a fan of | 19:16 |
| *** markvoelker has joined #openstack-nova | 19:18 | |
| efried | mriedem: http://paste.openstack.org/show/752899/ | 19:20 |
| efried | not sure what else to say | 19:20 |
| mriedem | efried: looks fine to me | 19:21 |
| efried | ight, thx | 19:21 |
| *** markvoelker has quit IRC | 19:23 | |
| *** maciejjozefczyk has quit IRC | 19:25 | |
| *** mdbooth has quit IRC | 19:27 | |
| mriedem | lyarwood: efried: did anyone talk about case here? https://review.opendev.org/#/c/663011/17/nova/virt/libvirt/blockinfo.py@275 | 19:31 |
| *** takamatsu has quit IRC | 19:32 | |
| efried | mriedem: doesn't look like it's ever uppercase, but I really don't know. | 19:34 |
| *** mdbooth has joined #openstack-nova | 19:34 | |
| mriedem | yeah the image meta props don't define an enum | 19:35 |
| mriedem | looks like libvirt / qemu uses q35 though http://logs.openstack.org/11/663011/17/check/tempest-full-py3/6a826d2/controller/logs/screen-n-cpu.txt.gz#_Jun_12_22_24_13_577256 | 19:37 |
| openstackgerrit | Merged openstack/nova stable/rocky: Move get_pci_mapping_for_migration to MigrationContext https://review.opendev.org/661499 | 19:38 |
| openstackgerrit | Merged openstack/nova stable/rocky: Allow driver to properly unplug VIFs on destination on confirm resize https://review.opendev.org/661500 | 19:38 |
| openstackgerrit | Merged openstack/nova stable/rocky: Fixes multi-registry config in Quobyte driver https://review.opendev.org/662695 | 19:38 |
| *** maciejjozefczyk has joined #openstack-nova | 19:38 | |
| openstackgerrit | Merged openstack/nova stable/rocky: tox: Use basepython of python3.5 https://review.opendev.org/663887 | 19:38 |
| openstackgerrit | Merged openstack/nova stable/rocky: docs: Don't version links to reno docs https://review.opendev.org/662178 | 19:38 |
| openstackgerrit | Merged openstack/nova stable/rocky: Include all network devices in nova diagnostics https://review.opendev.org/661962 | 19:38 |
| *** maciejjozefczyk_ has joined #openstack-nova | 19:41 | |
| *** maciejjozefczyk has quit IRC | 19:45 | |
| *** mdbooth has quit IRC | 19:45 | |
| *** maciejjozefczyk has joined #openstack-nova | 19:46 | |
| *** maciejjozefczyk_ has quit IRC | 19:47 | |
| *** whoami-rajat has quit IRC | 19:48 | |
| *** xek has joined #openstack-nova | 19:50 | |
| *** bbowen has quit IRC | 19:56 | |
| openstackgerrit | Merged openstack/nova master: Api-guide: Add Block Device Mapping https://review.opendev.org/522084 | 20:01 |
| *** eharney has quit IRC | 20:01 | |
| *** spatel has joined #openstack-nova | 20:07 | |
| spatel | sean-k-mooney: Question | 20:07 |
| spatel | building new cloud so should i go with linuxbridge or OVS ( any performance difference here? other than features) | 20:08 |
| mriedem | efried: fwiw i'm going through stephen's metadata docs change now | 20:15 |
| efried | mriedem: cool, thanks for that. | 20:15 |
| efried | mriedem: It still hasn't rendered, right? | 20:15 |
| efried | I guess I could pull it down and build it myself... | 20:15 |
| mriedem | that's what i did | 20:15 |
| mriedem | the docs are failing to build locally for me | 20:17 |
| *** markvoelker has joined #openstack-nova | 20:19 | |
| *** markvoelker has quit IRC | 20:24 | |
| *** artom has quit IRC | 20:28 | |
| *** READ10 has quit IRC | 20:31 | |
| mriedem | i see why now | 20:32 |
| *** pcaruana has quit IRC | 20:36 | |
| *** panda has quit IRC | 20:49 | |
| *** panda has joined #openstack-nova | 20:51 | |
| *** bnemec has joined #openstack-nova | 20:52 | |
| mriedem | gdi i've only made it 10 of 22 files in this change | 20:52 |
| *** xek has quit IRC | 20:57 | |
| *** dpawlik has quit IRC | 21:01 | |
| sean-k-mooney | spatel: linuxbirdge is not that well tested i personally am much more comfortable with ovs however some large public cloud have use linux birdge like rackspace | 21:02 |
| sean-k-mooney | spatel: in generall linuxbrige will have fewer fancy networking feature and tends not to scale beyond 10G | 21:03 |
| sean-k-mooney | but its also simpler | 21:03 |
| sean-k-mooney | so depending on your netwroking backgorund and the team that will be maintaining the clould you could be trading performace for lower operationgs costs | 21:04 |
| spatel | sean-k-mooney: thank you, i am already using LinuxBridge in my cloud but yes its very simple, first time i am going to play with OVS so not sure what and where i am going to face complexity | 21:04 |
| *** JamesBenson has quit IRC | 21:05 | |
| spatel | I may planning to use DPDK so i have to learn OVS | 21:05 |
| *** oomichi_ has joined #openstack-nova | 21:05 | |
| sean-k-mooney | if your starting out with ovs its simpler to start with kernel ovs and learn how to use ovs-dpdk after you are comfortable with openflow and ovs in general | 21:06 |
| *** BjoernT has quit IRC | 21:06 | |
| sean-k-mooney | it just helps not to add dpdk to the learning curve | 21:06 |
| sean-k-mooney | ovs is the default netrok backend for like 80% of all deployments in one form or another | 21:06 |
| *** kaisers1 has joined #openstack-nova | 21:07 | |
| spatel | sean-k-mooney: hmm | 21:07 |
| *** mchlumsky has quit IRC | 21:07 | |
| spatel | Is it possible i deploy ovs with general configuration and then slowly rollout ovs-dpdk compute by compute | 21:08 |
| *** kaisers has quit IRC | 21:09 | |
| spatel | Can i have mix environment like currently i have SR-IOV with general compute | 21:09 |
| sean-k-mooney | spatel: yes you can mix ovs and ovs-dpdk in the same cloud and you can also have sr-iov in that cloud | 21:10 |
| sean-k-mooney | generally you will use a different host aggage to seperate your dpdk nodes that need hugepages form the rest that dont | 21:11 |
| spatel | I think i should go with OVS so i can have all kind of function and feature available.. | 21:11 |
| spatel | what are the basic requirement for dpdk? | 21:12 |
| mriedem | efried: please don't fix the docs build break | 21:12 |
| mriedem | i just dumped 42 comments on that change | 21:12 |
| efried | mriedem: okay | 21:12 |
| spatel | NIC support, hugepages etc.. | 21:12 |
| sean-k-mooney | most nicst that support sriov support dpdk but there is a list here | 21:13 |
| sean-k-mooney | https://core.dpdk.org/supported/ | 21:13 |
| *** takamatsu has joined #openstack-nova | 21:13 | |
| sean-k-mooney | that is more a list of driver but you likely have a nic that is supported | 21:13 |
| sean-k-mooney | hugepages are needed for the guest and for the vswtich | 21:14 |
| sean-k-mooney | other then that there are no special requiremetns | 21:14 |
| *** hongbin has quit IRC | 21:14 | |
| mriedem | with that i think i'm done for the day | 21:14 |
| spatel | I have - 04:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01) | 21:14 |
| spatel | How much hugepage required for vswitch? | 21:15 |
| sean-k-mooney | spatel: yep they use the ixgbe dirver and are supported | 21:15 |
| spatel | My all compute nodes are 32G memory ( some 64G) | 21:15 |
| sean-k-mooney | dpending on your workload 2-4 G if you are using jumbo frams 1-2G is fine for standard 1500b packets | 21:16 |
| sean-k-mooney | i have tunned it to less in the past | 21:16 |
| sean-k-mooney | it jsut depends on your data rates and how long your queuse are going to be | 21:16 |
| *** luksky has joined #openstack-nova | 21:17 | |
| spatel | When you saying hugepage for vswitch does that means i have to give some dedicated hugepage to vSwitch right? | 21:17 |
| sean-k-mooney | spatel: https://developers.redhat.com/blog/2018/03/16/ovs-dpdk-hugepage-memory/ | 21:17 |
| sean-k-mooney | spatel: yes | 21:17 |
| spatel | Nice doc.. | 21:18 |
| sean-k-mooney | ovs-dpdk uses hugepage memroy to process packets so that it can DMA transfer packets directly to and from the nic bypassing the kernel | 21:18 |
| spatel | sean-k-mooney: do i also need to give some dedicated CPU to vSwitch? | 21:18 |
| *** mriedem has quit IRC | 21:19 | |
| sean-k-mooney | yes ideally 1 PMD(poll mode driver) core per numa node | 21:19 |
| *** markvoelker has joined #openstack-nova | 21:20 | |
| sean-k-mooney | ovs-dpdk scalse effectivly liniarly with the number of cpu cores you give it so you can tune it up or down depending on your data rates | 21:20 |
| spatel | I have two numa zone so i have to give 2 CPU core right? is that CPU core will be pin with vSwitch? | 21:20 |
| sean-k-mooney | yes dpdk will run in a bussy loop waiting for packets and will consume the full core | 21:21 |
| sean-k-mooney | even when idel | 21:21 |
| *** JamesBenson has joined #openstack-nova | 21:22 | |
| sean-k-mooney | it does that to miniumese latency but it basically means you cant run other thing on the core at the same time | 21:22 |
| sean-k-mooney | per packet ovs-dpdk is much more efficent then kernel ovs however kernel ovs use kernel thread to process the packet and does not consume cpus cycles when idel | 21:23 |
| sean-k-mooney | so its a trade off | 21:23 |
| sean-k-mooney | ovs-dpdk cpu usage will be fixed based on the cores you allcoate it | 21:23 |
| spatel | core allocation process is automatic or part of configuration? | 21:24 |
| sean-k-mooney | kernel ovs will use less or more cpus as needed includeing stealing cpu cycles form the guests if you do not tune the kernel to prevent that | 21:24 |
| sean-k-mooney | spatel: part of the configuration | 21:24 |
| spatel | ah! ok.. | 21:24 |
| *** markvoelker has quit IRC | 21:24 | |
| sean-k-mooney | you set a bit mask in the ovs db to choose what core dpdk is allows to use | 21:24 |
| spatel | Do i need to create neutron port for dpdk (like i am create currently for SR-IOV)? | 21:25 |
| sean-k-mooney | you can but no. it used vnic_type=normal which is the default | 21:25 |
| sean-k-mooney | so you dont need to precreate them | 21:25 |
| sean-k-mooney | form a user point of view its identiacl to linux bridge or kernel ovs | 21:25 |
| sean-k-mooney | other then the fact you need a flaovr with hugepages | 21:26 |
| spatel | Perfect!! i love that part, currently in SR-IOV i have to create port in advance to map with vms | 21:26 |
| sean-k-mooney | yes | 21:26 |
| sean-k-mooney | which is annoying but there are reason why we require that | 21:26 |
| spatel | anyway my all vms running with hugepages at moment | 21:26 |
| spatel | Question why does vms required hugepage for dpdk? | 21:27 |
| sean-k-mooney | it has to do with how the packeets are transfered between the vswitch and the vm | 21:27 |
| sean-k-mooney | we use a technology called vhost-user | 21:27 |
| sean-k-mooney | which allows the vswitch to mapp the virtio-rings( tx and rx queues) of the guest nic into the memory of the vswitch | 21:28 |
| spatel | Interesting.. | 21:29 |
| sean-k-mooney | to do that we need to use gest memroy that is mmap shared with an open file discripto, is contiguaon and preallcoated | 21:29 |
| sean-k-mooney | hugepages do all of the above | 21:29 |
| sean-k-mooney | technically you can acive the same wiht out hugepages | 21:29 |
| sean-k-mooney | its just slower, more work and not supported by openstack by defualt | 21:29 |
| spatel | Got it.. so hugepage has big role here | 21:30 |
| sean-k-mooney | if you are using a new enought ovs-dpdk you can actully get 0 copy packet transmistion betwwen the guest and the nic | 21:30 |
| openstackgerrit | Merged openstack/nova master: update constraints url https://review.opendev.org/664771 | 21:31 |
| sean-k-mooney | e.g. when the guest kernel rights the packet to the tx queue it is nver copyied and is drieectly trasmited by the nic similar to sriov | 21:31 |
| sean-k-mooney | 0 copy recive i think is also posibel but experimental | 21:31 |
| sean-k-mooney | that obviorsly depend on if you are using vxlan or vlans feature that may or may not be supported in your nic hardware | 21:32 |
| spatel | ah! ok.. in short it bypass the kernel | 21:32 |
| sean-k-mooney | yep and does everything in userspacce or in hardaware | 21:32 |
| spatel | Do you guys running dpdk in production? | 21:33 |
| spatel | very few people i found using dpdk in prod cloud | 21:33 |
| sean-k-mooney | well i dont run a production cloud. i just work on upstream stuff | 21:33 |
| sean-k-mooney | spatel: but redhat has customer using it in production yest | 21:33 |
| sean-k-mooney | *yes | 21:34 |
| sean-k-mooney | it is much less common then kernel ovs or sriov | 21:34 |
| spatel | Let see how successful i can run in my production. | 21:34 |
| sean-k-mooney | if you want the full intel pitch then https://software.intel.com/en-us/articles/yahoo-japan-accelerates-l7-performance-using-open-vswitch-with-dpdk | 21:35 |
| spatel | Thanks for that link.. | 21:35 |
| sean-k-mooney | hehe when i was working at intel i help some of the yahoo folk tuen there deployment alittle but after it was working i never heard form them how it worked out long term | 21:36 |
| spatel | sean-k-mooney: you are smarty pants :) | 21:37 |
| spatel | sean-k-mooney: hope dpdk support bonding too | 21:39 |
| sean-k-mooney | it does | 21:39 |
| sean-k-mooney | you can have dpdk do it(not recommended) or use ovs bonding of dpdk ports | 21:39 |
| sean-k-mooney | ovs bonding give you more contol and can be modifed while the switch is running | 21:40 |
| sean-k-mooney | dpdk bonding required the switch to be restareted, is less configurable but used to be slight faster at failover and throuput | 21:40 |
| sean-k-mooney | honestly however dpdk bonding not via ovs is an operations pain that is best avoid | 21:41 |
| sean-k-mooney | you should just use ovs bonding | 21:41 |
| sean-k-mooney | anyway im gong to call it a night o/ | 21:41 |
| *** slaweq has quit IRC | 21:43 | |
| *** luksky has quit IRC | 21:43 | |
| spatel | sean-k-mooney: Thank you for wonderful tips | 21:43 |
| spatel | Good night | 21:43 |
| *** slaweq has joined #openstack-nova | 21:44 | |
| *** mdbooth has joined #openstack-nova | 21:46 | |
| *** slaweq has quit IRC | 21:49 | |
| *** spatel has quit IRC | 22:03 | |
| *** bnemec has quit IRC | 22:05 | |
| *** mdbooth has quit IRC | 22:10 | |
| *** JamesBenson has quit IRC | 22:13 | |
| *** takamatsu has quit IRC | 22:25 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Functional recreate for revert_resize https://review.opendev.org/665253 | 22:25 |
| *** dave-mccowan has quit IRC | 22:26 | |
| *** slaweq has joined #openstack-nova | 22:36 | |
| *** oomichi_ has quit IRC | 22:40 | |
| *** bbowen has joined #openstack-nova | 22:40 | |
| *** oomichi_ has joined #openstack-nova | 22:40 | |
| *** slaweq has quit IRC | 22:41 | |
| *** rcernin has joined #openstack-nova | 22:45 | |
| openstackgerrit | Merged openstack/nova master: update comment on ignore_basepython_conflict https://review.opendev.org/665036 | 22:45 |
| *** mlavalle has quit IRC | 22:47 | |
| *** tesseract has quit IRC | 22:48 | |
| *** tkajinam has joined #openstack-nova | 22:51 | |
| *** ivve has quit IRC | 22:57 | |
| *** _erlon_ has quit IRC | 23:12 | |
| *** slaweq has joined #openstack-nova | 23:19 | |
| *** markvoelker has joined #openstack-nova | 23:22 | |
| *** markvoelker has quit IRC | 23:26 | |
| *** dave-mccowan has joined #openstack-nova | 23:40 | |
| *** artom has joined #openstack-nova | 23:40 | |
| *** JamesBenson has joined #openstack-nova | 23:47 | |
| *** JamesBenson has quit IRC | 23:51 | |
| *** slaweq has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!