| *** slaweq has joined #openstack-nova | 00:11 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova stable/rocky: Handle volume API failure in _post_live_migration https://review.openstack.org/611083 | 00:14 |
|---|---|---|
| *** gyee has quit IRC | 00:17 | |
| *** spatel has joined #openstack-nova | 00:31 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix best_match() deprecation warning https://review.openstack.org/611204 | 00:42 |
| *** slaweq has quit IRC | 00:45 | |
| *** lbragstad-503 is now known as lbragstad | 00:58 | |
| *** spatel has joined #openstack-nova | 01:07 | |
| *** hoangcx has quit IRC | 01:07 | |
| *** imacdonn has quit IRC | 01:09 | |
| *** slaweq has joined #openstack-nova | 01:12 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add API ref guideline for body text https://review.openstack.org/605628 | 01:14 |
| *** dave-mccowan has joined #openstack-nova | 01:19 | |
| *** tetsuro has joined #openstack-nova | 01:19 | |
| *** imacdonn has joined #openstack-nova | 01:22 | |
| *** erlon has quit IRC | 01:23 | |
| *** TuanDA has joined #openstack-nova | 01:24 | |
| *** mrsoul has quit IRC | 01:27 | |
| *** owalsh has quit IRC | 01:31 | |
| *** tetsuro has quit IRC | 01:32 | |
| *** owalsh has joined #openstack-nova | 01:32 | |
| *** jackyzhu has joined #openstack-nova | 01:32 | |
| *** hshiina has joined #openstack-nova | 01:35 | |
| *** slaweq has quit IRC | 01:44 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:48 | |
| *** spatel has quit IRC | 01:50 | |
| *** hongbin has joined #openstack-nova | 01:50 | |
| *** slaweq has joined #openstack-nova | 02:12 | |
| *** tetsuro has joined #openstack-nova | 02:19 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add nova-status upgrade check for consoles https://review.openstack.org/611214 | 02:35 |
| *** lei-zh has joined #openstack-nova | 02:41 | |
| *** Dinesh_Bhor has quit IRC | 02:44 | |
| *** slaweq has quit IRC | 02:44 | |
| *** lei-zh has quit IRC | 02:47 | |
| *** lei-zh has joined #openstack-nova | 02:48 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: Handle volume API failure in _post_live_migration https://review.openstack.org/611084 | 02:48 |
| *** whoami-rajat has joined #openstack-nova | 02:49 | |
| *** Dinesh_Bhor has joined #openstack-nova | 02:49 | |
| *** lei-zh1 has joined #openstack-nova | 02:51 | |
| *** psachin has joined #openstack-nova | 02:52 | |
| *** jackyzhu has quit IRC | 02:55 | |
| *** lei-zh1 has quit IRC | 02:55 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add rolling upgrade info to enable_consoleauth workaround reno https://review.openstack.org/611218 | 02:57 |
| *** slaweq has joined #openstack-nova | 03:11 | |
| *** dklyle has quit IRC | 03:25 | |
| *** dave-mccowan has quit IRC | 03:33 | |
| *** lei-zh has quit IRC | 03:34 | |
| *** lei-zh has joined #openstack-nova | 03:34 | |
| *** slaweq has quit IRC | 03:44 | |
| openstackgerrit | Merged openstack/nova master: Handle volume API failure in _post_live_migration https://review.openstack.org/609517 | 03:52 |
| openstackgerrit | Merged openstack/nova master: Increment versioning with pbr instruction https://review.openstack.org/610534 | 03:52 |
| *** tetsuro has quit IRC | 03:55 | |
| *** psachin has quit IRC | 03:56 | |
| *** hongbin has quit IRC | 04:04 | |
| *** hshiina has quit IRC | 04:04 | |
| *** hshiina has joined #openstack-nova | 04:04 | |
| *** Dinesh_Bhor has quit IRC | 04:07 | |
| *** slaweq has joined #openstack-nova | 04:11 | |
| *** Dinesh_Bhor has joined #openstack-nova | 04:39 | |
| *** psachin has joined #openstack-nova | 04:41 | |
| *** slaweq has quit IRC | 04:44 | |
| *** pcaruana has quit IRC | 04:46 | |
| *** lei-zh has quit IRC | 05:06 | |
| *** tetsuro has joined #openstack-nova | 05:10 | |
| *** slaweq has joined #openstack-nova | 05:12 | |
| *** janki has joined #openstack-nova | 05:15 | |
| *** lei-zh has joined #openstack-nova | 05:17 | |
| *** annp has quit IRC | 05:23 | |
| *** TuanDA has quit IRC | 05:23 | |
| *** lei-zh has quit IRC | 05:23 | |
| *** lei-zh has joined #openstack-nova | 05:24 | |
| *** slaweq has quit IRC | 05:30 | |
| *** slaweq has joined #openstack-nova | 05:31 | |
| *** ratailor has joined #openstack-nova | 05:39 | |
| *** gryf has quit IRC | 05:40 | |
| *** obre has quit IRC | 05:40 | |
| *** obre has joined #openstack-nova | 05:41 | |
| *** psachin has quit IRC | 05:43 | |
| *** Dinesh_Bhor has quit IRC | 05:49 | |
| *** slaweq has quit IRC | 05:54 | |
| *** icey has joined #openstack-nova | 05:55 | |
| *** Dinesh_Bhor has joined #openstack-nova | 06:01 | |
| *** cfriesen has quit IRC | 06:01 | |
| gmann | API office hour time | 06:02 |
| gmann | alex_xu: ping | 06:02 |
| alex_xu | gmann: pong | 06:02 |
| gmann | #startmeeting nova api | 06:02 |
| openstack | Meeting started Wed Oct 17 06:02:30 2018 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. | 06:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 06:02 |
| *** openstack changes topic to " (Meeting topic: nova api)" | 06:02 | |
| openstack | The meeting name has been set to 'nova_api' | 06:02 |
| gmann | who else here today ? | 06:02 |
| *** TuanDA has joined #openstack-nova | 06:03 | |
| gmann | #link https://wiki.openstack.org/wiki/Meetings/NovaAPI#Agenda_for_next_Office_hours | 06:04 |
| gmann | agenda ^^ | 06:04 |
| gmann | #topic API Subteam Tracking | 06:04 |
| *** openstack changes topic to "API Subteam Tracking (Meeting topic: nova api)" | 06:04 | |
| gmann | #link https://etherpad.openstack.org/p/stein-nova-subteam-tracking | 06:04 |
| gmann | 1. Boot instance specific storage backend | 06:06 |
| gmann | this is merged which was in runway | 06:06 |
| alex_xu | yup, we done last week | 06:06 |
| gmann | yeah | 06:06 |
| *** slaweq has joined #openstack-nova | 06:06 | |
| gmann | https://review.openstack.org/#/q/topic:bp/api-extensions-merge-stein+status:open | 06:06 |
| gmann | this is also next in runway but i saw you already reviewed most of them | 06:07 |
| alex_xu | yea, part of review for the last patch | 06:07 |
| alex_xu | https://review.openstack.org/#/c/607088/5/nova/api/openstack/compute/views/servers.py | 06:08 |
| alex_xu | gmann: I guess you just forget to remove one line code | 06:08 |
| gmann | yeah, i am sure i did but got restore from rebse form PS4 to 5 | 06:09 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Remove more code related to extensions and testing https://review.openstack.org/607088 | 06:10 |
| gmann | alex_xu: ^^ done | 06:10 |
| gmann | alex_xu: thanks for reviewing that series. | 06:10 |
| alex_xu | gmann: np, I'm done also :) | 06:11 |
| gmann | alex_xu: thanks :) | 06:11 |
| gmann | other than that there is no updated from other BP code. | 06:11 |
| gmann | spec review is soemthing we can do. | 06:11 |
| gmann | melwitt set the 23rd as spec review day. | 06:12 |
| alex_xu | yea | 06:12 |
| gmann | i will cover those during spec day | 06:12 |
| gmann | that is all from tracking topic, alex_xu you have anything else | 06:13 |
| alex_xu | no more from me | 06:13 |
| gmann | ok | 06:15 |
| gmann | #topic Bug Triage/Discussion | 06:15 |
| *** openstack changes topic to "Bug Triage/Discussion (Meeting topic: nova api)" | 06:15 | |
| *** Dinesh_Bhor has quit IRC | 06:15 | |
| *** mmedvede has quit IRC | 06:15 | |
| *** obre has quit IRC | 06:16 | |
| *** mmedvede has joined #openstack-nova | 06:16 | |
| gmann | no response from previous bug we discussed so i marked incomplete as of now | 06:16 |
| *** obre has joined #openstack-nova | 06:17 | |
| *** dpawlik has joined #openstack-nova | 06:18 | |
| alex_xu | ok | 06:19 |
| alex_xu | i'm planning to trigger this by a fake cell timeout exception, but I didn't get a chance | 06:20 |
| *** brinzhang has joined #openstack-nova | 06:21 | |
| gmann | this one ? https://bugs.launchpad.net/nova/+bug/1789382 | 06:21 |
| openstack | Launchpad bug 1789382 in OpenStack Compute (nova) "openstack server list error" [Undecided,Incomplete] | 06:21 |
| alex_xu | yes | 06:21 |
| *** Dinesh_Bhor has joined #openstack-nova | 06:23 | |
| gmann | ok, not sure it will be reproducible on that but god to give try | 06:23 |
| gmann | good | 06:23 |
| gmann | next is new bug | 06:24 |
| gmann | https://bugs.launchpad.net/nova/+bug/1791478 | 06:24 |
| openstack | Launchpad bug 1791478 in OpenStack Compute (nova) "nova-api get 504 when keepalived restart" [Undecided,New] | 06:24 |
| *** gryf has joined #openstack-nova | 06:24 | |
| gmann | this is 504 gateway timeout | 06:24 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on updated_at when getting instance action records https://review.openstack.org/607801 | 06:26 |
| gmann | it seems nova-api does not recognize the new node | 06:27 |
| gmann | untill it is running on old node | 06:27 |
| gmann | i am not sure why, alex_xu you ? | 06:28 |
| gmann | also not sure if this is nova-api isssue, or keepalive switch ip issue | 06:29 |
| alex_xu | why nova api should recognize the new node? | 06:29 |
| gmann | might be he means keepalive does not recognize the new node nova-api service (and does not switch the vip)? | 06:31 |
| *** tetsuro has quit IRC | 06:32 | |
| alex_xu | doesn't sounds like nova api problem, it should be his ha configuration problem | 06:33 |
| gmann | yea, sounds like | 06:33 |
| *** pcaruana has joined #openstack-nova | 06:34 | |
| gmann | added the comment on bug | 06:35 |
| gmann | and keeping it open until next week and then we can make it invalid or incomplete | 06:35 |
| gmann | last new bug is this- https://bugs.launchpad.net/nova/+bug/1793606 | 06:35 |
| openstack | Launchpad bug 1793606 in OpenStack Compute (nova) "Setting vendordata_providers to DynamicJSON causes config drive to stop working" [Undecided,New] | 06:35 |
| gmann | i remember, there was some fix for DynamicJSON option. | 06:37 |
| alex_xu | i'm not familar with this one | 06:39 |
| gmann | me too, but i can try to reproduce this on local tomorrow and see whats happening | 06:41 |
| *** moshele has joined #openstack-nova | 06:43 | |
| gmann | its default value is StaticJSON https://github.com/openstack/nova/blob/042f08ee968fc83b80e554efe58270c8dfd6265c/nova/conf/api.py#L84 | 06:43 |
| gmann | i will check this later and reply on bug | 06:44 |
| gmann | that's all on bug side for today | 06:44 |
| gmann | alex_xu: you have any bug to discuss or we can move to open discussion | 06:44 |
| alex_xu | we can move on | 06:45 |
| gmann | #topic Open Discussion | 06:46 |
| *** openstack changes topic to "Open Discussion (Meeting topic: nova api)" | 06:46 | |
| gmann | i have 1 quick topic | 06:47 |
| gmann | policy related change in upgrade-check CLI | 06:47 |
| gmann | #link http://lists.openstack.org/pipermail/openstack-dev/2018-October/135779.html | 06:47 |
| gmann | ^^ i replied to matt email on upgrade-check and i feel most of the policy changes are candidate for upgrade-check CLi | 06:47 |
| gmann | alex_xu: what you say ? | 06:48 |
| alex_xu | that is good idea | 06:48 |
| alex_xu | better than operator to read the boring release note | 06:48 |
| gmann | they can be checked by comparing the previous release policy sample file json and new one and build the upgrade msg | 06:49 |
| gmann | yeah | 06:49 |
| gmann | or some more optimize way. | 06:49 |
| alex_xu | and some action to take for their custom policy config | 06:49 |
| gmann | yeah | 06:50 |
| gmann | that's all on this from my side. | 06:51 |
| alex_xu | me too | 06:51 |
| gmann | anything else alex_xu or we can close the office hour | 06:51 |
| gmann | ok | 06:51 |
| gmann | thank alex_xu for joining | 06:52 |
| gmann | #endmeeting | 06:52 |
| *** openstack changes topic to "Current runways: use-nested-allocation-candidates -- This channel is for Nova development. For support of Nova deployments, please use #openstack." | 06:52 | |
| openstack | Meeting ended Wed Oct 17 06:52:10 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 06:52 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/nova_api/2018/nova_api.2018-10-17-06.02.html | 06:52 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/nova_api/2018/nova_api.2018-10-17-06.02.txt | 06:52 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/nova_api/2018/nova_api.2018-10-17-06.02.log.html | 06:52 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on updated_at when getting migrations https://review.openstack.org/607798 | 06:59 |
| *** slaweq has quit IRC | 06:59 | |
| *** slaweq has joined #openstack-nova | 06:59 | |
| openstackgerrit | liuming proposed openstack/nova master: Deletes evacuated instance files when source host is ok https://review.openstack.org/605987 | 07:02 |
| *** adrianc has joined #openstack-nova | 07:02 | |
| openstackgerrit | Merged openstack/nova master: Handle online_data_migrations exceptions https://review.openstack.org/608091 | 07:07 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on updated_at when getting instance action records https://review.openstack.org/607801 | 07:08 |
| *** rcernin has quit IRC | 07:09 | |
| amorin | bauzas: I found out my issue about live migration with image not found | 07:11 |
| amorin | I confirm that the bug is fixed on my newton infra | 07:11 |
| *** hamdyk has joined #openstack-nova | 07:12 | |
| openstackgerrit | Jie Li proposed openstack/nova-specs master: Support volume-backed server rebuild https://review.openstack.org/532407 | 07:13 |
| *** slaweq has quit IRC | 07:13 | |
| *** hshiina_ has joined #openstack-nova | 07:13 | |
| *** slaweq has joined #openstack-nova | 07:14 | |
| *** slaweq has quit IRC | 07:14 | |
| *** slaweq has joined #openstack-nova | 07:14 | |
| openstackgerrit | Jie Li proposed openstack/nova-specs master: Support volume-backed server rebuild https://review.openstack.org/532407 | 07:16 |
| *** hshiina has quit IRC | 07:17 | |
| *** helenafm has joined #openstack-nova | 07:19 | |
| *** ralonsoh has joined #openstack-nova | 07:20 | |
| *** Luzi has joined #openstack-nova | 07:26 | |
| *** jpena|off is now known as jpena | 07:30 | |
| *** jangutter has quit IRC | 07:34 | |
| *** jangutter has joined #openstack-nova | 07:34 | |
| *** brinzhang has quit IRC | 07:41 | |
| *** brinzhang has joined #openstack-nova | 07:42 | |
| bauzas | good morning Nova | 07:42 |
| bauzas | amorin: cool, what was the issue ? | 07:42 |
| gibi | bauzas: good morning | 07:42 |
| *** takashin has quit IRC | 08:06 | |
| openstackgerrit | huanhongda proposed openstack/nova master: Fix instance delete stuck in deleting task_state https://review.openstack.org/598084 | 08:09 |
| *** elod_off is now known as elod | 08:13 | |
| *** slaweq has quit IRC | 08:14 | |
| *** slaweq has joined #openstack-nova | 08:15 | |
| *** brinzhang has quit IRC | 08:18 | |
| *** alexchadin has joined #openstack-nova | 08:20 | |
| *** brinzhang has joined #openstack-nova | 08:20 | |
| *** Dinesh_Bhor has quit IRC | 08:25 | |
| *** tssurya has joined #openstack-nova | 08:25 | |
| *** TuanDA has quit IRC | 08:36 | |
| *** TuanDA has joined #openstack-nova | 08:49 | |
| *** hshiina_ has quit IRC | 08:56 | |
| *** cfriesen has joined #openstack-nova | 08:57 | |
| *** Dinesh_Bhor has joined #openstack-nova | 09:06 | |
| *** dpawlik has quit IRC | 09:10 | |
| *** dtantsur|afk is now known as dtantsur | 09:12 | |
| *** dpawlik has joined #openstack-nova | 09:29 | |
| *** mikeoschen has quit IRC | 09:34 | |
| *** lei-zh has quit IRC | 09:38 | |
| *** Dinesh_Bhor has quit IRC | 09:42 | |
| *** adrianc has quit IRC | 09:53 | |
| *** TuanDA has quit IRC | 09:59 | |
| *** adrianc has joined #openstack-nova | 10:09 | |
| *** yikun has joined #openstack-nova | 10:19 | |
| *** wznoinsk has joined #openstack-nova | 10:21 | |
| *** brinzh has joined #openstack-nova | 10:26 | |
| *** brinzhang has quit IRC | 10:30 | |
| *** Dinesh_Bhor has joined #openstack-nova | 10:30 | |
| openstackgerrit | Merged openstack/nova master: Add recreate test for bug 1798163 https://review.openstack.org/611113 | 10:38 |
| openstack | bug 1798163 in OpenStack Compute (nova) "Placement incomplete consumers online migration fails" [Critical,In progress] https://launchpad.net/bugs/1798163 - Assigned to Matt Riedemann (mriedem) | 10:38 |
| *** pooja_jadhav has quit IRC | 10:41 | |
| kashyap | stephenfin: Hey, is there a less-strain-on-the-eye version of this? -- https://docs.openstack.org/nova/rocky/configuration/sample-config.html | 10:41 |
| kashyap | Renderings for older releases had a table for config files | 10:42 |
| kashyap | Now it is all one big sphaghetti noodle "sample-config.html" file | 10:42 |
| kashyap | Ah, found it - https://docs.openstack.org/nova/rocky/configuration/config.html | 10:44 |
| kashyap | stephenfin: Disregard me | 10:44 |
| *** tbachman has quit IRC | 10:44 | |
| *** tbachman has joined #openstack-nova | 10:44 | |
| *** brinzh has quit IRC | 10:48 | |
| *** Dinesh_Bhor has quit IRC | 10:50 | |
| *** tbachman has quit IRC | 10:50 | |
| *** erlon has joined #openstack-nova | 11:06 | |
| *** panda is now known as panda|lunch | 11:18 | |
| *** moshele has quit IRC | 11:22 | |
| *** pcaruana has quit IRC | 11:28 | |
| *** ShilpaSD has quit IRC | 11:28 | |
| *** moshele has joined #openstack-nova | 11:29 | |
| *** ratailor has quit IRC | 11:35 | |
| *** pooja_jadhav has joined #openstack-nova | 11:35 | |
| *** jpena is now known as jpena|lunch | 11:37 | |
| *** lei-zh has joined #openstack-nova | 11:39 | |
| *** READ10 has quit IRC | 11:41 | |
| *** mvkr has quit IRC | 11:46 | |
| *** dtantsur is now known as dtantsur|brb | 11:48 | |
| *** slaweq has quit IRC | 11:54 | |
| *** slaweq has joined #openstack-nova | 11:54 | |
| *** janki has quit IRC | 11:57 | |
| *** panda|lunch is now known as panda | 12:03 | |
| *** moshele has quit IRC | 12:04 | |
| openstackgerrit | Martin Midolesov proposed openstack/nova master: Implementing graceful shutdown. https://review.openstack.org/608704 | 12:04 |
| *** moshele has joined #openstack-nova | 12:05 | |
| *** tbachman has joined #openstack-nova | 12:08 | |
| *** ShilpaSD has joined #openstack-nova | 12:10 | |
| *** dave-mccowan has joined #openstack-nova | 12:13 | |
| *** eharney has joined #openstack-nova | 12:16 | |
| openstackgerrit | Merged openstack/nova master: Use unique consumer_id when doing online data migration https://review.openstack.org/611115 | 12:18 |
| *** pcaruana has joined #openstack-nova | 12:24 | |
| *** lei-zh has quit IRC | 12:35 | |
| *** jpena|lunch is now known as jpena | 12:44 | |
| *** moshele has quit IRC | 12:49 | |
| *** moshele has joined #openstack-nova | 12:51 | |
| *** mvkr has joined #openstack-nova | 12:53 | |
| *** dtantsur|brb is now known as dtantsur | 12:55 | |
| *** janki has joined #openstack-nova | 12:58 | |
| openstackgerrit | Martin Midolesov proposed openstack/nova master: Implementing graceful shutdown. https://review.openstack.org/608704 | 13:02 |
| *** lei-zh has joined #openstack-nova | 13:02 | |
| openstackgerrit | Merged openstack/nova master: Fix formatting non-templated cell URLs with no config https://review.openstack.org/611094 | 13:05 |
| *** mriedem has joined #openstack-nova | 13:10 | |
| *** mchlumsky has joined #openstack-nova | 13:15 | |
| *** dklyle has joined #openstack-nova | 13:16 | |
| pooja_jadhav | mriedem: https://review.openstack.org/#/c/608771/1/nova/tests/functional/test_boot_from_volume.py | 13:21 |
| pooja_jadhav | In this functional tests, is it creating instance in real?? | 13:21 |
| mriedem | it creates a server in the sqlite database using the fake virt driver | 13:22 |
| mriedem | i'm not sure what you mean by "real" | 13:22 |
| amorin | bauzas the issue is that we are upgrading our openstack from juno to newton | 13:22 |
| pooja_jadhav | real means in actual environment | 13:23 |
| pooja_jadhav | like | 13:23 |
| amorin | and some of our hosts were not configured correctly AFAIK | 13:23 |
| pooja_jadhav | when we do nova list, we can see that instance | 13:23 |
| amorin | maybe that was because nova rely on glanceclient, and glanceclient was not upgraded | 13:23 |
| mriedem | pooja_jadhav: well the test doesn't list instances, but it's able to show the instance, otherwise it wouldn't be able to wait for its status to be ACTIVE | 13:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Add recreate test for bug 1798163 https://review.openstack.org/611314 | 13:25 |
| openstack | bug 1798163 in OpenStack Compute (nova) rocky "Placement incomplete consumers online migration fails" [Critical,Confirmed] https://launchpad.net/bugs/1798163 - Assigned to lihuayuliang (lihuayuliang) | 13:25 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Use unique consumer_id when doing online data migration https://review.openstack.org/611315 | 13:25 |
| mriedem | lyarwood: bauzas: pretty serious upgrade issue if you can review those backports quick ^ | 13:25 |
| pooja_jadhav | mriedem: then after getting test passed, these instance should be deleted. Is it deleting by any existing mechanism or we need to delete by calling delete API.?? | 13:26 |
| mriedem | pooja_jadhav: each test case has the db setup and torn down | 13:26 |
| mriedem | so we don't need to explicitly delete resources since each test case has a fresh db | 13:26 |
| pooja_jadhav | mriedem: Ohk, Thank you :) | 13:27 |
| pooja_jadhav | mriedem: Can u plz locate to the code where this db setup and tear down is added for the functional tests? | 13:30 |
| *** dklyle has quit IRC | 13:31 | |
| mriedem | pooja_jadhav: see nova.test.TestCase.setUp | 13:31 |
| mriedem | the database fixtures in there | 13:31 |
| *** lei-zh has quit IRC | 13:32 | |
| pooja_jadhav | mriedem : thnnks :) | 13:32 |
| *** irclogbot_4 has joined #openstack-nova | 13:32 | |
| *** dims has quit IRC | 13:40 | |
| *** munimeha1 has joined #openstack-nova | 13:47 | |
| *** jmlowe has quit IRC | 13:49 | |
| *** mgoddard has quit IRC | 13:49 | |
| *** eharney has quit IRC | 13:51 | |
| openstackgerrit | Martin Midolesov proposed openstack/nova master: Implementing graceful shutdown. https://review.openstack.org/608704 | 13:53 |
| *** mhen has joined #openstack-nova | 13:54 | |
| openstackgerrit | Merged openstack/nova master: Fix NoneType error in _notify_volume_usage_detach https://review.openstack.org/609518 | 13:54 |
| mriedem | tssurya: you want to backport this to rocky? https://review.openstack.org/#/c/611094/ | 13:54 |
| *** mgoddard has joined #openstack-nova | 13:54 | |
| tssurya | mriedem: yea, doing it | 13:55 |
| mriedem | artom: i see you wasted no time https://review.openstack.org/#/q/Icb0bdaf454935b3713c35339394d260b33520de5 | 13:55 |
| dansmith | mriedem: we have some pressure on that one | 13:56 |
| dansmith | rocky should be okay now yeah? | 13:56 |
| tssurya | yea | 13:56 |
| mriedem | dansmith: i just +2d it | 13:57 |
| *** jmlowe has joined #openstack-nova | 13:58 | |
| *** irclogbot_4 has quit IRC | 13:58 | |
| dansmith | ah was just looking at the conflict, | 13:58 |
| dansmith | but I see you called it out | 13:58 |
| *** irclogbot_4 has joined #openstack-nova | 13:59 | |
| mriedem | anyone know if tripleo is running nova-status upgrade check yet? | 14:03 |
| mriedem | either on install and/or upgrade? | 14:03 |
| *** Luzi has quit IRC | 14:03 | |
| *** dims has joined #openstack-nova | 14:03 | |
| mriedem | looks like only OSA and kolla-ansible so far | 14:04 |
| *** READ10 has joined #openstack-nova | 14:05 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix NoneType error in _notify_volume_usage_detach https://review.openstack.org/611326 | 14:08 |
| openstackgerrit | Surya Seetharaman proposed openstack/nova stable/rocky: Fix formatting non-templated cell URLs with no config https://review.openstack.org/611327 | 14:09 |
| *** awaugama has joined #openstack-nova | 14:09 | |
| *** irclogbot_4 has quit IRC | 14:11 | |
| bauzas | mriedem: lyarwood is on PTO for the next two weeks, but lemme look | 14:11 |
| *** sapd1_ has quit IRC | 14:13 | |
| *** sapd1_ has joined #openstack-nova | 14:16 | |
| *** dims has quit IRC | 14:16 | |
| tssurya | if any cores have some time (other than mriedem and efried): would be good to have this in https://review.openstack.org/#/c/611162/ | 14:16 |
| mriedem | i was just going to lobby for that :) | 14:17 |
| mriedem | since it's an upgrade impact | 14:17 |
| tssurya | :) | 14:17 |
| *** dims has joined #openstack-nova | 14:19 | |
| bauzas | tssurya: mriedem: mmm, is that only an Ironic thing ? | 14:21 |
| bauzas | mriedem: I was looking for like 2 weeks why some ComputeNode UUID was changing from a libvirt node | 14:21 |
| tssurya | its an only ironic driver thing | 14:21 |
| bauzas | anyway, +Wing | 14:21 |
| tssurya | thanks bauzas | 14:22 |
| bauzas | tssurya: mriedem: can't see a stable/rocky one, please ping me the change once done | 14:22 |
| tssurya | yea proposing one now | 14:22 |
| bauzas | of course, I'll +2 it once the master lands | 14:22 |
| mriedem | bauzas: this is a fun reqspec regression thing too https://review.openstack.org/#/q/topic:bug/1797580+(status:open+OR+status:merged) | 14:24 |
| mriedem | goes back to queens | 14:24 |
| bauzas | mriedem: after 2 weeks of internal bug triage, I love you | 14:24 |
| bauzas | because you help me doing other stuff | 14:24 |
| tssurya | yea we hit that too the marker stuff :( | 14:24 |
| mriedem | ^ isn't marker stuff | 14:24 |
| bauzas | oh this one | 14:25 |
| bauzas | the requested_destination field | 14:25 |
| mriedem | tssurya: https://review.openstack.org/#/c/610974/ | 14:25 |
| tssurya | ah hehe yea | 14:25 |
| tssurya | sorry | 14:25 |
| mriedem | but yeah would be good to get that in also since it affects upgrades | 14:26 |
| mriedem | johnthetubaguy: are you around? | 14:26 |
| openstackgerrit | Surya Seetharaman proposed openstack/nova stable/rocky: Ignore uuid if already set in ComputeNode.update_from_virt_driver https://review.openstack.org/611337 | 14:26 |
| artom | mriedem, yeah, like I said, downstream pressure :) | 14:27 |
| tssurya | bauzas, efried: ^ the backport to rocky for after the master merges | 14:27 |
| mriedem | +1 | 14:28 |
| stephenfin | Can someone help me understand what's going on here? paste.openstack.org/show/732338/ | 14:34 |
| stephenfin | Was discussing it with bauzas earlier today. We've configured 'cpu_allocation=1' and yet it still seems like overallocation is happening | 14:34 |
| bauzas | stephenfin: like I said to you, we should check the instances number | 14:36 |
| *** openstackgerrit has quit IRC | 14:36 | |
| stephenfin | bauzas: Yeah, there's one instance on that host before we do the host-evacuate. Once we do that, two instances land on it | 14:36 |
| stephenfin | One instance has 2 vcpu, the other has 1 vcpu | 14:37 |
| bauzas | given vcpus_used is generated by looking up the instances | 14:37 |
| bauzas | host-evacuate is racy | 14:37 |
| bauzas | you should really avoid it, and do two evacuations concurrently | 14:37 |
| bauzas | I thought we commented that btw. | 14:37 |
| stephenfin | Yup, vcpus_used appears to be correct. Compare lines 24-29 to 90-95 | 14:38 |
| bauzas | https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/contrib/host_evacuate.py | 14:38 |
| stephenfin | The second hypervisor (dcfb3fb7) goes from 5/6 used to 7/6 | 14:39 |
| *** logan_ has joined #openstack-nova | 14:40 | |
| stephenfin | bauzas: But its done client side. It seems odd that the scheduler wouldn't honour overcommit ratios just because the requests came in at the same time. Is this known behavior? | 14:41 |
| * stephenfin can't remember the last time he booted an instance without dedicated cores, heh | 14:41 | |
| *** hamzy has quit IRC | 14:41 | |
| mriedem | is the target host in an aggregate with an allocation ratio metadata? | 14:42 |
| stephenfin | sec - checking | 14:42 |
| mriedem | compute-0.localdomain was forced down but was the service actually stopped? | 14:42 |
| bauzas | otp but looking | 14:44 |
| mriedem | because vcpus_used didn't change on compute-0 after the evacuation | 14:44 |
| mriedem | though i guess i wouldn't expect them to until compute-0 is restarted and removes locally evacuated instances | 14:45 |
| mriedem | but not really safe to have the guest running on the source and dest at the same time... | 14:45 |
| stephenfin | I don't see vcpus_used reported for compute-0 in those logs. Mixing it up with compute-1? | 14:45 |
| mriedem | oh yeah you're right | 14:46 |
| mriedem | pi | 14:46 |
| mriedem | you're just wondering why they are evacuated to compute-2 rather than compute-0? | 14:46 |
| mriedem | or not spread between them? | 14:46 |
| stephenfin | spread between compute-1 and compute-2 | 14:46 |
| stephenfin | Well | 14:47 |
| mriedem | right you expect compute_0_instance_1_vcpu to go to compute-1 and compute_0_instance_2_vcpu to go to compute-2 | 14:47 |
| mriedem | which would fill up each of those computes | 14:47 |
| stephenfin | More that the second migration wouldn't fail because it would bump compute-2 over our overallocation ratio | 14:47 |
| stephenfin | Sure, though the opposite could also happen | 14:47 |
| mriedem | well, which release is this? is the CoreFilter enabled? | 14:47 |
| awaugama | I'm the one who tried it, the hosts were not in aggregates | 14:48 |
| stephenfin | so compute_0_instance_1_vcpu goes to compute-2 and then compute_0_instance_2_vcpu gets rejected | 14:48 |
| awaugama | I can retry with that | 14:48 |
| mriedem | if the CoreFilter is not enabled, there will be no cpu limits sent to the compute to claim in the RT | 14:48 |
| mriedem | which means you can overcommit | 14:48 |
| mriedem | if this is pike+, the filter scheduler should be claiming allocations in placement and we don't need to care about vcpu claims in the RT | 14:49 |
| mriedem | so i'd take a look at your resource provider inventory/allocations | 14:49 |
| *** munimeha1 has quit IRC | 14:49 | |
| *** knikolla has quit IRC | 14:49 | |
| *** devananda has quit IRC | 14:49 | |
| *** vabada has quit IRC | 14:49 | |
| *** alex_xu has quit IRC | 14:49 | |
| *** logan- has quit IRC | 14:49 | |
| mriedem | using the osc-placement commands | 14:49 |
| *** logan_ is now known as logan- | 14:49 | |
| mriedem | https://docs.openstack.org/osc-placement/latest/ | 14:50 |
| *** eharney has joined #openstack-nova | 14:50 | |
| awaugama | ok thanks | 14:50 |
| *** hamdyk has quit IRC | 14:51 | |
| *** openstackgerrit has joined #openstack-nova | 14:52 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Handle missing marker during online data migration https://review.openstack.org/611343 | 14:52 |
| *** mlavalle has joined #openstack-nova | 14:53 | |
| awaugama | mriedem: sorry I'm confused about something. you're saying placement has it's own ratios that will cause the configured compute node ratios to be ignored? | 14:56 |
| dansmith | awaugama: nova's ratios are mostly irrelevant now, yes | 14:57 |
| awaugama | so that's where the corefilter comes in? | 14:58 |
| mriedem | well, | 14:58 |
| mriedem | is the CoreFilter enabled? it shouldn't be. | 14:58 |
| mriedem | if you're >=pike and using the filter scheduler | 14:59 |
| mriedem | no one told me what release this is | 14:59 |
| awaugama | Sorry, this is Rocky | 14:59 |
| mriedem | the compute service will create a resource provider in the placement service and report it's inventory (vcpu/ram/disk) and allocation ratios for each of those resource classes | 14:59 |
| mriedem | the scheduler will atomically allocate resources from each provider during the evacuate | 15:00 |
| mriedem | so it should only overcommit if the VCPU inventory allocation_ratio in placement is > 1.0 | 15:00 |
| mriedem | so, find the uuid for the compute-2 resource provider using "openstack resource provider list" | 15:00 |
| mriedem | then using that uuid, check the inventory for that provider: openstack resource provider inventory show $uuid | 15:01 |
| mriedem | sorry, openstack resource provider inventory list $uuid | 15:01 |
| mriedem | if the allocation_ratio for the VCPU resource class is 1.0 and we still overcommitted, then something else is going on | 15:02 |
| awaugama | is there a microversion I need to use for the openstack cli? openstack resource provider list is giving me no results | 15:03 |
| mriedem | hmm, shouldn't need to | 15:04 |
| mriedem | are you using admin creds? | 15:04 |
| mriedem | you'd get a 403 otherwise | 15:04 |
| mriedem | if there are no resource providers....you shouldn't have been able to evacuate, you'd get a NoValidHost | 15:04 |
| awaugama | Sorry, I'm getting that openstack resource provider list is not an openstack command | 15:05 |
| awaugama | wasn't clear with no results | 15:05 |
| mriedem | you probably don't have the osc-placement plugin installed | 15:06 |
| mriedem | i'm assuming you guys have an rpm for that | 15:06 |
| awaugama | yeah, I've done that before, it's not enabled by default. I'll have to find my logs for that | 15:06 |
| openstackgerrit | Matthew Edmonds proposed openstack/nova master: Use tempfile for powervm config drive https://review.openstack.org/610174 | 15:07 |
| *** slaweq has quit IRC | 15:09 | |
| *** slaweq has joined #openstack-nova | 15:10 | |
| *** hamzy has joined #openstack-nova | 15:12 | |
| *** janki has quit IRC | 15:13 | |
| openstackgerrit | Ivaylo Mitev proposed openstack/nova master: VMware: OVA and StrOpt images as VM templates https://review.openstack.org/609736 | 15:15 |
| *** alexchadin has quit IRC | 15:15 | |
| *** liuyulong has joined #openstack-nova | 15:16 | |
| *** k_mouza has joined #openstack-nova | 15:17 | |
| *** slaweq has quit IRC | 15:21 | |
| *** slaweq has joined #openstack-nova | 15:22 | |
| mriedem | efried: you can drop the -2 on https://review.openstack.org/#/c/521041/ | 15:26 |
| efried | ... | 15:26 |
| *** moshele has quit IRC | 15:27 | |
| efried | mriedem: Is this going to be filed under reshaper or vgpu? | 15:27 |
| mriedem | i left a comment that it should be reshaper i think | 15:27 |
| efried | If that's the case, then the -2 should be dropped once the commit message references that bp? | 15:29 |
| awaugama | mriedem: The allocation ratio for VCPU is 16. that's where the issue is coming from | 15:29 |
| mriedem | awaugama: aha | 15:29 |
| mriedem | awaugama: that's the compute_nodes.cpu_allocation_ratio for compute-2 in the nova cell db? | 15:30 |
| mriedem | efried: just -1 | 15:30 |
| mriedem | awaugama: *what's | 15:30 |
| awaugama | checking | 15:31 |
| mriedem | you might be missing this fix https://github.com/openstack/nova/commit/01265b98c4cd2b1377e891a06ce748fc6f8f3425 | 15:31 |
| mriedem | that went into 18.0.1 | 15:31 |
| mriedem | if cpu_allocation_ratio is 0.0 in the compute_nodes table, it will default to 16.0 in code and that's what gets put into the resource provider VCPU inventory in placement | 15:32 |
| *** macza has joined #openstack-nova | 15:39 | |
| *** vdrok has quit IRC | 15:41 | |
| *** vdrok has joined #openstack-nova | 15:42 | |
| *** jpena is now known as jpena|off | 15:47 | |
| *** itlinux has joined #openstack-nova | 15:49 | |
| mnaser | any stable-maint folks around to push https://review.openstack.org/#/c/611314/1 and https://review.openstack.org/#/c/611315/1 out? :) | 15:53 |
| *** Luzi has joined #openstack-nova | 15:55 | |
| *** k_mouza has quit IRC | 15:57 | |
| artom | Dammit what do I need to mock so that can_send_version from compute rpcapi returns what I want >_< | 15:58 |
| *** hamzy has quit IRC | 16:01 | |
| dansmith | mnaser: yes | 16:01 |
| *** hamzy has joined #openstack-nova | 16:01 | |
| mnaser | dansmith: thank you, didn't want to keep carrying a patch :) | 16:01 |
| * dansmith nods | 16:03 | |
| efried | artom: Have you tried mock.patch('oslo_messaging.rpc.client.RPCClient.can_send_version') ? | 16:06 |
| efried | artom: Or you could mock nova.rpc.get_client and then set its return_value.can_send_version.return_value | 16:08 |
| efried | depending how far back you want to go. | 16:08 |
| *** priteau has joined #openstack-nova | 16:08 | |
| artom | efried, so, I was listing the mock_calls incorrectly, turns out mocking oslo_messaging.RPClient.can_send_version is what I need | 16:08 |
| mriedem | another clean rocky backport ready to go https://review.openstack.org/#/c/611327/ | 16:09 |
| artom | I can see the call I care about in here. New problem: it seems to be having no effect :( | 16:09 |
| efried | artom: cool. | 16:09 |
| efried | with that, I must leave you. | 16:09 |
| *** efried is now known as efried_pto | 16:09 | |
| artom | efried, hehe, thanks :) enjoy | 16:09 |
| awaugama | mriedem: cpu_allocation_ratio is set to 1 in the db | 16:10 |
| awaugama | sorry got yanked into a meeting | 16:10 |
| mriedem | awaugama: hmm | 16:10 |
| awaugama | for compute node 2 | 16:10 |
| mriedem | well this is the code that sets the cpu_allocation_ratio in the inventory record we send to placement https://github.com/openstack/nova/blob/stable/rocky/nova/compute/resource_tracker.py#L106 | 16:11 |
| mriedem | unless we have some code in rocky that is using a cache and thinks we don't have any updates... | 16:11 |
| mriedem | i.e. the cache might say we already have that allocation ratio even though placement doesn't | 16:11 |
| awaugama | stephenfin: is that the code you saw earlier? | 16:12 |
| stephenfin | awaugama: Yup, that's the one | 16:12 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: fixtures: Track volume attachments within CinderFixtureNewAttachFlow https://review.openstack.org/587013 | 16:12 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add regression test for bug#1784353 https://review.openstack.org/587014 | 16:12 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: conductor: Recreate volume attachments during a reschedule https://review.openstack.org/587071 | 16:12 |
| mriedem | have you tried restarting the compute-2 nova-compute service to see if that flushes the cpu_allocation_ratio 1.0 change to placement? | 16:13 |
| stephenfin | mriedem: If I'm reading that right, if we send that once, we'll never be able to update it, right? | 16:13 |
| mriedem | stephenfin: we should be able to update it any time we change config and restart nova-compute | 16:13 |
| stephenfin | Because placement will always return 'allocation_ratio' once it's been set https://github.com/openstack/nova/blob/stable/rocky/nova/compute/resource_tracker.py#L105 | 16:13 |
| mriedem | the allocation ratio config is read and set on the compute node object here https://github.com/openstack/nova/blob/18.0.0/nova/compute/resource_tracker.py#L624 | 16:14 |
| mriedem | no, | 16:14 |
| *** tssurya has quit IRC | 16:14 | |
| mriedem | that cpu_inv is coming from the virt driver, | 16:14 |
| mriedem | and none of the virt driver's have an allocation_ratio in their inventory dicts | 16:14 |
| stephenfin | Correct. My mistake | 16:14 |
| mriedem | :param inv_data: Dict, keyed by resource class, of inventory information returned from virt driver's get_inventory() method | 16:14 |
| mriedem | so try restarting compute-2's nova-compute service | 16:15 |
| mriedem | if you changed the config but never restarted the service, the change wouldn't get to placement, | 16:15 |
| mriedem | if you did, but something is f'ed in the local provider tree cache, that should refresh the cache | 16:15 |
| mriedem | the provider tree cache stuff is all black magic to me | 16:15 |
| awaugama | let's give it a shot | 16:17 |
| awaugama | No luck. I'm still seeing 16 set for placement for compute node 2 | 16:19 |
| stephenfin | awaugama: Dumb question but this config option is configured on the compute node 2 and not the controller, right? | 16:20 |
| awaugama | yeah | 16:20 |
| *** k_mouza has joined #openstack-nova | 16:21 | |
| awaugama | [root@compute-2 ~]# cat /var/lib/config-data/puppet-generated/nova_libvirt/etc/nova/nova.conf | grep cpu_allocation | 16:21 |
| awaugama | #cpu_allocation_ratio=0.0 | 16:21 |
| awaugama | cpu_allocation_ratio=1 | 16:21 |
| stephenfin | Indeed. Cool, just checking | 16:21 |
| * stephenfin initially thought that had to be set on the controller before seeing we report it back via the ComputeNode object | 16:21 | |
| bauzas | wait, just catching the discussion | 16:23 |
| bauzas | awaugama: so you're saying that placement ratio != nova conf option ? | 16:24 |
| awaugama | yeah | 16:24 |
| *** munimeha1 has joined #openstack-nova | 16:24 | |
| *** knikolla has joined #openstack-nova | 16:24 | |
| *** vabada has joined #openstack-nova | 16:24 | |
| bauzas | awaugama: PM me the compute logs | 16:26 |
| bauzas | that's... weirdo | 16:26 |
| *** moshele has joined #openstack-nova | 16:26 | |
| bauzas | the only case where we bypass the ratios is when we have 0.0 | 16:26 |
| bauzas | but mriedem fixed it recently | 16:26 |
| mriedem | i mentioned https://github.com/openstack/nova/commit/01265b98c4cd2b1377e891a06ce748fc6f8f3425 but that's kind of a different issue | 16:27 |
| mriedem | awaugama: when nova-compute starts up, it will dump the config options at debug level | 16:27 |
| *** dtantsur is now known as dtantsur|afk | 16:27 | |
| mriedem | are you able to see cpu_allocation_ratio in the logs after you restarted compute-2? | 16:28 |
| bauzas | yup, but we also audit them IIRC | 16:28 |
| bauzas | lemme double check on the gate | 16:28 |
| awaugama | will check in a little bit, I have an appointment I can't miss that I have to leave for. will let you know | 16:28 |
| *** awaugama is now known as awaugama_appoint | 16:28 | |
| *** k_mouza has quit IRC | 16:33 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Fail to live migration if instance has a NUMA topology https://review.openstack.org/611088 | 16:35 |
| *** helenafm has quit IRC | 16:35 | |
| *** panda is now known as panda|off | 16:36 | |
| *** moshele has quit IRC | 16:38 | |
| *** erlon_ has joined #openstack-nova | 16:38 | |
| *** erlon has quit IRC | 16:40 | |
| melwitt | 16:45 | |
| mriedem | what's going on with the handling a down cell stuff? seems that is blocked or languishing or other? tssurya is already gone for the day though | 16:52 |
| dansmith | last I saw I think it was getting nailed by zuul failures | 16:52 |
| mriedem | it also looks like people are putting more specs in runways https://etherpad.openstack.org/p/nova-runways-stein | 16:53 |
| dansmith | I don't think I've seen it pop back up recently | 16:53 |
| mriedem | melwitt: you might want to send a reminder to the ML that runways aren't for spec reviews | 16:53 |
| mriedem | dansmith: was the full series even passing? | 16:53 |
| mriedem | like last i looked i thought the change with the microversion wasn't passing tests | 16:54 |
| dansmith | mriedem: I think there is one patch you're holding a -2 on that you can drop, as she moved the microversion stuff to the end and isn't doing the check-for-unmerged-microversion It hink | 16:54 |
| melwitt | mriedem: ack | 16:54 |
| mriedem | dansmith: yeah i can do that | 16:54 |
| dansmith | mriedem: yeah the top one looks sick, but the others are fallout from zuul fails I think | 16:54 |
| mriedem | since it's one of our "themes" i'm worried it's just sitting in a pile of poo | 16:54 |
| mriedem | https://review.openstack.org/#/c/609807/ | 16:55 |
| dansmith | oh some functional fails on one of them too | 16:55 |
| mriedem | was there agreement on how exceptions would be returned when a scatter/gather fails? | 16:55 |
| mriedem | https://review.openstack.org/#/c/607934/ | 16:55 |
| dansmith | yeah I think so | 16:56 |
| dansmith | lemme pull this down and see if I can fix up her functional failures | 16:56 |
| mriedem | so rather than the sentinel, we just return an instance of the thing that failed | 16:56 |
| dansmith | an instance of the exception you mean right? | 16:57 |
| mriedem | right | 16:59 |
| *** derekh has quit IRC | 17:00 | |
| dansmith | that's what we agreed yeah | 17:00 |
| *** mvkr has quit IRC | 17:01 | |
| *** Luzi has quit IRC | 17:02 | |
| *** erlon__ has joined #openstack-nova | 17:04 | |
| *** erlon_ has quit IRC | 17:07 | |
| *** hamzy has quit IRC | 17:07 | |
| *** hamzy has joined #openstack-nova | 17:08 | |
| mriedem | ok -1 on https://review.openstack.org/#/c/567785/ which is the api plumbing patch, | 17:10 |
| mriedem | mostly b/c of tests and lack of docs and not explaining it's plumbing | 17:11 |
| mriedem | but i also think it might be doing the queries wrong | 17:11 |
| mriedem | but i've dropped the -2 | 17:11 |
| *** hamzy has quit IRC | 17:12 | |
| mriedem | i'd have to look at the spec again, but it's querying instances from down cells for the user's request context project_id, | 17:12 |
| *** hamzy has joined #openstack-nova | 17:12 | |
| mriedem | which if it's an admin listing instances for all_tenants, or a specific project_id, would be wrong | 17:12 |
| dansmith | it changes that behavior? | 17:13 |
| dansmith | that shoudl only be enabled if it's enabled in config and not doing all tenants | 17:13 |
| dansmith | unless I'm misunderstanding what you mean | 17:13 |
| macza | hello, ive run into an issue on ocata using nova with ceph, a compute hypervisor crashed and now all the instances wont connect to their drives after bringing the hypervisor back up | 17:14 |
| macza | hoping i could get some help | 17:16 |
| *** spatel has joined #openstack-nova | 17:19 | |
| spatel | I am seeing very strange issue | 17:19 |
| spatel | In /etc/default/grub hugepagesz=2M hugepages=12288 | 17:19 |
| spatel | 1 instance running on it with 8G | 17:20 |
| *** adrianc has quit IRC | 17:20 | |
| spatel | trying to spin up one more instance but getting error qemu-kvm: cannot set up guest memory 'pc.ram': Cannot allocate memory | 17:20 |
| spatel | I do have 32G memory on compute node | 17:20 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova stable/pike: Handle volume API failure in _post_live_migration https://review.openstack.org/611093 | 17:20 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova stable/pike: Handle volume API failure in _post_live_migration https://review.openstack.org/611093 | 17:23 |
| *** k_mouza has joined #openstack-nova | 17:25 | |
| *** Swami has joined #openstack-nova | 17:31 | |
| spatel | nevermind, my bad i forgot to add hugepage properties to flavor | 17:33 |
| cfriesen | spatel: that'll do it. :) | 17:34 |
| mriedem | dansmith: i mean this https://review.openstack.org/#/c/567785/20/nova/compute/api.py@2295 | 17:39 |
| mriedem | while listing instances, if there are down cells, it lists instances via the mappings from the down cells but filters on the user's request context | 17:39 |
| dansmith | oh for the minimal fill ins | 17:39 |
| mriedem | and if i'm an admin listing instances for another project via the project_id filter, that woudl be wrong | 17:39 |
| mriedem | right | 17:39 |
| dansmith | yeah I guess | 17:39 |
| dansmith | I was thinking we talked about that and said "kinda meh" but I might be misremembering | 17:40 |
| mriedem | https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/handling-down-cell_new.html#edge-cases | 17:40 |
| mriedem | yeah i remember we talked about it too, glad it's in the spec | 17:40 |
| *** panda|off has quit IRC | 17:40 | |
| dansmith | um, is it? | 17:41 |
| dansmith | the --deleted thing is mentioned there | 17:41 |
| dansmith | --all-tenants is a little different | 17:41 |
| *** awaugama_appoint is now known as awaugama | 17:41 | |
| mriedem | "Filters: If the user is listing servers using filters the results from the down cell will be skipped and no minimalistic construct will be provided since there is no way of validating the filtered results from the down cell if the value of the filter key itself is missing. " | 17:41 |
| mriedem | so like if i'm filtering on status or something | 17:41 |
| mriedem | we said in the spec we'd just ignore what's in down cells since we can't know | 17:42 |
| dansmith | right | 17:42 |
| dansmith | for filters | 17:42 |
| dansmith | but your point was --all-tenants right? | 17:42 |
| mriedem | that's one of them | 17:42 |
| mriedem | and not called out specifically | 17:42 |
| mriedem | for --all-tenants, | 17:42 |
| mriedem | i'd think we could just list all instance mappings in down cells, | 17:42 |
| *** panda has joined #openstack-nova | 17:42 | |
| mriedem | not filtered on project_id | 17:42 |
| mriedem | up to whatever limit | 17:43 |
| *** ralonsoh has quit IRC | 17:47 | |
| mriedem | so apparently filtering on project_id only works if you're also using all_tenants | 17:49 |
| mriedem | otherwise we just filter on the context.project_id | 17:49 |
| dansmith | um, what? | 17:50 |
| dansmith | meaning --all_tenants is required in order to say --but-only-this-one ? | 17:51 |
| mriedem | https://github.com/openstack/nova/blob/9c5d4eb2008df02837985779d87fedb17b4f70bf/nova/api/openstack/compute/servers.py#L206L248 | 17:51 |
| mriedem | yes... | 17:51 |
| *** macza has quit IRC | 17:52 | |
| *** dklyle has joined #openstack-nova | 17:52 | |
| dansmith | hmm | 17:52 |
| *** macza has joined #openstack-nova | 17:52 | |
| *** mgoddard has quit IRC | 18:11 | |
| *** k_mouza has quit IRC | 18:11 | |
| *** mgoddard has joined #openstack-nova | 18:17 | |
| *** k_mouza has joined #openstack-nova | 18:21 | |
| openstackgerrit | Jack Ding proposed openstack/nova master: Add I/O Semaphore to limit concurrent disk ops https://review.openstack.org/609180 | 18:24 |
| *** k_mouza has quit IRC | 18:26 | |
| *** cdent has joined #openstack-nova | 18:27 | |
| *** david-lyle has joined #openstack-nova | 18:29 | |
| *** dklyle has quit IRC | 18:29 | |
| awaugama | mriedem: when you have a chance, it looks like the value is set on the compute node. after a reboot I saw this in the nova-compute log | 18:32 |
| awaugama | 2018-10-17 17:44:00.277 1 DEBUG oslo_service.service [req-c9c4f04c-cfc6-4fe3-868d-206f9329419d - - - - -] cpu_allocation_ratio = 1.0 log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:3023 | 18:32 |
| mriedem | jroll: where in the ironic API reference would one find anything about conductor_groups? https://developer.openstack.org/api-ref/baremetal/ | 18:33 |
| mriedem | i mean i see https://docs.openstack.org/ironic/latest/contributor/webapi-version-history.html#rocky-11-1-0 | 18:33 |
| mriedem | but nothing in the API reference | 18:33 |
| jroll | mriedem: apparently I forgot to update that when I added it :( | 18:34 |
| mriedem | awaugama: ok so it's 1.0 in config, it's 1.0 in the compute_nodes.cpu_allocation_ratio column in the db, but it's 16.0 in the associated resource provider VCPU inventory in placement | 18:34 |
| awaugama | yeah | 18:34 |
| mriedem | well wtf | 18:34 |
| jroll | mriedem: it would be in node CRUD as its own field, I'll get that done real quick | 18:34 |
| mriedem | jroll: a uuid or what? | 18:34 |
| mriedem | jroll: and you can update a node's conductor_group? | 18:35 |
| jroll | mriedem: a string, up to 255 characters IIRC: https://github.com/openstack/ironic/blob/b8ffcc0f0298fca5b4b36ad016e2c3b2f0e81710/ironic/common/utils.py#L530 | 18:35 |
| mriedem | ok so it's just some tag | 18:35 |
| awaugama | I'll sit down with sylvain tomorrow and we can do some debugging, will let you know if we find anything | 18:35 |
| mriedem | special tag | 18:35 |
| jroll | yeah | 18:35 |
| jroll | alphanumeric, plus - _ . | 18:36 |
| *** moshele has joined #openstack-nova | 18:37 | |
| *** READ10 has quit IRC | 18:40 | |
| artom | - _ . is what I look like after a few drinks | 18:42 |
| jroll | hah | 18:42 |
| *** tbachman has quit IRC | 18:51 | |
| jroll | mriedem: api-ref for you https://review.openstack.org/611415 | 18:53 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova-specs master: Re-propose numa-aware-live-migration spec https://review.openstack.org/599587 | 18:53 |
| mriedem | jroll: i've brought the wrath | 18:58 |
| dansmith | hrm, pretty sure this functional timeout on the down cell series is a real deadlock on our cell cache | 18:58 |
| *** manjeets has quit IRC | 18:59 | |
| jroll | mriedem: thanks, valid points | 18:59 |
| jroll | for context I haven't touched our API ref in a long time :P | 18:59 |
| *** manjeets has joined #openstack-nova | 19:00 | |
| mriedem | ugh | 19:00 |
| mriedem | so GET /v1/nodes/detail is deprecated for GET /v1/nodes?detail=True, | 19:00 |
| mriedem | but the request filter and response parameters for the latter don't mention anything possible in the former | 19:01 |
| *** tbachman has joined #openstack-nova | 19:06 | |
| jroll | I'm not sure it's even properly deprecated | 19:08 |
| jroll | added a note to the detail=True parameter | 19:08 |
| *** tbachman has quit IRC | 19:10 | |
| *** moshele has quit IRC | 19:15 | |
| *** tbachman has joined #openstack-nova | 19:16 | |
| *** awaugama has quit IRC | 19:20 | |
| *** hamzy has quit IRC | 19:21 | |
| *** hamzy has joined #openstack-nova | 19:21 | |
| mriedem | so we're deprecating the force flag from the evacuate and live migration apis, | 19:29 |
| *** READ10 has joined #openstack-nova | 19:29 | |
| mriedem | wouldn't it behoove us to deprecate that as an option from nova commands as well? | 19:29 |
| mriedem | or at least doc it up real good that you shouldn't use it? | 19:29 |
| *** hamzy has quit IRC | 19:30 | |
| *** dklyle has joined #openstack-nova | 19:31 | |
| artom | mriedem, wait, deprecate or remove? | 19:32 |
| melwitt | dansmith: would appreciate your review on mah backport https://review.openstack.org/610673 | 19:33 |
| artom | Because for removal the usual novaclient microversion stuff applies, no? | 19:33 |
| *** david-lyle has quit IRC | 19:33 | |
| mriedem | artom: if we don't want people using the force flag to live migrate or evacuate an instance, | 19:34 |
| mriedem | so much so that we're deprecating the api parameter, | 19:34 |
| mriedem | you could argue that we should not have it in the CLI either | 19:34 |
| mriedem | like, at all | 19:34 |
| mriedem | even for older microversions | 19:34 |
| artom | mriedem, but it still exists for old microversions | 19:34 |
| artom | In the API | 19:34 |
| mriedem | yes i know | 19:34 |
| artom | So, we have to keep client support | 19:34 |
| mriedem | not really | 19:35 |
| artom | So if they specifically --os-compute-version <old>, they have it | 19:35 |
| artom | Otherwise, it's gone | 19:35 |
| mriedem | once all allocations are nested, you won't be able to force at all | 19:35 |
| mriedem | regardless of microversion | 19:35 |
| mriedem | anyway, it was just a thought | 19:35 |
| mriedem | should probably start by putting the big fat warnings in the API reference into the CLI option descriptions | 19:36 |
| *** tbachman has quit IRC | 19:36 | |
| artom | mriedem, ah I see. Well we still have to keep the old API intact, no? Just now we'll return a 400 or something. | 19:36 |
| artom | If they send a force flag | 19:37 |
| *** dklyle has quit IRC | 19:37 | |
| mriedem | it'll be some kind of error | 19:37 |
| *** dklyle has joined #openstack-nova | 19:38 | |
| mriedem | don't know if it's a 400 or 409 | 19:38 |
| mriedem | it's in gibi's spec | 19:38 |
| mriedem | aspiers: just a few hundred comments in your spec https://review.openstack.org/#/c/609779/ | 19:40 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix typo in libvirt.hw_machine_type help https://review.openstack.org/611422 | 19:43 |
| *** macza has quit IRC | 19:43 | |
| *** macza has joined #openstack-nova | 19:44 | |
| openstackgerrit | Jack Ding proposed openstack/nova-specs master: High Precision Event Timer (HPET) on x86 guests https://review.openstack.org/607989 | 19:48 |
| *** moshele has joined #openstack-nova | 19:49 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Document each libvirt.sysinfo_serial choice https://review.openstack.org/611426 | 19:51 |
| *** dklyle has quit IRC | 19:52 | |
| *** spatel has quit IRC | 19:53 | |
| artom | mriedem, you led me astray, I demand a refund: https://review.openstack.org/#/c/587439/4/nova/tests/unit/compute/test_compute_mgr.py@7439 | 19:55 |
| *** dklyle has joined #openstack-nova | 19:55 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: Ensure attachment cleanup on failure in driver.pre_live_migration https://review.openstack.org/587439 | 19:59 |
| mriedem | umm | 20:00 |
| mriedem | what's the point of even asserting those then | 20:00 |
| artom | I guess Matt was trying to make sure we call all of the things? But... in a for loop, because he didn't feel like writing out each individual method name? | 20:01 |
| mriedem | ^O^ | 20:01 |
| mriedem | that's me shrugging, not a bat | 20:01 |
| artom | Or a yelling Asian person? | 20:02 |
| artom | (Can I say that? Is that racist?) | 20:02 |
| *** moshele has quit IRC | 20:02 | |
| mriedem | it's very racist | 20:02 |
| artom | Dammit. Hilter 2.0 right here, friends. | 20:03 |
| openstackgerrit | Merged openstack/nova master: Merge used_limits extension response into limit view builder https://review.openstack.org/606031 | 20:05 |
| *** erlon__ has quit IRC | 20:05 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix block_device_mapping_v2 mention in server create API reference https://review.openstack.org/611433 | 20:15 |
| *** dklyle has quit IRC | 20:15 | |
| *** hamzy has joined #openstack-nova | 20:17 | |
| *** tbachman has joined #openstack-nova | 20:20 | |
| *** mchlumsky has quit IRC | 20:22 | |
| *** READ10 has quit IRC | 20:34 | |
| openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Recommend against using --force for evacuate/live migration https://review.openstack.org/611436 | 20:34 |
| *** dklyle has joined #openstack-nova | 20:46 | |
| *** eharney has quit IRC | 20:55 | |
| *** dklyle has quit IRC | 20:59 | |
| *** dklyle has joined #openstack-nova | 20:59 | |
| *** dklyle has quit IRC | 21:10 | |
| *** dklyle has joined #openstack-nova | 21:10 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add nova-status upgrade check for consoles https://review.openstack.org/611214 | 21:15 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add rolling upgrade info to enable_consoleauth workaround reno https://review.openstack.org/611218 | 21:15 |
| *** dklyle has quit IRC | 21:17 | |
| *** rcernin has joined #openstack-nova | 21:18 | |
| *** dklyle has joined #openstack-nova | 21:18 | |
| *** slaweq has quit IRC | 21:25 | |
| *** dklyle has quit IRC | 21:40 | |
| *** cdent has quit IRC | 21:44 | |
| *** dklyle has joined #openstack-nova | 21:44 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Support initial allocation ratios https://review.openstack.org/552105 | 21:48 |
| *** priteau has quit IRC | 21:50 | |
| mriedem | holy fart i'm +2 on the initial allocation ratios spec https://review.openstack.org/#/c/552105/ | 21:51 |
| *** takashin has joined #openstack-nova | 21:51 | |
| *** dklyle has quit IRC | 21:52 | |
| melwitt | and the peasants rejoiced! | 21:52 |
| *** dklyle has joined #openstack-nova | 21:57 | |
| *** mriedem has quit IRC | 22:00 | |
| *** bnemec has quit IRC | 22:01 | |
| *** mvkr has joined #openstack-nova | 22:04 | |
| *** mlavalle has quit IRC | 22:14 | |
| *** munimeha1 has quit IRC | 22:20 | |
| *** itlinux has quit IRC | 22:22 | |
| *** dklyle has quit IRC | 22:36 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Add API ref guideline for examples https://review.openstack.org/604060 | 22:37 |
| *** READ10 has joined #openstack-nova | 23:04 | |
| *** macza has quit IRC | 23:17 | |
| openstackgerrit | Merged openstack/nova stable/rocky: Handle volume API failure in _post_live_migration https://review.openstack.org/611083 | 23:19 |
| openstackgerrit | Merged openstack/nova master: Ignore uuid if already set in ComputeNode.update_from_virt_driver https://review.openstack.org/611162 | 23:19 |
| *** itlinux has joined #openstack-nova | 23:22 | |
| *** lbragstad is now known as lbragstad_503 | 23:26 | |
| *** erlon__ has joined #openstack-nova | 23:33 | |
| *** alex_xu has joined #openstack-nova | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!