Wednesday, 2018-07-11

*** shaohe_feng has quit IRC00:01
*** shaohe_feng has joined #openstack-nova00:02
*** moshele has joined #openstack-nova00:04
*** shaohe_feng has quit IRC00:12
*** shaohe_feng has joined #openstack-nova00:12
*** phuongnh has joined #openstack-nova00:12
*** shaohe_feng has quit IRC00:22
openstackgerritTakashi NATSUME proposed openstack/nova master: Transform metrics.update notification  https://review.openstack.org/58056700:23
openstackgerritMerged openstack/os-vif master: Remove unnecessary pyNN testenv sections  https://review.openstack.org/57768600:24
*** shaohe_feng has joined #openstack-nova00:24
openstackgerritMerged openstack/os-vif master: Remove [tox:jenkins] section from tox.ini  https://review.openstack.org/58088600:30
*** shaohe_feng has quit IRC00:32
*** shaohe_feng has joined #openstack-nova00:33
*** vivsoni has joined #openstack-nova00:37
*** yamamoto has joined #openstack-nova00:40
*** vivsoni_ has quit IRC00:40
*** shaohe_feng has quit IRC00:42
*** shaohe_feng has joined #openstack-nova00:43
*** yamamoto has quit IRC00:45
*** phuongnh has quit IRC00:45
gmannmriedem: alex_xu yeah, those are being ignored as per additionalProporties=True00:48
gmannmriedem: alex_xu same legacy issue we had to keep about --all-tenant which i tired to fix but its more UX tradeoff and where we should fix in APi or on client side - https://review.openstack.org/#/c/198184/00:49
gmannmriedem: alex_xu how about doing additionalProperties=False with microversion in stein so that user can get clear 400 instead of confusing ignore things.00:50
gmannand we can include more API fixes in that as mriedem said. i can start a etherpad for that and then we choose appropriate issues to fix in single version bump00:50
gmannif you guys have not started that yet00:51
*** chyka has joined #openstack-nova00:51
*** shaohe_feng has quit IRC00:53
openstackgerritMerged openstack/nova master: update project/user for consumer in allocation  https://review.openstack.org/58113900:53
*** shaohe_feng has joined #openstack-nova00:54
*** moshele has quit IRC00:55
*** chyka has quit IRC00:56
*** hongbin has joined #openstack-nova00:59
mriedemgmann: no one has started an etherpad yet01:02
*** rnoriega has quit IRC01:02
gmannmriedem: ok.01:03
*** shaohe_feng has quit IRC01:03
*** shaohe_feng has joined #openstack-nova01:04
*** stakeda has joined #openstack-nova01:04
*** edmondsw has joined #openstack-nova01:05
Kevin_Zhengmriedem Hi, thanks for the review on my patch, I have one question and I left a comment on yikun's patch yesterday,01:05
mriedemok01:06
Kevin_Zhengon what scenario should I bump compute version to 5.1 and what scenario to 5.0?01:06
Kevin_Zhenglike the last 3 bump was still 5.0 but added 3 versions01:06
mriedemthose were for behavioral changes01:07
mriedemthe rpc api version itself changes if the rpc api changes01:07
mriedemlike adding/changing parameters in an rpc api method01:07
mriedemlike you did in live_migration_abort01:07
mriedeme.g. # Version 31: The compute manager checks if 'trusted_certs' are supported01:07
Kevin_ZhengOK, I got it, that also what I thought01:07
mriedemthere is a version check for minimum nova-compute >= 31 if trying to create a server with trusted_certs01:08
Kevin_Zhengok, so it checks for 31 instead of 5.001:08
*** yamahata has quit IRC01:08
mriedemyeah because 5.0 doesn't really tell us if trusted_certs is supported in the compute or not01:08
Kevin_Zhenggot it01:09
Kevin_ZhengI think I will reply other comments in the patch :)01:09
mriedemsimilar to yikun's late affinity check policy change01:09
*** edmondsw has quit IRC01:09
mriedemit's a behavior change in how we check late affinity on the compute01:09
mriedemnot an actual interface change in the rpc api01:09
*** BrinZhang has joined #openstack-nova01:13
*** shaohe_feng has quit IRC01:13
*** shaohe_feng has joined #openstack-nova01:14
*** rnoriega has joined #openstack-nova01:16
*** shaohe_feng has quit IRC01:23
*** luzC has quit IRC01:24
*** shaohe_feng has joined #openstack-nova01:25
*** luzC has joined #openstack-nova01:28
Kevin_Zhengmriedem replied on https://review.openstack.org/#/c/573136/ about why 2.24 sample files were added01:28
Kevin_ZhengI will try to be more clear about this, tring to remember all the details, has been a long time :)01:29
*** mrsoul has joined #openstack-nova01:30
*** mrsoul`` has joined #openstack-nova01:30
*** jdillaman has quit IRC01:33
*** mrsoul` has quit IRC01:33
*** mschuppert has quit IRC01:33
openstackgerritMatt Riedemann proposed openstack/nova master: Update some placement docs to reflect modern times  https://review.openstack.org/58115101:33
*** shaohe_feng has quit IRC01:34
openstackgerritMerged openstack/nova master: unquiesce instance after quiesce failure  https://review.openstack.org/55086501:34
*** shaohe_feng has joined #openstack-nova01:34
openstackgerritGhanshyam Mann proposed openstack/nova master: Use hard coded values in schema than reference  https://review.openstack.org/58128801:34
mriedemKevin_Zheng: i saw the inheritance but i still don't really understand it01:34
Kevin_Zhengmriedem, yeah, I'm trying to remember it but forgot some of the details.01:36
mriedemi see that we don't have any api samples for ServerMigrationsSampleJsonTestV2_2401:37
Kevin_Zhengyeah, because it dont have microversion01:37
mriedemi think it's the _read_template method01:38
mriedemKevin_Zheng: can you fix that separately?01:38
Kevin_Zhengit has api.microversion and read_template uses self.microversion01:38
mriedemfix the 2.24 api samples tests in a separate change before your new microversion01:38
mriedemhell push that up outside of the series now and i'll approve it01:39
Kevin_ZhengI don't know, because it was correct before my adding01:39
Kevin_ZhengOK01:39
Kevin_ZhengI still want to tell you the root cause though :)01:39
mriedemi see the root issue,01:39
Kevin_Zhengit is something in the setup01:39
mriedemyeah i see that01:39
mriedemServerMigrationsSamplesJsonTestV2_23 is doing it correctly01:40
Kevin_ZhengOK then I will make it a seperate patch01:40
Kevin_ZhengI have to rebase anyway01:40
mriedemthe 2.24 api sample test is posting the live migration and deleting it with 2.24 but using the 2.1 sample01:41
mriedembecause the actual request didn't change between then it works01:41
*** yamamoto has joined #openstack-nova01:42
Kevin_ZhengAh, I got it01:42
Kevin_Zhengif we inherent 2.2401:42
Kevin_Zhengit was setup first01:43
Kevin_Zhengand then again self.api.microversion was assigned to be 2.2501:43
Kevin_Zheng2.2401:43
Kevin_Zhengbecause self.api.microversion=2.24 is after super.setup()01:44
*** shaohe_feng has quit IRC01:44
*** tiendc has joined #openstack-nova01:44
*** shaohe_feng has joined #openstack-nova01:45
Kevin_Zhengand since it is wrpped in setUp, I can do nothing01:45
mriedemyup01:46
Kevin_Zhengyeah, I'm still young, I can remember things01:47
*** yamamoto has quit IRC01:47
mriedemwhat were we talking about again?01:48
Kevin_ZhengI cant remember01:48
*** yamamoto has joined #openstack-nova01:51
*** yikun has joined #openstack-nova01:52
*** shaohe_feng has quit IRC01:54
*** shaohe_feng has joined #openstack-nova01:55
*** jichen has joined #openstack-nova01:59
*** mriedem has quit IRC02:00
*** mriedem has joined #openstack-nova02:01
*** shaohe_feng has quit IRC02:04
*** shaohe_feng has joined #openstack-nova02:05
*** hshiina has joined #openstack-nova02:06
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Add policy to InstanceGroup object  https://review.openstack.org/56337502:07
*** yamamoto has quit IRC02:10
openstackgerritMatt Riedemann proposed openstack/nova master: Fix TypeError in prep_resize allocation cleanup  https://review.openstack.org/58154802:13
*** yamamoto has joined #openstack-nova02:15
*** shaohe_feng has quit IRC02:15
openstackgerritBrin Zhang proposed openstack/nova master: Add unshelve instance error info to fault table  https://review.openstack.org/57974702:15
*** tetsuro has joined #openstack-nova02:15
*** shaohe_feng has joined #openstack-nova02:15
jichenalex_xu: API meeting 2pm today ?02:17
yikunmriedem: hi, a question on policy notification, should we also do type convert in notification?02:18
openstackgerritMerged openstack/os-vif master: Fix docstrings to work with Sphinx 1.7.4  https://review.openstack.org/57702802:18
yikun^ https://review.openstack.org/#/c/563401/28/nova/tests/unit/compute/test_compute_utils.py@118102:18
yikunI mean should we change {"max_server_per_host": "3"} to {"max_server_per_host": 3} in notification02:19
mriedemyikun: i don't think it's necessary to duplicate that type conversion for the notifications02:19
yikunok, got it, I just thought should keep same format in obj and notification-obj.02:22
yikunand also if user create a group with {"max_server_per_host": 3}, but notification is {"max_server_per_host": "3"}02:23
yikunjust not sure it's okay for user or not.02:23
mriedemit doesn't matter too much to me either way, i just don't really like the duplicate code02:23
mriedemyou could ask gibi for his opinion when he's up02:24
*** shaohe_feng has quit IRC02:25
yikungibi: fortunately, gibi is online, could you give me some suggestion on it?02:25
openstackgerritLei Zhang proposed openstack/nova master: Add method to get cpu traits  https://review.openstack.org/56031702:25
mriedemit's about 4:30am for gibi02:26
mriedemso he's online but not relaly02:26
mriedem*really02:26
mriedemhe's in budapest02:26
*** shaohe_feng has joined #openstack-nova02:28
mriedemi on the other hand need to go to bed02:28
mriedemo/02:28
*** mriedem has quit IRC02:28
yikunha, have a good night.02:29
*** blkart has quit IRC02:31
*** armaan has quit IRC02:31
*** armaan has joined #openstack-nova02:32
*** blkart has joined #openstack-nova02:33
openstackgerritBrin Zhang proposed openstack/nova master: Add unshelve instance error info to fault table  https://review.openstack.org/57974702:33
*** bkopilov has quit IRC02:35
*** shaohe_feng has quit IRC02:35
*** shaohe_feng has joined #openstack-nova02:38
gmannjichen: yes, 2 PM for you.02:43
jichengmann: got it , thank you, I will arrange my time to join, in this channel, right?02:43
gmannjichen: if you are in china :) its 6 UTC though02:43
gmannjichen: yes, on nova channel only02:43
jichengmann: yes, I am in china  :)02:44
*** phuongnh has joined #openstack-nova02:44
*** shaohe_feng has quit IRC02:45
*** shaohe_feng has joined #openstack-nova02:47
*** edmondsw has joined #openstack-nova02:53
*** shaohe_feng has quit IRC02:56
openstackgerritMerged openstack/nova master: Remove unused variable in migration  https://review.openstack.org/58146402:56
*** shaohe_feng has joined #openstack-nova02:56
*** edmondsw has quit IRC02:57
openstackgerritZhenyu Zheng proposed openstack/nova master: Use ThreadPoolExecutor for max_concurrent_live_migrations  https://review.openstack.org/56350502:58
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Add policy field to ServerGroup notification object  https://review.openstack.org/56340103:00
dansmithyikun: if you get the .rules from the object, it will be type converted, so you don't need to convert again right?03:06
*** shaohe_feng has quit IRC03:06
*** shaohe_feng has joined #openstack-nova03:06
yikunI think no, because 'rules': fields.DictOfStringsField(nullable=True),  it will convert back to orignal by default.03:07
yikunhttps://review.openstack.org/#/c/563401/29/nova/notifications/objects/server_group.py@4603:07
gmanndansmith: yikun should we rename 'rules' to 'policy_rules' in REST API  - https://review.openstack.org/#/c/563401/28/nova/notifications/objects/server_group.py@4103:10
gmannotherwise it is confusing for users03:11
dansmithyikun: ah you're right I forgot the notification field was typed03:11
gmannin object and notification object 'rules' as it is ok.03:11
dansmithgmann: is it confusing? I guess we should do policy_name and policy_rules?03:12
dansmithpersonally, I'm okay with policy and rules but if you think it needs scoping, then okay03:12
gmanndansmith: yeah, it was ok under policy dict but if we make out of it then scoping with policy_* is much clear03:13
dansmithgmann: okay03:13
dansmithI've got to go now too, before I turn into a pumpkin03:13
gmann:) gn03:13
* dansmith leaves that reference hanging for gmann and yikun to google03:13
*** jichen_ has joined #openstack-nova03:15
gmannyikun: REST API change need spec update also (as it is nested dict in current version). i can quickly update that after my lunch if you busy with code things? and we can discuss in today API office hour also with alex_xu and feedback from matt and dansmith on gerrit .03:16
*** shaohe_feng has quit IRC03:16
*** jichen has quit IRC03:17
*** jichen_ is now known as jichen03:17
*** shaohe_feng has joined #openstack-nova03:18
yikungmann: sure, thanks for your help, and I will change api from nest to flat fist. and if we decided change policy_* I will append the change. :)03:19
*** shaohe_feng has quit IRC03:26
*** shaohe_feng has joined #openstack-nova03:28
*** shaohe_feng has quit IRC03:37
*** med_ has quit IRC03:38
*** shaohe_feng has joined #openstack-nova03:39
*** hongbin has quit IRC03:40
*** lpetrut_ has joined #openstack-nova03:45
*** shaohe_feng has quit IRC03:47
*** shaohe_feng has joined #openstack-nova03:47
*** sridharg has joined #openstack-nova03:48
*** udesale has joined #openstack-nova03:49
*** bkopilov has joined #openstack-nova03:55
*** david-lyle has joined #openstack-nova03:57
*** shaohe_feng has quit IRC03:57
*** dklyle has quit IRC03:58
*** mlavalle has quit IRC03:59
*** shaohe_feng has joined #openstack-nova03:59
*** dklyle has joined #openstack-nova04:00
*** david-lyle has quit IRC04:01
openstackgerritZhenyu Zheng proposed openstack/nova master: Compute: add support to abort queued live migration  https://review.openstack.org/56854204:05
*** shaohe_feng has quit IRC04:07
*** shaohe_feng has joined #openstack-nova04:09
*** jogo has joined #openstack-nova04:13
*** jogo has quit IRC04:13
*** jogo has joined #openstack-nova04:13
*** dklyle has quit IRC04:14
*** dklyle has joined #openstack-nova04:14
*** mdnadeem has joined #openstack-nova04:16
*** shaohe_feng has quit IRC04:18
openstackgerritZhenyu Zheng proposed openstack/nova master: Fix ServerMigrationSampleJsonTestsV2_24 to use its own sample file  https://review.openstack.org/58156204:18
*** chyka has joined #openstack-nova04:18
*** shaohe_feng has joined #openstack-nova04:18
*** david-lyle has joined #openstack-nova04:20
*** dklyle has quit IRC04:21
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Add policy field to ServerGroup notification object  https://review.openstack.org/56340104:22
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Change the ServerGroupAntiAffinityFilter to adapt to new policy  https://review.openstack.org/57116604:22
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Adapt _validate_instance_group_policy to new policy model  https://review.openstack.org/57146504:22
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Microversion 2.64 - Use new format policy in server group  https://review.openstack.org/56753404:22
*** chyka has quit IRC04:22
*** david-lyle has quit IRC04:23
*** dklyle has joined #openstack-nova04:23
*** phuongnh has quit IRC04:25
*** phuongnh has joined #openstack-nova04:26
*** lpetrut_ has quit IRC04:27
*** shaohe_feng has quit IRC04:28
gmannyikun: sounds good. we can get the consensus on patch first and then update spec for process.04:29
*** shaohe_feng has joined #openstack-nova04:31
*** phuongnh has quit IRC04:37
*** phuongnh has joined #openstack-nova04:38
*** shaohe_feng has quit IRC04:38
*** shaohe_feng has joined #openstack-nova04:40
*** shaohe_feng has quit IRC04:48
*** shaohe_feng has joined #openstack-nova04:49
*** abhishekk has joined #openstack-nova04:52
*** Bhujay has joined #openstack-nova04:53
*** flwang1 has quit IRC04:55
*** shaohe_feng has quit IRC04:59
*** Eran_Kuris has quit IRC05:00
*** shaohe_feng has joined #openstack-nova05:00
*** sahid has joined #openstack-nova05:01
*** links has joined #openstack-nova05:03
*** deepak_mourya has joined #openstack-nova05:05
*** shaohe_feng has quit IRC05:09
*** shaohe_feng has joined #openstack-nova05:09
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Add policy field to ServerGroup notification object  https://review.openstack.org/56340105:11
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Change the ServerGroupAntiAffinityFilter to adapt to new policy  https://review.openstack.org/57116605:11
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Adapt _validate_instance_group_policy to new policy model  https://review.openstack.org/57146505:11
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Microversion 2.64 - Use new format policy in server group  https://review.openstack.org/56753405:11
*** hamzy has quit IRC05:17
*** ratailor has joined #openstack-nova05:18
*** shaohe_feng has quit IRC05:19
*** shaohe_feng has joined #openstack-nova05:19
*** rodolof has quit IRC05:22
*** moshele has joined #openstack-nova05:23
*** myrat has joined #openstack-nova05:26
openstackgerritMerged openstack/nova master: Update some placement docs to reflect modern times  https://review.openstack.org/58115105:28
*** shaohe_feng has quit IRC05:29
*** takashin has left #openstack-nova05:30
*** shaohe_feng has joined #openstack-nova05:30
*** janki has joined #openstack-nova05:37
*** shaohe_feng has quit IRC05:40
*** shaohe_feng has joined #openstack-nova05:40
*** hamzy has joined #openstack-nova05:40
*** moshele has quit IRC05:41
*** shaohe_feng has quit IRC05:50
*** stelucz has joined #openstack-nova05:51
steluczHi, I just found in our cluster one nova aggregate with one host. It was probably created by temptest/benchmark in the past. Is it safe to delete this aggregate? or it will have some impact on running VMs at host?05:52
*** shaohe_feng has joined #openstack-nova05:53
*** brault has quit IRC05:57
gmannAPI office hour time05:59
gmann#startmeeting nova api06:00
openstackMeeting started Wed Jul 11 06:00:01 2018 UTC and is due to finish in 60 minutes.  The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot.06:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.06:00
*** openstack changes topic to " (Meeting topic: nova api)"06:00
openstackThe meeting name has been set to 'nova_api'06:00
gmannPING List: gmann, alex_xu06:00
gmannwho all here today06:00
alex_xuo/06:00
gmannalex_xu: hi06:00
alex_xugmann: good afternoon06:00
*** shaohe_feng has quit IRC06:00
*** ianw is now known as ianw_pto06:00
gmanngood afternoon06:00
jicheno/06:01
Kevin_Zhengo/06:01
gmannjichen: Kevin_Zheng : hi06:01
*** shaohe_feng has joined #openstack-nova06:01
gmannlet's start06:01
gmann#link https://wiki.openstack.org/wiki/Meetings/NovaAPI#Agenda_for_next_Office_hours06:01
gmann^^ generic agenda for this office hour06:02
yikuno/06:02
gmannyikun: hi06:02
gmann#topic Priorities06:02
*** openstack changes topic to "Priorities (Meeting topic: nova api)"06:02
gmann#link https://etherpad.openstack.org/p/rocky-nova-priorities-tracking06:02
gmannwe will go through the API section in above etherpad06:02
jichenok06:03
gmannline 5706:03
gmann1. Servers Ips non-unique network names06:03
gmannspec update is merged last week. #link https://review.openstack.org/#/c/558125/06:03
*** jichen_ has joined #openstack-nova06:05
gmannMaciej Kucia started code also but month  back06:05
jichen_Is there code patch ? I only saw a doc patch06:06
gmanni do not irc nick and changes should be straight forwards. if no updated since week, may be i can update the code for this BP06:06
gmannyea, that patch only. in same patch code needs to be updated.06:06
gmanni will check with Maciej  about progress.06:06
gmann2.Abort live migration in queued state:06:07
jichen_ok, got it06:07
gmannthis is from Kevin_Zheng and  i think good to review from us. i have added this in my today list.06:07
*** jichen has quit IRC06:07
Kevin_ZhengI got a question about this one06:08
gmannKevin_Zheng: anything you want to discuss on this then review06:08
Kevin_ZhengAPI related06:08
gmannyeah go head06:08
*** jichen has joined #openstack-nova06:08
Kevin_Zhengin the patch, I have a rpc version check06:08
Kevin_Zhenglike if the compute can do 5.1 then it can abort migration in queued status06:09
Kevin_Zhengif cannot do 5.106:09
Kevin_Zhengthen I trough an exception06:09
*** jichen__ has joined #openstack-nova06:09
Kevin_Zhengmy question is, should I mention that compute is not new enoguh?06:09
Kevin_Zhengenough06:09
gmannyou mean this -06:10
gmann#link https://review.openstack.org/#/c/56854206:10
Kevin_Zhengwill that be too much admin details for normal user?06:10
*** Eran_Kuris has joined #openstack-nova06:10
*** shaohe_feng has quit IRC06:10
Kevin_Zhengyeah06:10
*** jichen___ has joined #openstack-nova06:11
Kevin_Zhengif it is like what I did in the current patch06:11
*** shaohe_feng has joined #openstack-nova06:11
*** jichen___ has quit IRC06:11
Kevin_Zhengit seems a little wierd, as an user I requested 2.64 and then nova replied to me that migration is not in correct status06:11
alex_xuI think we have some example in the before06:11
*** jichen___ has joined #openstack-nova06:12
*** jichen_ has quit IRC06:12
Kevin_Zhengseems not correct with the mircorversion06:12
gmannyeah i am searching for that06:12
Kevin_ZhengYeah, we have some, yikun is also working on one with this kind of server version check06:12
*** jichen has quit IRC06:12
Kevin_Zhengjust wondering if it is good to let normal user know that some of the node is not new enough06:13
*** jichen has joined #openstack-nova06:13
yikun^ https://review.openstack.org/#/c/567534/30/nova/api/openstack/compute/server_groups.py@19006:13
gmannthis one -06:13
gmann#link https://github.com/openstack/nova/blob/d3fa585f5e2d11a91ee62744a0cf0c78921a47e4/nova/compute/api.py#L103306:13
*** myrat has quit IRC06:14
gmannyou can add new exception like this  - #link https://github.com/openstack/nova/blob/c8b93fa2493dce82ef4c0b1e7a503ba9b81c2e86/nova/exception.py#L229406:14
*** jichen__ has quit IRC06:14
*** myrat has joined #openstack-nova06:14
*** jchhatbar has joined #openstack-nova06:14
Kevin_Zhenghmm that seems better06:14
Kevin_ZhengYikun, maybe you should do something similar?06:15
yikunyeah, so we should NOT expose something like "all compute services needs upgrade to Rocky"06:15
gmannKevin_Zheng:  we will tell node is old we will just say this feature not supported so that user will contact admin for details06:15
Kevin_Zhengyeah06:15
alex_xualso this one #link https://github.com/openstack/nova/blob/stable/ocata/nova/compute/rpcapi.py#L111406:15
Kevin_Zhengmaybe the version thing should be appare in the log?06:15
gmannyeah crash dump one alaos06:15
*** myrat has quit IRC06:15
gmannKevin_Zheng: i think no.06:16
*** myrat has joined #openstack-nova06:16
*** jichen___ has quit IRC06:16
Kevin_Zhengwhy not, seems better for operators06:17
jichenyes, logs are only viewable to operators...06:17
alex_xuwhy we didn't check service version instead of rpc version?06:17
*** janki has quit IRC06:18
Kevin_Zhengthey are different cases06:18
gmannfor which case06:18
Kevin_Zhengyikun's case we check service version06:18
*** lpetrut has joined #openstack-nova06:18
Kevin_Zhengmy case check for rpc version06:18
Kevin_Zhengfor log, I mean both version06:18
Kevin_Zhengboth case06:18
gmannbut operator knows features support from reno and microversions info.06:19
alex_xuKevin_Zheng: I mean your case can be done by the service version check also?06:19
*** mrsoul is now known as mschuppert06:20
*** shaohe_feng has quit IRC06:21
Kevin_Zhengyeah, but this way is similar, I also did some negotiation in rpc06:21
gmannalex_xu: you mean in compute.api.py only like thrust certificate06:21
alex_xugmann: yea06:22
*** shaohe_feng has joined #openstack-nova06:22
gmannok, that seems better for me too06:22
alex_xuwe can stop the API early, and it is more clear in the code06:22
Kevin_ZhengI don't think so06:22
*** andreas_s has joined #openstack-nova06:22
Kevin_Zhengit is not the same with the certificate one06:23
Kevin_Zhengmy patch is depend on the migration status06:23
Kevin_Zhengand it changes06:23
Kevin_Zhengmaybe when we check at api, it is not running06:23
Kevin_Zhengbut if we check again at rpc, which is the last step before we go to compute06:24
Kevin_Zhengit changed to running06:24
Kevin_Zhengand the API call will be success06:24
Kevin_Zhengit is not like certificate that if it is not supported, it will be always not supported06:25
alex_xuthat is very short window...06:25
alex_xuI just feel the rpc version check is very low level. we should have consistent check by the service version in the future.06:26
Kevin_Zhengalso06:26
Kevin_ZhengDan have a comment about this in06:27
*** med__ has joined #openstack-nova06:27
*** hamzy has quit IRC06:27
*** hamzy_ has joined #openstack-nova06:27
Kevin_Zhenghttps://review.openstack.org/#/c/568542/15/nova/compute/rpcapi.py06:27
*** medberry has quit IRC06:27
alex_xuah, sounds like similar question, I will read it06:28
alex_xuI think we can move on, continue this discussion on the patch06:28
Kevin_ZhengNP06:29
gmannyeah, let's have discussion on patch and what dansmith is saying.06:29
gmannlet's move next06:29
gmann3. Complex anti-affinity policies:06:29
gmann#link https://review.openstack.org/#/q/topic:bp/complex-anti-affinity-policies+(status:open+OR+status:merged)06:29
gmannthis if yikun is working  and seems good amount of review06:30
gmannduring discussion on falt vs dict representation of policy, REST API response is changing here than what was approved ins epc06:31
gmannwhich seem better way06:31
*** shaohe_feng has quit IRC06:31
*** brault has joined #openstack-nova06:31
gmann#link https://review.openstack.org/#/c/567534/06:31
gmannmy concern is to make the same for reqeust also06:31
*** shaohe_feng has joined #openstack-nova06:31
yikunyeah, and some question we have:06:32
gmannyikun: any objection on that ^^ and alex_xu what you say about new format of 'policy' in request and response06:32
yikun1. should we change api response and req to flat policy?06:32
yikunhttps://review.openstack.org/#/c/546925/18/specs/rocky/approved/complex-anti-affinity-policies.rst@13006:32
yikun2. if we change these to flat, should we change to policy_*06:33
gmanni am fine for changing in both.06:33
gmann#link https://review.openstack.org/#/c/563401/28/nova/notifications/objects/server_group.py@4106:34
gmann^^ this is original discussion06:34
gmannfor me flat representation in both request and response with policy_name as string, policy_rules as dict seems good idea06:35
yikunflat vs nest example in here: http://paste.openstack.org/show/725526/06:35
alex_xuI need to read those discussion06:35
gmannsure,06:35
gmannplan is once we get agreement on thsoe, then yikun can update the spec also to reflect those changes06:36
*** alexchadin has joined #openstack-nova06:36
gmannyikun: anything else you want to discuss or we cna move next06:36
alex_xuyikun: for "rules": {"max_server_per_host": 3}, do you mean "rules": "{"max_server_per_host": 3}"06:37
alex_xuthe value of rules is a string?06:37
alex_xuyikun: sorry, I missed understand that, just ignore me06:37
yikundon't worry, you could take a look on example http://paste.openstack.org/show/725526/06:38
*** blkart has quit IRC06:38
yikungmann: only these 2 questions for me. :)06:39
*** ispp has joined #openstack-nova06:39
gmannflat2 should have policy_rules than rules_name06:39
gmannlike you doing in gerrit06:39
*** udesale has quit IRC06:40
yikunha,yeah, typo06:40
gmannok06:40
*** cfriesen has quit IRC06:41
*** shaohe_feng has quit IRC06:41
gmannalex_xu: you want to reply on patch. and we move next ?06:41
yikunhttp://paste.openstack.org/show/725528/06:41
yikunnew one06:41
alex_xugmann: yea, I will try to review the patch first06:41
gmannyikun: thanks06:42
yikunalex_xu: gmann ok, thanks06:42
gmannalex_xu: ok06:42
*** shaohe_feng has joined #openstack-nova06:42
gmannonly things with new flat format is we need to do validation in code for policy_rules only requested for  "anti-affinity" otherwise 40006:42
gmannbut that should be ok.06:42
*** udesale has joined #openstack-nova06:42
gmannlet's move next06:42
gmann4. Volume multiattach enhancements:06:43
gmannmatt mentoined in ML that he will be working on this.06:43
gmannonce it is ready we can add in our review list06:43
*** jichen_ has joined #openstack-nova06:44
gmanni forgot to mention that yikun BP in in nova runway this week and good to get this in06:44
gmannKevin_Zheng BP is in for next week so good to review it early and make it in good shape06:44
gmann5. API Extensions merge work06:45
gmann#link https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/api-extensions-merge-rocky06:45
openstackgerritOpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata  https://review.openstack.org/57801906:45
gmanni do not have progress in this BP during this week.06:45
*** jichen has quit IRC06:45
gmanni will update current patch and push more during end of week06:46
*** jichen_ is now known as jichen06:46
gmannlast one matt mentioned in ML is handling cell down06:46
gmann#link https://review.openstack.org/#/c/557369/06:46
gmanni have note read the spec yet but if any of you want to give more feedback there06:47
gmannnote->not06:47
gmannanything else on priority item otherwise we move to bug triage things06:48
gmannlet' move then06:49
gmann#topic Bug Triage/Discussion06:49
*** openstack changes topic to "Bug Triage/Discussion (Meeting topic: nova api)"06:49
gmann#link https://etherpad.openstack.org/p/nova-api-weekly-bug-report06:49
gmanni updated the weekly bug report06:49
gmannthere is not much change in that except i triaged 1 bug.06:49
*** moshele has joined #openstack-nova06:49
*** armaan has quit IRC06:49
*** armaan has joined #openstack-nova06:50
gmannwe discussed to keep reviewing 2-3 in-progress bugs patches every week.06:50
gmannwhich i did not do. :(06:50
alex_xuhah06:50
gmannalex_xu: how about you ?06:50
alex_xuI only review one, and not finish yet06:50
alex_xu#link https://review.openstack.org/#/c/486850/606:50
alex_xuthinking of we can make this move on06:50
gmannohk.06:51
*** blkart has joined #openstack-nova06:51
alex_xuKevin_Zheng: it is your patch, I'm still thinking we should use environ variable to build a correct url instead of strip the url06:51
* gmann forget about this 06:51
alex_xustrip the url is too hacky06:51
*** shaohe_feng has quit IRC06:51
Kevin_Zhengalready forgot details about that one :)06:52
*** shaohe_feng has joined #openstack-nova06:52
alex_xuhah06:52
alex_xuuse the bug report example. the req.environ['PATH_INFO'] can get '/v2.0', and the req.environ['SCRIPT_NAME'] can get '/compute'.06:53
* gmann our home work of cleaning up the in-progress patches is good idea to keep reminding the old work :)06:53
alex_xuwe should be able to figure which part of url is base url06:55
*** blkart has quit IRC06:56
alex_xuKevin_Zheng: it is ok we can continue the discussion in the patch, since you need remember the detail06:56
Kevin_Zhengyeah, I have to read it again06:57
gmannyea, i think we can recall on patch and continue06:57
*** moshele has quit IRC06:57
gmanni will do more in-progress review before next office hour06:57
gmann3 min left06:58
gmannanything on bug discussion?06:58
gmannlet's move to open in case of anything else06:59
gmann#topic Open Discussion06:59
*** openstack changes topic to "Open Discussion (Meeting topic: nova api)"06:59
*** Bhujay has quit IRC06:59
gmannany more things to bring up  ?06:59
*** hshiina has quit IRC06:59
Kevin_Zhengnope06:59
gmannok let's close the meeting then .07:00
gmannthanks everyone for joining. good to see more people in API discussion :)07:00
gmann#endmeeting07:00
*** openstack changes topic to "Current runways: abort-live-migration-in-queued-status / add-zvm-driver-rocky / glance-image-traits -- This channel is for Nova development. For support of Nova deployments, please use #openstack."07:00
openstackMeeting ended Wed Jul 11 07:00:40 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)07:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/nova_api/2018/nova_api.2018-07-11-06.00.html07:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/nova_api/2018/nova_api.2018-07-11-06.00.txt07:00
openstackLog:            http://eavesdrop.openstack.org/meetings/nova_api/2018/nova_api.2018-07-11-06.00.log.html07:00
*** shaohe_feng has quit IRC07:02
alex_xugmann: yea, good see a lot of people again :) thanks for leading the meeting07:02
*** shaohe_feng has joined #openstack-nova07:02
*** ccamacho has joined #openstack-nova07:02
gmannalex_xu: np!07:03
*** Bhujay has joined #openstack-nova07:05
*** myrat has quit IRC07:08
*** Bhujay has quit IRC07:09
*** tssurya has joined #openstack-nova07:12
*** tesseract has joined #openstack-nova07:12
*** bhagyashri_s has quit IRC07:12
*** shaohe_feng has quit IRC07:12
*** shaohe_feng has joined #openstack-nova07:13
*** Bhujay has joined #openstack-nova07:13
*** blkart has joined #openstack-nova07:13
*** jmlowe has quit IRC07:17
*** peereb has joined #openstack-nova07:17
*** ispp has quit IRC07:21
*** alexchadin has quit IRC07:22
*** shaohe_feng has quit IRC07:22
*** shaohe_feng has joined #openstack-nova07:23
*** ispp has joined #openstack-nova07:25
gmannalex_xu: scheduler would not come into pic in case of single host right ? i mean anti-affinity will always pass when only single host.07:29
*** alexchadin has joined #openstack-nova07:31
gmanni mean schd will return same single host always07:31
*** shaohe_feng has quit IRC07:32
*** shaohe_feng has joined #openstack-nova07:33
*** rodolof has joined #openstack-nova07:34
*** amoralej|off is now known as amoralej07:34
openstackgerrithuanhongda proposed openstack/nova-specs master: Add .idea folder to .gitignore  https://review.openstack.org/58161107:37
*** jichen_ has joined #openstack-nova07:40
*** myrat has joined #openstack-nova07:40
*** jchhatbar has quit IRC07:40
gmannalex_xu: currently i can boot 2 server on same host in case of anti-affinity policy on my single node env07:40
*** janki has joined #openstack-nova07:40
*** jichen has quit IRC07:41
alex_xugmann: that is bug?07:42
*** jichen_ is now known as jichen07:42
*** shaohe_feng has quit IRC07:43
gmannalex_xu: donot know, i feel that should fail like it does when i use different_host filter on my single node env07:43
*** shaohe_feng has joined #openstack-nova07:43
alex_xugmann: I also think so07:43
*** rodolof has quit IRC07:44
gmannbauzas: ^^07:46
*** mago_ has joined #openstack-nova07:48
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Microversion 2.64 - Use new format policy in server group  https://review.openstack.org/56753407:50
*** shaohe_feng has quit IRC07:53
*** chyka has joined #openstack-nova07:54
*** shaohe_feng has joined #openstack-nova07:54
*** flwang1 has joined #openstack-nova07:55
*** blkart has quit IRC07:56
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Address nit in afc7650e64753ab7687ae2c4f2714d4bb78a4e5a  https://review.openstack.org/58161607:58
*** chyka has quit IRC07:59
*** s10 has joined #openstack-nova07:59
*** ttsiouts has joined #openstack-nova08:01
*** rcernin has quit IRC08:03
*** shaohe_feng has quit IRC08:03
*** flwang1 has quit IRC08:03
*** rodolof has joined #openstack-nova08:04
*** bbock has joined #openstack-nova08:05
*** itlinux has joined #openstack-nova08:05
*** shaohe_feng has joined #openstack-nova08:05
*** markvoelker has quit IRC08:07
*** blkart has joined #openstack-nova08:09
*** Bhujay has quit IRC08:11
*** belmoreira has joined #openstack-nova08:12
*** shaohe_feng has quit IRC08:13
*** shaohe_feng has joined #openstack-nova08:14
*** armaan has quit IRC08:15
openstackgerritZhenyu Zheng proposed openstack/nova master: Compute: add support to abort queued live migration  https://review.openstack.org/56854208:16
*** derekh has joined #openstack-nova08:21
*** shaohe_feng has quit IRC08:24
*** shaohe_feng has joined #openstack-nova08:25
openstackgerritZhenyu Zheng proposed openstack/nova master: Fix ServerMigrationSampleJsonTestsV2_24 to use its own sample file  https://review.openstack.org/58156208:26
*** belmoreira has quit IRC08:26
*** bhagyashris has joined #openstack-nova08:29
*** ragiman has joined #openstack-nova08:31
*** belmoreira has joined #openstack-nova08:33
*** shaohe_feng has quit IRC08:34
*** shaohe_feng has joined #openstack-nova08:36
*** ivve has quit IRC08:37
*** itlinux has quit IRC08:37
*** Luzi has joined #openstack-nova08:39
*** bkopilov has quit IRC08:44
*** armaan has joined #openstack-nova08:44
*** shaohe_feng has quit IRC08:44
*** Luzi has quit IRC08:46
*** shaohe_feng has joined #openstack-nova08:46
*** flwang1 has joined #openstack-nova08:50
*** _pewp_ has quit IRC08:53
*** bbock has left #openstack-nova08:53
*** bbock has joined #openstack-nova08:53
*** _pewp_ has joined #openstack-nova08:53
*** shaohe_feng has quit IRC08:54
*** itlinux has joined #openstack-nova08:55
*** shaohe_feng has joined #openstack-nova08:56
*** alexchadin has quit IRC08:57
*** andreas_s has quit IRC08:59
openstackgerritYikun Jiang (Kero) proposed openstack/python-novaclient master: Microversion 2.64 - Use new format policy in server group  https://review.openstack.org/57826108:59
*** andreas_s has joined #openstack-nova08:59
*** andreas_s_ has joined #openstack-nova09:00
*** alexchadin has joined #openstack-nova09:00
*** andreas_s_ has quit IRC09:00
*** andreas_s_ has joined #openstack-nova09:01
*** myrat has quit IRC09:01
*** andreas_s_ has quit IRC09:01
*** s10 has quit IRC09:02
gibigmann: I think you see this https://bugs.launchpad.net/nova/+bug/174686309:03
openstackLaunchpad bug 1746863 in OpenStack Compute (nova) "scheduler affinity doesn't work with multiple cells" [High,In progress] - Assigned to melanie witt (melwitt)09:03
*** andreas_s has quit IRC09:04
*** shaohe_feng has quit IRC09:05
gmanngibi: but i have only single compute node no multi cell.09:05
*** shaohe_feng has joined #openstack-nova09:05
gibigmann: do you have superconductor?09:06
gibigmann: for example devstack by default starts with super conductor and cell1 conductor09:06
*** alexchadin has quit IRC09:08
*** mgoddard has joined #openstack-nova09:09
gmanngibi: yeah its superconductor09:10
*** alexchadin has joined #openstack-nova09:11
openstackgerritLee Yarwood proposed openstack/nova stable/ocata: libvirt: slow live-migration to ensure network is ready  https://review.openstack.org/55903409:12
openstackgerrithuanhongda proposed openstack/nova-specs master: Return "deleted" time of instance when vm_state is SOFT-DELETED  https://review.openstack.org/58163809:13
lyarwoodsahid: ^ can you take a look at that rebase, I can't seem to run tox against stable/ocata on F28 at the moment.09:13
gmanngibi: thanks.09:13
*** shaohe_feng has quit IRC09:15
*** shaohe_feng has joined #openstack-nova09:16
gibigmann: I have a tempest modification that reproduces the problem https://review.openstack.org/#/c/58134909:16
gibigmann: regardin that change I acutally wondering how can I test in tempest that booting in an affinity group goes to NoValid host due to a) the compute having a memeber from the group is full b) the compute is unavailable09:20
gibigmann: I think I cannot make a compute full or disable in tempest without breaking otuer parallel tempest cases09:20
gibis/otuer/other/09:21
gmanngibi: yeah, it is difficult to generate that scenario as we cannot disable compute service or full09:25
*** shaohe_feng has quit IRC09:25
gmanni was thinking with help of quota things but that would generate the scenario you want09:25
*** shaohe_feng has joined #openstack-nova09:26
*** Bhujay has joined #openstack-nova09:29
*** tetsuro has quit IRC09:29
*** andreas_s has joined #openstack-nova09:30
*** mgoddard has quit IRC09:30
gibigmann: but quota is not per compute so it wont remove a single compute from scheduling09:31
gmannyeah09:31
gibigmann: another way would be to create a flavor that is bigger than the remaining space on the compute the affinity group members are09:31
gmannbut how we would know how big flavor needs to be09:32
*** myrat has joined #openstack-nova09:32
gibigmann: are the multi node computes has fixed size or it depends on the node they are running?09:32
gmanngibi: it depends on node they are running.09:32
gmanngibi: and it can be any size on non-gate env09:33
gibigmann: than I would have to ask nova/placement about the available resources to create a big enough flavor09:33
gibigmann: ahh it is not good09:35
*** shaohe_feng has quit IRC09:35
gibigmann: I would need a flavor that requests more than available on the group's compute but still fits on the other compute (in multinode)09:36
*** shaohe_feng has joined #openstack-nova09:36
gmanngibi: another idea is to keep booting the server with same_host hint on anti-affinity compute till No-host09:37
gmannbut it can be very long scenario tests and need exact enabled_filter on setup09:37
gibigmann: and that could potentially interfere with test running in paralell09:38
*** issp has joined #openstack-nova09:38
gmannyeah. not good09:39
*** alexchadin has quit IRC09:39
*** ispp has quit IRC09:41
gibigmann: OK, I conclude that this cannot be covered in tempest. So we need to cover that in nova functional09:42
*** belmoreira has quit IRC09:43
*** andreas_s has quit IRC09:43
gmanngibi: yeah, functional tests is the way09:43
gmanngibi: but for this bug, your patch is ok right?  -https://bugs.launchpad.net/nova/+bug/177043409:43
openstackLaunchpad bug 1746863 in OpenStack Compute (nova) "duplicate for #1770434 scheduler affinity doesn't work with multiple cells" [High,In progress] - Assigned to melanie witt (melwitt)09:43
gmanngibi: i mean doing multiple request not single09:44
gibigmann: yeah, that seems doable. I will push a new PS soon fixing the TODO and mriedem's comments09:44
*** cdent has joined #openstack-nova09:44
gibigmann: it won't cover affinity + NoValid host case but that I will describe in the commit message09:45
gmanngibi: yeah, because at least covering that scenario is good09:45
gmannyeah09:45
gibigmann: thanks09:45
* gibi heading for lunch09:45
gmanngibi: i will check that tomorrow. added in my list09:45
gibigmann: thank you09:46
*** shaohe_feng has quit IRC09:46
gmanntime to go home, have good lunch09:46
gibihave a good afternoon09:46
*** shaohe_feng has joined #openstack-nova09:47
*** belmoreira has joined #openstack-nova09:48
*** mgoddard has joined #openstack-nova09:49
*** mago_ has quit IRC09:50
*** jmlowe has joined #openstack-nova09:50
*** andreas_s has joined #openstack-nova09:52
yikungibi: hi09:53
yikuncould you take a look on09:53
yikunhttps://review.openstack.org/#/c/563401/31/nova/tests/unit/compute/test_compute_utils.py@117909:53
yikunshould we change {"max_server_per_host": "3"} to {"max_server_per_host": 3} keep same use user passed?09:55
*** shaohe_feng has quit IRC09:56
yikunwe do conversion thing in group obj, see https://review.openstack.org/#/c/563375/39/nova/objects/instance_group.py@14009:56
*** belmoreira has quit IRC09:57
*** alexchadin has joined #openstack-nova09:57
*** shaohe_feng has joined #openstack-nova09:58
yikunI'm not sure should we keep same with it.10:00
*** alexchadin has quit IRC10:00
yikunand here is some message history:10:00
yikunhttp://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2018-07-11.log.html#t2018-07-11T02:18:5310:00
*** sambetts|afk is now known as sambetts10:00
yikun^ gibi10:00
*** andreas_s has quit IRC10:01
*** andreas_s has joined #openstack-nova10:01
*** ttsiouts has quit IRC10:04
*** andreas_s has quit IRC10:05
*** ttsiouts has joined #openstack-nova10:06
*** shaohe_feng has quit IRC10:06
*** shaohe_feng has joined #openstack-nova10:08
*** markvoelker has joined #openstack-nova10:08
*** spsurya_ has joined #openstack-nova10:08
*** andreas_s has joined #openstack-nova10:09
*** ttsiouts has quit IRC10:10
*** jmlowe has quit IRC10:11
*** damien_r has quit IRC10:12
*** andreas_s_ has joined #openstack-nova10:16
*** shaohe_feng has quit IRC10:16
*** moshele has joined #openstack-nova10:18
*** shaohe_feng has joined #openstack-nova10:18
*** andreas_s has quit IRC10:19
openstackgerritTakashi NATSUME proposed openstack/nova master: Transform metrics.update notification  https://review.openstack.org/58056710:19
*** andreas_s_ has quit IRC10:20
*** moshele has quit IRC10:21
*** alexchadin has joined #openstack-nova10:24
*** andreas_s has joined #openstack-nova10:26
*** shaohe_feng has quit IRC10:27
openstackgerritLee Yarwood proposed openstack/nova stable/ocata: libvirt: slow live-migration to ensure network is ready  https://review.openstack.org/55903410:28
*** shaohe_feng has joined #openstack-nova10:28
openstackgerrithuanhongda proposed openstack/nova master: Microversion 2.64 - Add "deleted" time in GET server response  https://review.openstack.org/57415910:28
*** bkopilov has joined #openstack-nova10:31
*** psachin has joined #openstack-nova10:35
*** andreas_s has quit IRC10:35
*** Guest88425 has joined #openstack-nova10:36
*** shaohe_feng has quit IRC10:37
*** shaohe_feng has joined #openstack-nova10:39
*** wwriverrat has joined #openstack-nova10:39
*** andreas_s has joined #openstack-nova10:40
*** mago_ has joined #openstack-nova10:40
*** markvoelker has quit IRC10:42
*** wwriverr_ has quit IRC10:43
openstackgerritVan Hung Pham proposed openstack/nova master: convert py35 jobs to py3  https://review.openstack.org/58166310:44
openstackgerritSurya Seetharaman proposed openstack/nova master: Update queued-for-delete from the ComputeAPI during deletion.  https://review.openstack.org/56681310:46
*** shaohe_feng has quit IRC10:47
openstackgerritSurya Seetharaman proposed openstack/nova master: Update queued-for-delete from the ComputeAPI during deletion/restoration  https://review.openstack.org/56681310:48
*** blkart has quit IRC10:48
*** shaohe_feng has joined #openstack-nova10:48
*** andreas_s has quit IRC10:53
*** jichen has quit IRC10:55
gibiyikun: it is on my list10:55
*** stelucz has quit IRC10:56
*** cnf has joined #openstack-nova10:56
cnfhi, anyone know what would cause the creation of a new instance to give me "Block Device Mapping is Invalid" ?10:57
cnfon -cinder, someone said it looks like a nova issue10:57
*** shaohe_feng has quit IRC10:57
*** shaohe_feng has joined #openstack-nova10:58
*** yamamoto has quit IRC11:01
*** ttsiouts has joined #openstack-nova11:04
*** blkart has joined #openstack-nova11:05
*** andreas_s has joined #openstack-nova11:07
*** shaohe_feng has quit IRC11:08
*** shaohe_feng has joined #openstack-nova11:11
yikungibi: ok, thanks :)11:11
*** andreas_s has quit IRC11:13
*** andreas_s has joined #openstack-nova11:13
*** lpetrut has quit IRC11:17
*** shaohe_feng has quit IRC11:18
*** shaohe_feng has joined #openstack-nova11:22
*** Guest88425 has quit IRC11:22
*** blkart has quit IRC11:23
*** blkart has joined #openstack-nova11:23
*** Bhujay has quit IRC11:24
*** shaohe_feng has quit IRC11:28
openstackgerrithuanhongda proposed openstack/nova master: Microversion 2.64 - Add "deleted" time in GET server response  https://review.openstack.org/57415911:28
*** chyka has joined #openstack-nova11:29
*** udesale has quit IRC11:29
*** shaohe_feng has joined #openstack-nova11:32
*** chyka has quit IRC11:35
*** EvilienM is now known as EmilienM11:36
*** aloga has quit IRC11:37
*** shaohe_feng has quit IRC11:38
*** shaohe_feng has joined #openstack-nova11:39
*** markvoelker has joined #openstack-nova11:40
*** pchavva has joined #openstack-nova11:48
*** shaohe_feng has quit IRC11:49
*** shaohe_feng has joined #openstack-nova11:49
*** damien_r has joined #openstack-nova11:51
*** alexchadin has quit IRC11:56
*** alexchadin has joined #openstack-nova11:57
*** alexchadin has quit IRC11:57
*** alexchadin has joined #openstack-nova11:58
*** alexchadin has quit IRC11:58
*** alexchadin has joined #openstack-nova11:59
*** alexchadin has quit IRC11:59
*** shaohe_feng has quit IRC11:59
*** alexchadin has joined #openstack-nova11:59
*** shaohe_feng has joined #openstack-nova12:00
*** alexchadin has quit IRC12:00
*** yamamoto has joined #openstack-nova12:02
*** lyan has joined #openstack-nova12:05
*** lyan is now known as Guest9214212:05
*** alexchadin has joined #openstack-nova12:07
*** BrinZhang has quit IRC12:07
*** shaohe_feng has quit IRC12:09
*** shaohe_feng has joined #openstack-nova12:10
*** markvoelker has quit IRC12:12
*** tianhui has quit IRC12:13
*** yamamoto has quit IRC12:15
*** jmlowe has joined #openstack-nova12:16
*** shaohe_feng has quit IRC12:19
*** amoralej is now known as amoralej|lunch12:20
*** shaohe_feng has joined #openstack-nova12:21
*** markvoelker has joined #openstack-nova12:22
*** ragiman has quit IRC12:24
gibiyikun: did I expressed my opininon in every open issue in https://review.openstack.org/#/c/563401/28/nova/notifications/objects/server_group.py@41 or did I missed something?12:26
gibiyikun: ahh the conversion thing12:27
gibiyikun: i missed that12:27
yikunyep, : )12:27
yikunconversion12:27
gibiyikun: would it be hard to make the int conversion in the notification body as well?12:27
*** edmondsw has joined #openstack-nova12:27
yikunNo, just need put some duplicate code into notification obj.12:28
yikunlike https://review.openstack.org/#/c/563375/39/nova/objects/instance_group.py@14012:29
*** alexchadin has quit IRC12:29
*** shaohe_feng has quit IRC12:30
gibiyikun: I see. We can only store a DictOfString even if your values will be in different types in the future12:30
*** shaohe_feng has joined #openstack-nova12:30
gibiyikun: can we call the rules property from the notification payload generation? let me look a bit deeper in the code...12:31
*** udesale has joined #openstack-nova12:31
*** maciejjozefczyk has quit IRC12:31
gibiyikun: I guess we call the rules property but then we store the result in a DictOfString in the payload as well12:32
gibiyikun: which converts it back to string12:32
*** eharney has joined #openstack-nova12:32
yikunyes, so if we don't do convert, we will get back a "3"12:32
gibiyikun: as we don't have a Dict type with variable value type12:32
gibiyikun: we cannot define a proper field in the payload class12:33
*** mriedem has joined #openstack-nova12:33
gibiyikun: and the payload class is something we think about as a contract for the consumers12:33
yikunyes, at least I can't see a DictField type in ovo. :)12:33
gibiyikun: let's keep it as "3"12:34
*** tiendc has quit IRC12:34
gibiyikun: I would not like to have a DictOfString defined in the class but we would emit an int as a value12:34
gibiin the notification12:34
*** udesale_ has joined #openstack-nova12:35
gibiyikun: commenting it in the review...12:35
*** phuongnh has quit IRC12:35
*** udesale_ has quit IRC12:36
*** ttsiouts has quit IRC12:36
*** udesale has quit IRC12:36
*** alexchadin has joined #openstack-nova12:37
gibiyikun: done12:37
gibiyikun: thanks for your patientes12:38
yikunOK, I see, much thanks for your time and help. :)12:38
gibis/patientes/patience12:38
yikun:), ha, I know12:39
gibi:)12:39
*** ragiman has joined #openstack-nova12:39
*** shaohe_feng has quit IRC12:40
yikunand it time to leave and back home, have a good day.12:40
gibigood day to you too12:40
*** shaohe_feng has joined #openstack-nova12:41
mriedemgibi: so we're leaving it as a string in the notification payload?12:41
gibimriedem: yes12:41
gibimriedem: because the field type is DictOfString and when we finally generate json schemas for notification classes then that will also created from the field typw12:42
gibitype12:42
*** med_ has joined #openstack-nova12:43
mriedemi'm assuming yikun pointed out that for InstanceGroup.rules we coerce the known max_server_per_host field value to an int12:44
*** armaan has quit IRC12:45
gibimriedem: yes, I saw that. gmann also pointed out that can be lead to some json validation inconvenience12:47
gibimriedem: if we flatten the API12:47
openstackgerritMatt Riedemann proposed openstack/nova master: Skip ServerShowV257Test.test_rebuild_server for cells v1 job  https://review.openstack.org/58171712:49
*** yamamoto has joined #openstack-nova12:50
*** shaohe_feng has quit IRC12:50
mriedemfor the rest api, we accept strings or ints apparently12:51
*** ragiman has quit IRC12:51
*** ttsiouts has joined #openstack-nova12:51
mriedemwhich was news to me12:51
mriedemfor the positive_integer type12:51
*** shaohe_feng has joined #openstack-nova12:51
mriedemhttps://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L23812:51
gibimriedem: and we patternmatch, I see12:51
mriedemso i'm not sure why {'max_server_per_host': 3} vs {'max_server_per_host': '3'}12:52
mriedemwill make a difference12:52
mriedemtbc, the InstanceGroup.rules doing the cast to int for that value is for internal convenience more than anything,12:52
mriedemso we don't have to remember to cast the value in all of the code that uses it, like the view builder in the api, the scheduler filter and the late affinity check in the compute12:52
*** jchhatbar has joined #openstack-nova12:53
gibimriedem: I think the api accepts strings for historical reasons12:54
*** janki has quit IRC12:55
gibimriedem: I sure I'm OK to cast it to int internally for counting and comparison12:55
*** med_ has quit IRC12:56
*** myrat has quit IRC12:57
mriedemok. i just don't understand the json schema validation concern12:57
mriedemis that a concern for the consumer of the notification?12:57
*** belmoreira has joined #openstack-nova12:57
gibimriedem: no it is not. I refer to gmann's comment in https://review.openstack.org/#/c/563401/28/nova/notifications/objects/server_group.py@41 about the need to move the validation from json to code12:59
*** belmoreira has quit IRC12:59
*** shaohe_feng has quit IRC13:00
mriedemoy13:01
mriedemthere is only 1 policy per group, and the rules are applied to that policy on that group,13:01
mriedemwe've already named the InstanceGroup.rules field13:01
*** openstack has joined #openstack-nova13:03
*** ChanServ sets mode: +o openstack13:03
gibiyeah that is gmann's point13:03
mriedemwhich i think is ok, it's like a 2 line validator13:03
gibiI agree13:04
mriedemif group.policy == 'anti-affinity' and group.rules: raise HTTPBadRequest13:04
openstackgerritStephen Finucane proposed openstack/nova master: Revert "docs: Disable smartquotes"  https://review.openstack.org/57884113:04
mriedemoops13:04
mriedem!= 'anti-affinity'13:04
*** lifeless has quit IRC13:04
gibisure13:05
gibipolicy_rule feels a better name for me but I accept rule with a good api doc13:05
openstackgerritsahid proposed openstack/nova stable/queens: hardware: fix hugepages memory usage per intances  https://review.openstack.org/58173613:06
openstackmriedem: Error: "=" is not a valid command.13:08
*** abhishekk has quit IRC13:09
*** janki has joined #openstack-nova13:09
*** jchhatbar has quit IRC13:09
*** shaohe_feng has quit IRC13:11
*** shaohe_feng has joined #openstack-nova13:11
*** amoralej|lunch is now known as amoralej13:13
*** itlinux has quit IRC13:14
*** med_ has joined #openstack-nova13:14
*** med_ has quit IRC13:14
*** med_ has joined #openstack-nova13:14
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: Fix TypeError in prep_resize allocation cleanup  https://review.openstack.org/58174113:17
mriedemgibi: i think you might have officially made yourself the other core to review https://review.openstack.org/#/c/563375/13:17
mriedemdansmith would +2 it but he made some non-trivial changes to it13:17
*** spsurya_ has quit IRC13:18
openstackgerritChris Dent proposed openstack/nova master: [placement] add error.code on a ConcurrentUpdateDetected  https://review.openstack.org/58174213:19
*** aarents1 has joined #openstack-nova13:21
*** shaohe_feng has quit IRC13:21
mriedemgibi: btw, you've been quite about reviews on the bw resource provider series - what's the status on that? are you looking for reviews on incremental things? is it done end to end or still working on changes for the full stack in nova (and maybe deps in neutron)?13:21
mriedemmy giant port binding live migration series is basically blocked on neutron dependencies13:21
*** shaohe_feng has joined #openstack-nova13:21
*** burt has joined #openstack-nova13:22
*** aarents has quit IRC13:22
*** alexpilotti has quit IRC13:23
*** armaan has joined #openstack-nova13:23
openstackgerritChris Dent proposed openstack/nova master: Test for unsanitized consumer UUID  https://review.openstack.org/58113713:24
gibimriedem: I will go through the remainings of https://review.openstack.org/#/c/563375/ soon13:28
gibimriedem: the status of bandwidth is that the code up on review stopped working after the official support for nested allocation_candidate support is merged13:29
gibimriedem: the solution is to support the new placement API versions from the scheduler report client13:29
gibimriedem: for that I started with a failing functional test here https://review.openstack.org/#/c/527728/13:30
mriedemyikun: gmann: alex_xu: so i see there must have been some discussion on https://review.openstack.org/#/c/567534/ and the request format wasn't changed but the response fields were changed...but no comments in the review about that decision?13:31
*** shaohe_feng has quit IRC13:31
*** hongbin has joined #openstack-nova13:32
*** shaohe_feng has joined #openstack-nova13:34
alex_xumriedem: no decision yet, I will try to go through the review13:37
alex_xumriedem: do you remember whether we discuss what should we do if the image traits check failed for rebuild https://review.openstack.org/#/c/569498/ ?13:38
*** ratailor has quit IRC13:39
*** ttsiouts has quit IRC13:41
*** shaohe_feng has quit IRC13:41
*** shaohe_feng has joined #openstack-nova13:42
mriedemalex_xu: glancing at the comments, the question is if the instance should be in ACTIVE or ERROR state?13:43
alex_xumriedem: yea, that is the question13:43
mriedemif we hit the scheduler during rebuild today because the image changes, and we get NoValidHost, we put the instance into ERROR state13:43
mriedemso i'd expect we do the same if the new image traits are invalid for the instance.host - which is essentially the same thing as a NoValidHost we'd get from the scheduler13:44
mriedemyou can rebuild an instance in ERROR state btw13:45
alex_xuoh, we can rebuild an instance in error, that sounds better13:45
*** ttsiouts has joined #openstack-nova13:46
mriedemhttps://review.openstack.org/#/q/Ibb7bee15a3d4ee6f0ef53ba12e8b41f65a1fe99913:46
mriedemsetting the instance to ERROR is a relatively new change13:46
*** ragiman has quit IRC13:46
alex_xuotherwise I feel the user just chocie a wrong image, then his instance get into error, he only can get help from admin13:46
*** mdnadeem has quit IRC13:46
mriedemthey can rebuild with a good image13:47
mriedemi remember this change and brought it up during the ptg in dublin13:47
mriedemdiscussing maybe re-architecting how rebuild works,13:47
mriedemthe reason we set it to ERROR state is not just because scheduling failed,13:47
*** lifeless has joined #openstack-nova13:47
mriedembut also because we've changed values on the instance itself in the api before casting to conductor,13:47
mriedemso at that point, we can't rollback those changes13:47
mriedemsee my comments in https://review.openstack.org/#/c/536268/13:48
*** Eran_Kuris has quit IRC13:50
alex_xumriedem: ah, i see now, that problem is out of the scope https://review.openstack.org/#/c/569498/13:50
*** psachin has quit IRC13:51
mriedemyup13:51
mriedemi need to review that rebuild + image traits change too13:51
mriedemsince i'm the one that got it stuck in committee during spec review13:51
*** shaohe_feng has quit IRC13:52
mriedemdansmith: so you don't see a need to name the new fields in the server group API policy_name and policy_rules right?13:52
alex_xumriedem: yea13:52
*** shaohe_feng has joined #openstack-nova13:52
mriedembecause yikun re-wrote the api change to do that, which makes it inconsistent with both the notification payload and the internal object modeling13:52
dansmithmriedem: I don't, but gmann said he thought it was important13:52
mriedemi don't think it's important or really confusing13:53
dansmithme either13:53
mriedemi think it's more important that we have consistency up and down the stack13:53
mriedeminternal and external13:53
*** jistr is now known as jistr|mtg13:56
openstackgerritEric Fried proposed openstack/nova master: Delete orphan compute nodes before updating resources  https://review.openstack.org/57992213:57
*** rmart04 has joined #openstack-nova13:57
*** eharney has quit IRC13:58
*** mriedem has quit IRC14:01
*** shaohe_feng has quit IRC14:02
*** mriedem has joined #openstack-nova14:02
mriedemgibi: just ping me if / when you need reviews on the bw provider changes14:03
*** shaohe_feng has joined #openstack-nova14:03
*** awaugama has joined #openstack-nova14:04
*** armaan has quit IRC14:07
*** armaan has joined #openstack-nova14:07
*** felipemonteiro_ has joined #openstack-nova14:08
openstackgerritMerged openstack/nova master: Remove irrelevant comment  https://review.openstack.org/57882114:12
*** shaohe_feng has quit IRC14:12
*** shaohe_feng has joined #openstack-nova14:13
*** janki has quit IRC14:15
openstackgerritSurya Seetharaman proposed openstack/nova master: Update queued-for-delete from the ComputeAPI during deletion/restoration  https://review.openstack.org/56681314:16
*** alexpilotti has joined #openstack-nova14:16
*** felipemonteiro__ has joined #openstack-nova14:16
*** cfriesen has joined #openstack-nova14:18
*** felipemonteiro_ has quit IRC14:20
*** tidwellr has joined #openstack-nova14:20
*** alexpilotti has quit IRC14:21
*** alexpilotti has joined #openstack-nova14:22
*** shaohe_feng has quit IRC14:22
*** mlavalle has joined #openstack-nova14:23
*** shaohe_feng has joined #openstack-nova14:23
*** Bhujay has joined #openstack-nova14:25
*** alexpilotti has quit IRC14:26
*** linkmark has joined #openstack-nova14:28
*** alexpilotti has joined #openstack-nova14:28
*** Bhujay has quit IRC14:30
*** armaan has quit IRC14:31
*** armaan has joined #openstack-nova14:31
gibimriedem: thanks.14:33
*** shaohe_feng has quit IRC14:33
efriedalex_xu: Are you still -1 on that change? Given that we've concluded NoValidHost is the right thing, having those logs be ERROR seems appropriate, 对吧14:34
*** shaohe_feng has joined #openstack-nova14:34
*** antosh has joined #openstack-nova14:36
*** spsurya_ has joined #openstack-nova14:37
mriedemefried: we don't ERROR for NoValidHost in the scheduler14:37
mriedemERROR means the operator needs to investigate b/c there is a problem in the system,14:37
mriedemin this case, it's really a user error like a 40014:37
mriedemwe don't log errors for that14:37
mriedemdebug at most14:37
efriedmriedem: So in this case, how would the user know what went wrong?14:38
mriedemdo they get a fault recorded?14:39
mriedemset_vm_state_and_notify will record a fault on the instance for the NoValidHost,14:39
*** amrith has joined #openstack-nova14:40
mriedemand the user should be able to see the fault message for the error14:40
*** Guest2110 has quit IRC14:40
*** amrith is now known as Guest4894014:40
mriedemhttps://developer.openstack.org/api-guide/compute/faults.html#instance-faults14:40
mriedemwhich reminds me, you can only see faults for ERROR or DELETED servers, so that's another reason to set the server to ERROR in this case so they can see the fault14:41
mriedemso as long as the NoValidHost has a reasonable message we should be fine14:41
mriedemthat would be a good wrinkle for the functional test though right?14:42
openstackgerritdo3meli proposed openstack/nova master: docs: add nova host-evacuate command to evacuate documentation  https://review.openstack.org/57804014:43
*** shaohe_feng has quit IRC14:43
*** markvoelker has quit IRC14:44
mriedemwhy is all this shared provider stuff in this change? https://review.openstack.org/#/c/569498/12/nova/tests/functional/libvirt/test_shared_resource_provider.py14:44
*** markvoelker has joined #openstack-nova14:44
efriedmriedem: I asked for it.14:44
*** shaohe_feng has joined #openstack-nova14:45
efriedmriedem: Because it's important that we're checking all and only providers involved in the allocation.14:45
efriedmriedem: So we should also be checking nested, but letting that slide for now because we don't have any scenarios that actually use nested yet.14:45
mriedemhmm, ok.14:46
mriedemand those unit tests look to be completely redundant for conductor14:46
mriedemredundant with the functional tests14:46
mriedemwe don't need both for the same scenarios14:46
mriedemhttps://review.openstack.org/#/c/569498/12/nova/tests/functional/libvirt/test_shared_resource_provider.py is just a really weird location for rebuild tests...14:47
efriedif it's testing how shared resource providers are handled by rebuild by libvirt, it seems as good as anything.14:48
*** ttsiouts has quit IRC14:48
*** markvoelker has quit IRC14:49
mriedemthe validation being added to conductor is virt-agnostic14:49
mriedemso it's a weird place to have those tests14:49
mriedemis all14:49
efriedbut libvirt is the only driver that's doing shared providers (properly) at the moment.14:50
mriedemyeah i know, but that doesn't really matter here does it?14:50
mriedemit's just a convenient place to put the tests given the setUp14:50
mriedemwe could have just as easily written a functional test that populate providers and allocations in placement and then run rebuild14:51
mriedemanyway, i'm not -1 on it, it's just weird14:51
efriedAgree the main motivator was probably convenience as you say.14:51
*** shaohe_feng has quit IRC14:53
*** rodolof has quit IRC14:53
*** shaohe_feng has joined #openstack-nova14:54
*** jistr|mtg is now known as jistr14:54
*** ttsiouts has joined #openstack-nova14:55
mriedemi'd like to nuke all of these checks http://codesearch.openstack.org/?q=We%20need%20to%20mock%20that%20the%20old%20way&i=nope&files=&repos=14:57
mriedemseems we can do that now right?14:57
mriedemoh nvm, we haven't dropped the migrate_instances_add_request_spec online data migration yet14:58
*** eharney has joined #openstack-nova14:58
*** alexpilotti has quit IRC14:59
mriedemdansmith: it's probably not very possible to do a blocker migration for the request spec migration is it?14:59
mriedemsince it's api db + multi-cell14:59
*** ivve has joined #openstack-nova14:59
mriedemwe could add a nova-status check pretty easily14:59
mriedemiterate all cells and make sure all non-deleted instances have a request spec14:59
dansmithfor which migration?15:00
dansmithah15:00
*** alexpilotti has joined #openstack-nova15:00
dansmithyeah15:00
mriedemmigrate_instances_add_request_spec15:00
*** alexpilotti has quit IRC15:00
mriedemyeah - i want to kill all of these "if not request_spec: populate from garbage" in the code15:00
dansmithhah yeah15:00
mriedemwe can't do a blocker db migration in db sync15:00
*** rodolof has joined #openstack-nova15:00
mriedemwell, shouldn't15:00
mriedem?15:00
mriedembut nova-status could do the job, i mean this migration has been around since newton15:01
dansmithI think people have also expressed displeasure with the blocker migrations, so any time we're not enforcing some really structural thing (like a column being non-null or something) a nova-status would be better15:01
mriedemso nova-status in rocky and drop in stein15:01
mriedemdrop compat in stein i mean15:01
dansmithaye15:01
openstackgerritChris Dent proposed openstack/nova master: Add placement.concurrent_udpate to generation pre-checks  https://review.openstack.org/58177115:02
*** shaohe_feng has quit IRC15:03
*** TxGirlGeek has joined #openstack-nova15:05
*** shaohe_feng has joined #openstack-nova15:05
*** rodolof has quit IRC15:07
*** shaohe_feng has quit IRC15:14
*** markvoelker has joined #openstack-nova15:14
*** alexchadin has quit IRC15:14
*** shaohe_feng has joined #openstack-nova15:14
*** rmart04 has left #openstack-nova15:15
*** links has quit IRC15:16
tssuryamriedem: I was trying to understand the marker logic you have in _heal_instances_in_cell (want to do something similar for the migration tool for updating queued_for_delete), wanted to confirm something: every time the command is run it will loop through all the instances which might have been processed before as well right ? like there is no way to check and skip the ones we have already covered. All we can ensure is each time it does m15:20
*** yamahata has joined #openstack-nova15:21
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Fix nits in the handling down cell spec  https://review.openstack.org/58124315:21
mriedemtssurya: heal_allocations doesn't use a global marker like map_instances15:22
mriedemso yes today it will hit all instances,15:22
tssuryamriedem: hmm yea15:22
mriedemit's only using a per-cell marker while iterating instances in the cell when we have a limit specified (default of 50)15:23
tssuryabecause I have two online migration tools which are like this needing  a marker logic15:23
mriedemmigrate_instances_add_request_spec uses a global marker15:23
* tssurya looking15:24
mriedemso i'd follow that pattern15:24
*** shaohe_feng has quit IRC15:24
mriedemmap_instances is slightly different15:25
mriedemit uses a sentinel project_id and munges the last real instance mapping uuid15:25
tssuryayea that I am aware , but I don't want to insert a new row and all15:26
mriedemyou're going to have to either way15:26
mriedemotherwise how would you find the marker record between runs of the command?15:27
tssuryaah just looked at migrate_instances_add_request_spec; it is using the same logic15:27
tssuryawith a FAKE_UUID15:27
mriedemyes15:27
tssuryawhat if more than 1 tool has this FAKE_UUID ?15:28
mriedemi don't care about out of tree tools15:28
*** jmlowe has quit IRC15:28
mriedemthe alternative is you do like map_instances and use a fake project_id and munge the uuid from an existing mapping15:28
mriedembut some people hate that too15:29
mriedemsince it's not a real uuid15:29
*** shaohe_feng has joined #openstack-nova15:29
tssuryauh-huh yea, well I guess it might be better than starting a series of FAKE_UUIDs15:29
mriedemthe sentinel uuid is always the same 00000000-0000-0000-0000-00000000000015:30
mriedemwe don't have anything else in nova, as far as i know, that inserts instance mappings with that uuid15:30
tssuryaoh okay15:31
mriedembetween migrate_instances_add_request_spec and map_instances i'd just personally follow what was done in migrate_instances_add_request_spec15:31
mriedemit's nearly the same thing you need15:31
mriedemminus creating a request spec and all that - so your thing will be much lighter15:32
mriedemone thing about migrate_instances_add_request_spec is that it's not multi-cell aware15:32
mriedembut i think we said in vancouver that running db sync and online data migrations per cell is not a big deal15:33
*** yamamoto has quit IRC15:33
tssuryamriedem: I was just getting to that actually15:33
tssuryathinking if my marker should be in instance_mappings or instances15:33
mriedemL39 https://etherpad.openstack.org/p/YVR18-cellsv2-migration-sync-with-operators15:34
tssuryaas I will be fetching deleted (or soft_deleted) instances and then updating only those instance_mappings to true15:34
*** shaohe_feng has quit IRC15:34
mriedemmigrate_instances_add_request_spec puts the marker in the request_specs table15:34
mriedemso again, i'd just follow that pattern :)15:34
mriedemsince i will assume people thought about all of this when migrate_instances_add_request_spec was written15:34
*** kaisers__ has joined #openstack-nova15:34
tssuryamriedem: okay then! I will go for the same consistency and follow the migrate_instances_add_request_spec stuff15:35
*** shaohe_feng has joined #openstack-nova15:35
tssuryadansmith: are we having cells meeting today ? (I don't have anything new we haven't discussed and have the needed feedback for the spec to work on)15:37
dansmithtssurya: I don't have anything either, mriedem, melwitt ?15:38
*** felipemonteiro__ has quit IRC15:40
*** med_ has quit IRC15:42
*** sapd1 has joined #openstack-nova15:42
sapd1Hi everyone, How can I remove an object from BlockDeviceMappingList object.15:43
*** bbock has quit IRC15:43
*** chyka has joined #openstack-nova15:44
*** tssurya has quit IRC15:44
*** shaohe_feng has quit IRC15:44
mriedemdansmith: nothing pressing, just starting reviews on tssurya's changes and need reviews on this bug fix which is related to cells v2 (build requests and quota counting): https://review.openstack.org/#/q/topic:bug/1780373+(status:open+OR+status:merged)15:45
*** shaohe_feng has joined #openstack-nova15:45
dansmithack\15:46
mriedemsapd1: can you provide more context?15:46
*** peereb has quit IRC15:46
openstackgerritdo3meli proposed openstack/nova master: docs: add nova host-evacuate command to evacuate documentation  https://review.openstack.org/57804015:47
sapd1mriedem: I would like to customize bdms list. In this function: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L275215:48
mriedemsapd1: because you're trying to rebuild a volume-backed instance?15:50
mriedemsapd1: if so, you should read this spec first https://review.openstack.org/#/c/532407/15:51
sapd1mriedem: I'm trying, I don't find any change which can work, So I'm trying to make it run.15:52
mriedemwell read the spec and discussion in there15:54
mriedemor if you're going to fork it anyway, you could look at https://review.openstack.org/#/c/528740/ but i wouldn't use that method15:54
*** andreas_s has quit IRC15:54
*** shaohe_feng has quit IRC15:55
*** andreas_s has joined #openstack-nova15:55
mriedemthere are in fact several attempts at this in https://bugs.launchpad.net/nova/+bug/137868915:55
openstackLaunchpad bug 1378689 in OpenStack Compute (nova) "error when rebuilding a instance booted from volume" [Undecided,In progress] - Assigned to Jie Li (ramboman)15:55
*** issp has quit IRC15:55
*** shaohe_feng has joined #openstack-nova15:56
*** andreas_s has quit IRC15:59
*** tesseract has quit IRC16:01
*** wwriverrat has quit IRC16:02
sapd1mriedem: This feature is sound like have too many bugs :D16:02
mriedemwell, that's what happens when we add features without thinking about implications to existing APIs16:04
mriedemi'd like to see https://review.openstack.org/#/c/532407/ get agreement on a direction in stein so we can finally fix this16:04
*** shaohe_feng has quit IRC16:05
sapd1mriedem: Yep, Because attach/detach flow in cinder has changed. So the rebuild process will be failed.16:05
mriedemi'm not sure i follow16:05
mriedemare you talking about the volume attachments API in cinder?16:06
*** shaohe_feng has joined #openstack-nova16:06
sapd1mriedem: Could you tell me how to remove an item in BlockDeviceMappingList in the function?16:06
mriedemthat doesn't really have anything to do with the fact nova has never supported rebuilding a volume-backed server with a new image16:06
sapd1mriedem: Yep. Because multi-attach was introduced. So attach/detach flow have to change.16:07
mriedemsapd1: those are backward compatible changes and nova opts into using those apis16:07
mriedemso it doesn't really mean anything for what you're trying to do16:07
sapd1I want to remove old root disk block device mapping in that list.16:08
mriedemwe added a change to the api in queens such that you literally cannot rebuild a volume-backed server with a new image16:08
mriedemthe user gets an error16:08
*** jamesdenton has joined #openstack-nova16:08
mriedemhttps://review.openstack.org/#/c/520660/16:09
*** ttsiouts has quit IRC16:09
sapd1Use this patch I can bypass https://review.openstack.org/#/c/528740/8/nova/compute/api.py16:10
sapd1:D16:10
mriedemif you want to fork a hack into your product that's up to you16:11
mriedembut i'm not going to spend time helping you do it, sorry16:11
mriedemi'm only interested in https://review.openstack.org/#/c/532407/16:11
sapd1mriedem: So Has anyone implemented this spec yet?16:12
mriedemsapd1: no, see my -1 on the spec16:13
mriedemneed to agree on the design first16:13
mriedemwhich is why the 20 other hack fixes for this problem have been rejected in the past16:13
sapd1mriedem: I'm trying :D thanks16:14
mriedemmy comment from march 27 is where i think it stalled16:15
mriedem"The cleanest / best solution to this is to add a volume action API to cinder for re-imaging the volume. Once that is available in a new cinder v3 microversion, nova can use it. The reason I think this should be done in Cinder with re-imaging the volume there is (1) it's cleaner from the nova side and (2) then Cinder is in control of how that re-image should happen, along with any details it needs to update, e.g. the volume's16:15
mriedemlume_image_metadata" information would need to be updated.We really don't want to do the volume create/delete/swap orchestration thing since that entails issues with the volume type being gone, going over quota, what to do about deleting the old volume, etc.So please propose a spec to Cinder and start working the API changes there and then nova can depend on a new Cinder API."16:15
*** maciejjozefczyk has joined #openstack-nova16:15
*** shaohe_feng has quit IRC16:15
mriedemi think of this like shelve offloading and unshelving a volume-backed server but with a new image16:15
mriedemand i'd like cinder to control the api for re-imaging the volume16:16
mriedembecause cinder owns the volume16:16
*** shaohe_feng has joined #openstack-nova16:16
mriedemmaybe i need to work with tommylikehu on this in stein16:17
mriedemmelwitt: it might not be too early to start a stein ptg etherpad for random items16:18
*** mgoddard has quit IRC16:18
*** armaan has quit IRC16:21
*** armaan has joined #openstack-nova16:21
*** efried is now known as efried_off16:23
sapd1mriedem: Yep. I think It can work. with this patch.16:24
sapd1mriedem: http://paste.ubuntu.com/p/V3MhY7Hp3X/16:24
sapd1I change flow to detach old root disk device and create attachment ID for new rootdisk device16:25
*** shaohe_feng has quit IRC16:25
*** armaan has quit IRC16:25
*** shaohe_feng has joined #openstack-nova16:26
openstackgerritMerged openstack/nova master: Test for unsanitized consumer UUID  https://review.openstack.org/58113716:27
gibimriedem: left comments in the complex-anti-affinity-policies series but I haven't finished yet. I run out of time. I can check the API patch tomorrow if needed16:30
*** efried_off has quit IRC16:31
*** efried_off has joined #openstack-nova16:31
*** yamamoto has joined #openstack-nova16:33
openstackgerritMatt Rabe proposed openstack/nova master: Add destination MSP IP address to PowerVM migrate data  https://review.openstack.org/57967616:33
*** shaohe_feng has quit IRC16:36
*** yamahata has quit IRC16:39
*** yamamoto has quit IRC16:39
*** yamahata has joined #openstack-nova16:39
*** shaohe_feng has joined #openstack-nova16:39
*** arvindn05 has quit IRC16:45
*** shaohe_feng has quit IRC16:46
*** shaohe_feng has joined #openstack-nova16:47
*** felipemonteiro has joined #openstack-nova16:50
*** markvoelker has quit IRC16:52
*** shaohe_feng has quit IRC16:56
*** shaohe_feng has joined #openstack-nova16:57
*** derekh has quit IRC17:00
*** yamahata has quit IRC17:03
*** imacdonn has quit IRC17:03
*** imacdonn has joined #openstack-nova17:03
*** sapd1 has quit IRC17:04
*** amoralej is now known as amoralej|off17:05
*** jdillaman has joined #openstack-nova17:06
*** shaohe_feng has quit IRC17:06
*** shaohe_feng has joined #openstack-nova17:07
*** _pewp_ has quit IRC17:08
*** _pewp_ has joined #openstack-nova17:09
*** mago_ has quit IRC17:16
*** shaohe_feng has quit IRC17:17
*** shaohe_feng has joined #openstack-nova17:17
*** moshele has joined #openstack-nova17:18
*** gyee has joined #openstack-nova17:21
*** markvoelker has joined #openstack-nova17:24
melwittmriedem: yeah, probably a good idea to start one. I've seen others posted to the ML already, at least for cinder17:25
*** shaohe_feng has quit IRC17:27
*** armaan has joined #openstack-nova17:27
*** shaohe_feng has joined #openstack-nova17:28
*** sambetts is now known as sambetts|afk17:29
*** armaan has quit IRC17:31
*** armaan has joined #openstack-nova17:32
*** sahid has quit IRC17:32
*** damien_r1 has joined #openstack-nova17:34
*** damien_r has quit IRC17:34
*** yamamoto has joined #openstack-nova17:35
*** kaisers__ has quit IRC17:37
*** shaohe_feng has quit IRC17:37
*** yamahata has joined #openstack-nova17:37
*** belmoreira has joined #openstack-nova17:39
*** shaohe_feng has joined #openstack-nova17:39
*** yamamoto has quit IRC17:41
*** med_ has joined #openstack-nova17:41
*** moshele has quit IRC17:42
*** felipemonteiro has quit IRC17:42
melwittdansmith: I don't have anything for a cells meeting17:45
dansmithack17:45
*** shaohe_feng has quit IRC17:47
*** shaohe_feng has joined #openstack-nova17:48
melwittmriedem: comment on https://review.openstack.org/580755 , I've wondered why we don't use the CastAsCall fixture anymore in functional tests where we want to GET servers after we create them. everything else looks good in the tests17:50
*** armaan has quit IRC17:54
mriedemmelwitt: because CastAsCall isn't what you'd really get as a real user17:56
mriedemso unless there is a good reason to use, it don't know why we would17:56
*** shaohe_feng has quit IRC17:58
mriedemmelwitt: replied inline17:58
melwittmriedem: I was thinking because of the assumption of the servers before they're active, but in this case "active" is not needed or relevant in the test17:58
*** shaohe_feng has joined #openstack-nova17:58
mriedemright we don't care about status in these tests17:59
melwittI was thinking POST followed by GET without waiting for the status, but if you don't care about the status, then it doesn't matter. maybe that's the difference. none of the assertions you're doing depend on the boot process being further along17:59
melwitta few weeks ago I was working on a test that kept failing until I used CastAsCall, but it must have been that an assertion I had depended on the boot process getting to a certain point. sorry for the noise18:00
mriedemwe have utilities for waiting for the server to be a certain status as well18:01
mriedemso if i needed to wait, i'd use that instead of CastAsCall18:01
* mriedem moves rooms18:01
melwitttrue, I'll keep that in mind for next time18:01
*** mriedem1 has joined #openstack-nova18:02
*** maciejjozefczyk has quit IRC18:03
*** maciejjozefczyk has joined #openstack-nova18:03
*** maciejjozefczyk has quit IRC18:04
*** mriedem has quit IRC18:05
*** edmondsw has quit IRC18:07
*** edmondsw has joined #openstack-nova18:07
*** shaohe_feng has quit IRC18:08
*** sridharg has quit IRC18:08
*** shaohe_feng has joined #openstack-nova18:09
*** mriedem1 is now known as mriedem18:12
*** shaohe_feng has quit IRC18:18
*** kuzko has quit IRC18:18
*** shaohe_feng has joined #openstack-nova18:20
openstackgerritMerged openstack/nova master: [placement] add error.code on a ConcurrentUpdateDetected  https://review.openstack.org/58174218:20
*** shaohe_feng has quit IRC18:28
*** felipemonteiro has joined #openstack-nova18:29
*** shaohe_feng has joined #openstack-nova18:30
openstackgerritMatt Riedemann proposed openstack/nova master: Add nova-status upgrade check for request spec migrations  https://review.openstack.org/58181318:34
*** yamamoto has joined #openstack-nova18:37
*** shaohe_feng has quit IRC18:39
*** felipemonteiro_ has joined #openstack-nova18:39
*** shaohe_feng has joined #openstack-nova18:39
melwittmriedem: ptg etherpad if you had some topic(s) you wanted to jot down https://etherpad.openstack.org/p/nova-ptg-stein18:41
mriedemthanks18:42
*** felipemonteiro has quit IRC18:42
*** yamamoto has quit IRC18:43
*** tssurya has joined #openstack-nova18:48
*** shaohe_feng has quit IRC18:49
*** itlinux has joined #openstack-nova18:49
*** shaohe_feng has joined #openstack-nova18:50
*** cdent has quit IRC18:52
*** kuzko has joined #openstack-nova18:59
*** shaohe_feng has quit IRC18:59
*** harlowja has joined #openstack-nova19:00
*** itlinux has quit IRC19:02
*** shaohe_feng has joined #openstack-nova19:04
*** tidwellr has quit IRC19:05
*** antosh has quit IRC19:05
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: Add functional regressions tests for server_group_members OverQuota  https://review.openstack.org/58184519:06
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: Fix server_group_members quota check  https://review.openstack.org/58184619:06
*** tidwellr has joined #openstack-nova19:06
*** shaohe_feng has quit IRC19:09
*** shaohe_feng has joined #openstack-nova19:12
openstackgerritMatt Riedemann proposed openstack/nova stable/pike: Add functional regressions tests for server_group_members OverQuota  https://review.openstack.org/58186619:17
openstackgerritMatt Riedemann proposed openstack/nova stable/pike: Fix server_group_members quota check  https://review.openstack.org/58186719:17
*** antosh has joined #openstack-nova19:19
*** shaohe_feng has quit IRC19:20
*** antosh has quit IRC19:20
*** belmoreira has quit IRC19:21
*** shaohe_feng has joined #openstack-nova19:22
*** maciejjozefczyk has joined #openstack-nova19:24
mriedemcrazy croats19:25
*** jmlowe has joined #openstack-nova19:29
*** shaohe_feng has quit IRC19:30
*** flwang1 has quit IRC19:30
*** shaohe_feng has joined #openstack-nova19:31
*** tidwellr has quit IRC19:35
*** mgoddard has joined #openstack-nova19:36
*** tidwellr has joined #openstack-nova19:39
*** maciejjozefczyk has quit IRC19:40
*** yamamoto has joined #openstack-nova19:40
*** shaohe_feng has quit IRC19:40
*** mgoddard has quit IRC19:40
*** mdbooth has quit IRC19:41
*** shaohe_feng has joined #openstack-nova19:41
*** edmondsw has quit IRC19:42
*** edmondsw has joined #openstack-nova19:43
*** yamamoto has quit IRC19:45
*** tidwellr has joined #openstack-nova19:46
*** shaohe_feng has quit IRC19:50
*** shaohe_feng has joined #openstack-nova19:51
*** awaugama has quit IRC19:51
mriedemdansmith: comments on your bfv req spec change https://review.openstack.org/#/c/580720/19:54
*** itlinux has joined #openstack-nova19:55
*** itlinux has quit IRC19:57
*** TxGirlGeek has quit IRC19:59
*** tidwellr has quit IRC20:00
*** shaohe_feng has quit IRC20:01
dansmithooh20:02
*** shaohe_feng has joined #openstack-nova20:02
*** pchavva has quit IRC20:04
*** antosh has joined #openstack-nova20:04
*** alexpilotti has joined #openstack-nova20:05
*** TxGirlGeek has joined #openstack-nova20:06
mriedemso this isn't backportable b/c of the version change, but in thinking about this more - persisting the is_bfv flag is good b/c for any new instances created after this, moving them also benefits from the flag being set20:10
mriedemthe only gap is existing instances that don't have is_bfv set in the request spec, as noted in the commit message20:10
mriedemwhich we can easily detect and heal during a move20:10
mriedemthere are a few comments in the bug report about how once people got to ocata the allocations for volume-backed instances were fixed, and that's because the resource tracker had the is_bfv logic to not account for flavor.root_gb,20:11
*** shaohe_feng has quit IRC20:11
mriedembut then in pike we stopped creating allocations in the RT and based the allocations of the request spec, so we regressed that....20:11
mriedemso this (re)fixes that20:12
mriedembut we'd have a gap in pike and queens20:12
dansmithsorry, was looking at the review window20:12
dansmithI just replied,20:12
dansmithbut basically I didn't even consider this as backportable since it changes what we allocate and stuff and thought it wasn't worth the risk20:13
mriedemalmost seems like we could do a hybrid of https://review.openstack.org/#/c/428481/ and your new patch which could be backportable20:13
dansmithI'd rather just roll forward and heal20:13
mriedemi get that,20:14
*** shaohe_feng has joined #openstack-nova20:14
mriedembut this is a pretty high flame count bug or whatever20:14
mriedemso backportability seems worth at least thinking through20:14
dansmithour downstream requirement for stuff like this is that you have to be able to yum update components in any order on any machine, and having things starting to claim differently because you updated one conductor and didn't do another until a month later20:14
dansmitheven still, the scheduler is the thing that needs to know, not conductor like you said,20:14
dansmithso I'm not sure how we could do it without an object or rpc change20:15
dansmithif this was a regression in the last release then I'd care more I guess, but this has _always_ been this way, which makes it nearly a feature20:15
mriedemwell, it was fixed in ocata and then regressed again in pike20:16
dansmitheh? this has never worked that I know of20:16
mriedemi think anyway20:16
dansmithpretty sure not,20:16
mriedemin ocata the computes would create the allocations right?20:16
dansmithah, but, it has always been broken in RT :)20:16
*** tojuvone has quit IRC20:16
mriedemthe RT / report client code that created the allocations in ocata took bfv into account for DISK_GB20:16
dansmithso before we were doing allocations wrong, we were doing RT claiming wrong and failing super late20:16
mriedemRT == claim?20:16
mriedemwell,20:16
dansmithI mean old school claims20:16
*** tojuvone has joined #openstack-nova20:17
mriedemif you dropped the DiskFilter you wouldn't hit the claim20:17
dansmithpfft :)20:17
mriedemso in ocata, if you're using placement and remove DiskFilter,20:17
mriedemfixed20:17
dansmithwhere did we not allocate for volume-backed disks in RT?20:17
*** tidwellr has joined #openstack-nova20:18
mriedemhttps://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L16920:18
mriedemthis comment in the bug is a pretty good one https://bugs.launchpad.net/nova/+bug/1469179/comments/3720:18
openstackLaunchpad bug 1469179 in OpenStack Compute (nova) "instance.root_gb should be 0 for volume-backed instances" [Medium,In progress] - Assigned to Dan Smith (danms)20:18
*** arvindn05 has joined #openstack-nova20:18
mriedem"After disabling the RamFilter and DiskFilter, and instead relying on the  Placement API for scheduling, most of the "scheduling" aspect of this  problem has been addressed. I was able to drop a local patch to handle  this root_gb!=0 for EBS volumes, and for most real-life use cases it is  working. The placement API is not recording disk allocations for EBS  volumes."20:19
mriedemhttps://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L35120:19
dansmithso it kinda worked almost by accident for a release?20:20
mriedemso in ocata we wouldn't report root_gb for bfv instance allocations, you could still fail scheduling due to the old school RT disk claim if DiskFilter was enabled20:20
* dansmith isn't convinced20:20
mriedemi agree with the 'by accident' part20:20
mriedemand i don't think it was until at least pike that we said to disable the DiskFilter if you're using placement and remove it from the default enabled filters20:21
*** eharney has quit IRC20:21
dansmithright, I was thinking ocata still required that20:21
dansmithwell,20:21
*** shaohe_feng has quit IRC20:21
dansmithdidn't recommend placement as a complete replacement20:21
*** shaohe_feng has joined #openstack-nova20:22
mriedemyeah dropped CPU/Ram/DiskFilters from default enabled filters in pike20:22
mriedemwe'd been saying since at least newton that placement was the eventual fix for this problem :) that's why we kept putting off melwitt's workaround / RT patch20:23
mriedemand sure now it is20:23
dansmithaye20:23
dansmithwell, to be fair, we could have done this in queens at least too20:24
dansmithmeaning, it could have been the fix already20:24
mriedemsure20:24
mriedemand people could hack in a point-of-scheduling time fix in conductor if they need it..20:24
melwittdoes all of the "fixed" situations require that a flavor with root_gb=0 has been used? I'm reading through the comments trying to understand20:27
dansmithno, the right fix does not20:27
melwittbecause I thought, as of ocata, if flavor has root_gb > 0 placement would filter it out of consideration and removing DiskFilter would not help you20:28
dansmiththe middle solution requires sending a hacked flavor with root=0 to the scheduler as the signal to avoid a real signal20:28
dansmithright I think mriedem is saying that if you align all the planets on ocata, including set root=0, you can get it to kinda work if you don't sneeze near it20:29
melwittgot it20:29
mriedemwe actually don't want people using root_gb=0 in flavors20:29
dansmithright20:29
mriedemthere is a security issue for that20:29
dansmithand it's broken because it requires using specific flavors for bfv20:30
mriedemhttps://github.com/openstack/nova/commit/763fd62464e9a0753e061171cc1fd826055bbc0120:30
melwittyeah, just wanted to make sure I understood what the definition of "fixed" was in the convo20:30
mriedemi have to run, be back in a few20:30
mriedemdansmith: let's add the release note and change related-bug to partial-bug (for my ocd) and then i'm +220:30
dansmithack20:30
*** shaohe_feng has quit IRC20:31
dansmithmriedem: I'll remind you I added related-bug for you :)20:33
*** shaohe_feng has joined #openstack-nova20:35
*** spsurya_ has quit IRC20:36
dansmithdamn, you said "link" not specifically related20:36
dansmithI'20:36
dansmithI'll get you next time20:36
*** itlinux has joined #openstack-nova20:38
openstackgerritDan Smith proposed openstack/nova master: Avoid requesting DISK_GB allocation for root_gb on BFV instances  https://review.openstack.org/58072020:39
*** yamamoto has joined #openstack-nova20:41
*** shaohe_feng has quit IRC20:42
*** shaohe_feng has joined #openstack-nova20:45
*** yamamoto has quit IRC20:47
*** tidwellr has quit IRC20:48
*** tidwellr has joined #openstack-nova20:49
*** SpamapS has joined #openstack-nova20:49
*** itlinux has quit IRC20:50
*** shaohe_feng has quit IRC20:52
*** shaohe_feng has joined #openstack-nova20:55
*** flwang1 has joined #openstack-nova21:02
mriedemmuwahahaha21:02
*** shaohe_feng has quit IRC21:02
*** wwriverrat has joined #openstack-nova21:04
*** shaohe_feng has joined #openstack-nova21:04
*** antosh has quit IRC21:05
*** tidwellr has quit IRC21:07
mriedemdansmith: see my hedge in the release note21:08
mriedemi wonder if we should be clear that if you're not using the FilterScheduler (and/or using the DiskFilter) this is still a problem21:08
dansmithsure makes sense21:08
mriedembut, CachingScheduler is deprecated and DiskFilter should be disabled if using FilterScheduler21:08
*** jaypipes has quit IRC21:09
dansmithhow's this:21:10
openstackgerritDan Smith proposed openstack/nova master: Avoid requesting DISK_GB allocation for root_gb on BFV instances  https://review.openstack.org/58072021:10
mriedemlgtm21:11
mriedempew pew21:11
mriedemat some point we should drop the ironic flavor migration stuff, since then we can drop the remaining allocation stuff in the report client for the RT21:12
*** shaohe_feng has quit IRC21:12
mriedemthe only driver that reports allocations in the compute is the ironic driver21:13
*** shaohe_feng has joined #openstack-nova21:14
mriedem# TODO(johngarbutt) we should remove this once the resource class21:14
mriedem    # migration has been completed.21:14
mriedem    requires_allocation_refresh = True21:14
mriedemwhat's our signal that the resource class migration has been completed?21:14
dansmithtime and I think it's passed now21:15
mriedemwe do have a nova-status check for the ironic instance embedded flavor migration21:15
mriedemand the online data migration21:15
mriedemonline/offline21:15
mriedemand21:15
mriedem# TODO(jaypipes): Completely remove the reporting of VCPU, MEMORY_MB,21:15
mriedem        # and DISK_GB resource classes in early Queens when Ironic nodes will21:15
mriedem        # *always* return the custom resource class that represents the21:15
mriedem        # baremetal node class in an atomic, singular unit.21:15
mriedemso we could dump all of that now, or wait until early stein21:16
mriedemi'd like to start dropping compat code that's hung around since newton/ocata times21:16
mriedemjroll: penick: do you guys have your ironic/nova deployment fully migrated to using resource classes?21:17
jrollmriedem: I did that patch already: https://review.openstack.org/#/c/565841/21:17
jrolljust haven't taken the time to rebase21:17
jrollwe're still on juno, so... no. but will do in ocata afaik21:17
mriedemthe bm cloud is juno and the vm cloud is ocata?21:18
jrollyeah21:18
jrollstill working on ocata for bm21:18
mriedemok,21:18
mriedemwould be nice to know people have rolled bm deployments through this transition and there aren't any major issues21:18
mriedembefore we drop all of that code21:18
*** ccamacho has quit IRC21:19
mriedemeveryone seems to want an 18-24+ month window of support on transition code...21:19
jrollTheJulia or dtantsur|afk might have people that have been through that21:20
mriedemcern has bm and they are on queens now,21:20
* TheJulia raises an eyebrow21:20
mriedemtssurya: do you know if the cern baremetal deployment has gone through the custom resource class transition for ironic flavors?21:20
tssuryamriedem: yes21:21
jroll\o/21:21
jrollmriedem: I can get back to 565841 next week maybe, or someone can take it over if they want21:22
tssuryameaning we use resource classes for ironic flavors (reading the convo to find out what transition)21:22
*** shaohe_feng has quit IRC21:23
jrolltssurya: there's an online data migration for existing instances21:23
edmondswmriedem I think your changes in https://review.openstack.org/#/c/528835/ are related to a bug I just opened... take a look when you get a chance?  https://bugs.launchpad.net/nova/+bug/178128621:23
openstackLaunchpad bug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during rebuild" [Undecided,New]21:23
TheJuliajroll: mriedem: I'm sure some folks have in our org... That being said unless it is a stated policy I've noticed people upstream tend to try and push the envelope... and downstream seems to pay for it... I guess I'm just rambling that long, but not too long is preferable to short windows21:23
*** shaohe_feng has joined #openstack-nova21:24
jrollTheJulia: dropping it now would be two cycles21:24
mriedemTheJulia: downstream consumers would have upstream maintain everything forever if possible21:25
TheJuliato support skip-leveling, two would be necessary, right now most people seem to be focused on fast forwarding since that is essentially what we end up testing21:25
TheJuliamriedem: :(21:25
TheJuliamriedem: I do not disagree21:25
mriedemone release is necessary for FFU21:25
mriedemthere is no skip level21:25
*** antosh has joined #openstack-nova21:25
jrolloh there is skip level :P21:25
mriedemFFU means  rolling through each release, not going directly from ocata to queens21:26
TheJuliajroll: heh21:26
mriedemsure, there is, but it's totally undefined21:26
mriedemand therefore impossible to support21:26
jrollright21:26
jrolland having to deal with online data migrations gone missing in skip level is already an issue21:26
mriedemgiven we're close to the end of rocky, i don't know if we just punt until early stein for the big drop or do it now21:27
TheJuliaAnyway folks, I'm going to go like... enjoy... the big blue room.21:27
jrollat any rate, there's a patch up that needs rebasing, I'm fine either way21:27
mriedemwe might as well just wait21:27
TheJulia+1 to punting21:27
jrollmmmkay21:28
* TheJulia really goes to the big blue room now21:28
* jroll waits for eavesdrop to update so he can link this in a gerrit comment21:28
TheJulialol21:28
*** jmlowe has quit IRC21:29
openstackgerritMerged openstack/nova master: Add queued for delete to instance_mappings table.  https://review.openstack.org/56678821:32
openstackgerritMerged openstack/nova master: Fix TypeError in prep_resize allocation cleanup  https://review.openstack.org/58154821:32
mriedemjroll: ok added to https://etherpad.openstack.org/p/nova-ptg-stein21:32
openstackgerritMerged openstack/nova master: Add functional regressions tests for server_group_members OverQuota  https://review.openstack.org/58075521:33
openstackgerritMerged openstack/nova master: Fix server_group_members quota check  https://review.openstack.org/58068421:33
*** shaohe_feng has quit IRC21:33
mriedemi have no idea what the big blue room is21:33
*** shaohe_feng has joined #openstack-nova21:34
mriedemoh i see, the cool way to say go outside21:34
mriedemedmondsw: "The nova_cell1.conf does have [database]connection set:" - that's not the db we're failing on21:37
mriedemedmondsw: build requests are in the api db, which devstack won't configure for n-cond-cell121:37
mriedemsince the cell shouldn't be able to reach the api db21:37
mriedemedmondsw: if you have a recreate,21:38
*** felipemonteiro_ has quit IRC21:38
mriedemcan you add a debug line to dump filter_properties?21:38
edmondswyeah, I thought it might be the api db from a comment in your other change21:38
mriedembecause21:38
mriedemnum_attempts = filter_properties.get(21:38
mriedem                'retry', {}).get('num_attempts', 1)21:38
edmondswotp right now, will check back in a min21:38
mriedemis it a rolm phone?21:38
*** takashin has joined #openstack-nova21:39
*** oomichi has joined #openstack-nova21:40
*** shaohe_feng has quit IRC21:43
*** yamamoto has joined #openstack-nova21:43
*** Guest92142 has quit IRC21:43
*** shaohe_feng has joined #openstack-nova21:44
*** brault has quit IRC21:45
*** jmlowe has joined #openstack-nova21:45
*** yamamoto has quit IRC21:49
edmondswmriedem rolm?21:51
edmondswI think I can get the data you asked for21:52
mriedemhttp://www.wikicu.com/ROLM_Phone21:52
mriedemedmondsw: are you by chance forcing the host/node during server create21:52
mriedemusing --availability-zone nova:host:node?21:52
mriedemsomething like that?21:52
edmondswnot that I know of, but I'm not sure21:52
edmondswthis is in the PowerVM CI environment21:53
mriedemwell, are you issuing a nova boot?21:53
mriedemoh...21:53
edmondswso it's all automatated (hit this while redeploying it)21:53
edmondswso I'm not sure what it's doing without digging into the automation21:53
edmondswthis is the undercloud21:53
*** shaohe_feng has quit IRC21:53
mriedemok i guess check to see if it's forcing a host/node21:54
edmondswwill do21:54
mriedemor if CONF.scheduler.max_attempts = 121:54
*** shaohe_feng has joined #openstack-nova21:54
edmondswmax_attempts is the default (3)21:55
mriedemok21:55
edmondswmriedem I think these instances are being created by nodepool, if that tells you anything21:59
*** mchlumsky has quit IRC22:00
mriedemi'm going to see if i can recreate with a functional test quick22:00
edmondswI doubt we're specifying a specific host/node but I'm looking22:00
mriedembut no nodepool doesn't tell me anything22:00
*** brault has joined #openstack-nova22:00
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Create specs directory for Stein  https://review.openstack.org/57360222:00
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (5)  https://review.openstack.org/57084222:01
edmondswand whether we do or not, I don't know whether nodepool would22:01
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (6)  https://review.openstack.org/57133022:01
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (7)  https://review.openstack.org/57199222:02
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (8)  https://review.openstack.org/57199322:02
*** shaohe_feng has quit IRC22:04
*** brault has quit IRC22:05
*** shaohe_feng has joined #openstack-nova22:05
*** tssurya has quit IRC22:05
edmondswmriedem I think this is an example request body: body: {"server": {"name": "PowerVM_CI-PowerVM_DevStacked-440", "imageRef": "0dd943fd-ae60-4cdf-85fb-709d00f87db1", "flavorRef": "55b0569d-4f51-47fd-a326-8b333fedcce3", "max_count": 1, "min_count": 1, "networks": [{"uuid": "a4a8157c-da2f-462f-aa73-5c97fc7e17f1"}], "metadata": {"nodepool": "{\"provider_name\": \"PowerVM_DevStacked\", \"node_id\": 440, \"image_name\": \"22:06
edmondswPowerVM_Ubuntu_Base\"}", "groups": "[\"PowerVM_DevStacked\", \"PowerVM_Ubuntu_Base\"]"}}} {{(pid=31995) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:604}}22:06
edmondswno availability zone specified22:07
mriedemi don't know what \"node_id\": 440 is22:11
edmondswmriedem I think that's just something nodepool wants to remember, not important to nova22:13
edmondswmriedem the Exception block you pasted in comment 1 is not entered22:14
*** shaohe_feng has quit IRC22:14
edmondswthe error isn't until after that22:14
*** shaohe_feng has joined #openstack-nova22:14
*** rcernin has joined #openstack-nova22:15
*** antosh_ has joined #openstack-nova22:18
mriedemok log local_filter_props before the other instance of calling _destroy_build_request22:18
*** antosh has quit IRC22:20
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3)  https://review.openstack.org/57410422:21
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4)  https://review.openstack.org/57410622:22
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5)  https://review.openstack.org/57411022:22
mriedemedmondsw: what is the devstack env like? a single host but with multiple nova-compute services running the powervm driver?22:23
mriedemso a 1:M host:node setup?22:23
edmondswlocal_filter_props isn't set until a few lines after the error22:23
edmondswthe environment is a controller that runs most things (api, single conductor, etc.) and then multiple compute nodes22:24
*** shaohe_feng has quit IRC22:24
edmondswthe multiple compute nodes are all on actual physical nodes22:24
edmondswsince this is the undercloud22:25
*** shaohe_feng has joined #openstack-nova22:25
mriedemoh shite i see,22:25
mriedemget_host_availability_zone22:25
mriedemthat's an up-call to get aggregates for the host22:25
mriedemto find the az22:25
mriedemso we reschedule, pop another host off the selection list,22:27
mriedemand then call host22:28
mriedemoops22:28
mriedeminstance.availability_zone = (22:28
mriedem                availability_zones.get_host_availability_zone(context,22:28
mriedem                        host.service_host))22:28
mriedemwhich is an up-call to get aggregates in the api db22:28
mriedemdansmith: ^ add it to the list :)22:28
*** med_ has quit IRC22:28
mriedemhmm, i wouldn't think it's possible to reschedule out of an az once you're initially scheduled into one22:28
melwittyou can if you never requested a specific AZ and went to the default AZ. you won't necessarily stay in the default AZ during a move22:29
mriedemi have a patch to test that..22:30
mriedemhttps://review.openstack.org/#/c/567701/22:30
melwittso while a 'nova show' will show you're in the default AZ, your requested AZ is actually "None" aka I don't care what AZ I'm in22:30
melwitt(this came up recently on our bug call)22:30
mriedemhttps://bugs.launchpad.net/nova/+bug/177186022:31
openstackLaunchpad bug 1771860 in OpenStack Compute (nova) "instance.availability_zone reports wrong az after live migration" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)22:31
mriedemso if you don't specify an az during server create, you can freely move across azs,22:32
mriedembut if you do specify an az during server create, you must be restricted to that az during moves22:32
melwittright22:32
edmondswthat's what I'd expect22:32
mriedemfor live migration if you don't specify an az on create and are live migrated to another one, we aren't updating the instance az22:32
mriedemwhich is that bug fix22:32
*** felipemonteiro_ has joined #openstack-nova22:32
melwittk, that makes sense22:33
*** felipemonteiro__ has joined #openstack-nova22:34
*** shaohe_feng has quit IRC22:34
*** shaohe_feng has joined #openstack-nova22:35
mriedemedmondsw: ok updated comments in your bug22:36
edmondswmriedem tx22:36
mriedemhmm,22:37
*** hongbin has quit IRC22:37
mriedemso we could put the az on the Selection object for a given host while we're in the controller and pass that down to the cell,22:37
mriedembut there would be a window of time where the host could be moved into / out of an aggregate / AZ during the reschedule loop22:37
*** yassine has joined #openstack-nova22:37
*** felipemonteiro_ has quit IRC22:37
mriedemnot likely, and kind of tough luck if that happens...22:37
*** yassine is now known as Guest4428922:38
mriedemthis might also help us resolve up-call #5 at https://docs.openstack.org/nova/latest/user/cellsv2-layout.html#operations-requiring-upcalls22:38
mriedemwhen [cinder]/cross_az_attach=False22:38
mriedembecause the compute could get the host az off the Selection object22:38
*** owalsh_ has joined #openstack-nova22:39
mriedemwe really need a way to do split mq fixturization in our functional testes22:41
mriedem*tests22:41
edmondswwhy does the AZ have to be set if the user didn't specify one? I assume it's to fill that on a later GET, but couldn't it just be populated dynamically there?22:41
mriedemhttps://review.openstack.org/#/c/446053/22:41
edmondswso that's what caused it22:43
mriedemyes,22:43
mriedemi believe the reason is,22:43
mriedemset the az on the instance at the top where we can reach aggregates in the api db,22:43
mriedemso we don't need to do that in the computes22:43
*** owalsh has quit IRC22:43
mriedemwhere we can't reach the api db22:43
mriedembut dansmith would have to confirm that22:43
mriedem"which isn't as useful to the things that need to consider what AZ the instance is actually in (without doing all the aggregate math to determine it on the fly)."22:44
mriedemthat reads "up call" to me ^22:44
edmondswbut what needs to consider the AZ if the user didn't specify?22:44
edmondswe.g. we just said migrate wouldn't care about the AZ in that case22:44
*** shaohe_feng has quit IRC22:45
mriedemnot sure off the top of my head honestly22:45
*** yamamoto has joined #openstack-nova22:45
*** shaohe_feng has joined #openstack-nova22:45
mriedemmaybe [cinder]/cross_az_attach=False ?22:46
mriedembut as far as i know that is still broken22:47
*** owalsh has joined #openstack-nova22:47
*** owalsh_ has quit IRC22:48
mriedemugh, so we have a cache in nova.availability_zones,22:49
mriedembut we don't populate it when calling get_host_availability_zone at the top22:49
mriedemso when we get to the compute and check the instance.az (set at the top) against the host az from the cache, the latter is None and we say, 'the cache is invalid, so fetch the host az again'22:49
edmondswmriedem, actually git blame shows this was introduced with https://github.com/openstack/nova/commit/03b4c67b22f49d325386bc3ebd2ade79b44fa69922:49
mriedemwhich is the up-call that fails22:49
mriedemedmondsw: that's the same change22:50
*** TxGirlGeek has quit IRC22:50
mriedemoh but the cache on the compute won't have the az set in it anyway22:50
edmondswoh, dang it it is... I saw "Updated 9 weeks ago" in that review and thought it was new, whereas the commit was old, and assumed they were different22:50
mriedemb/c we get the host az at in a different process22:51
mriedemunless you have a distributed  cache...22:51
*** yamamoto has quit IRC22:51
*** shaohe_feng has quit IRC22:55
*** TxGirlGeek has joined #openstack-nova22:56
*** shaohe_feng has joined #openstack-nova22:57
*** TxGirlGeek has quit IRC22:58
*** harlowja has quit IRC23:05
*** shaohe_feng has quit IRC23:05
*** shaohe_feng has joined #openstack-nova23:06
*** antosh_ has quit IRC23:06
*** felipemonteiro__ has quit IRC23:08
*** edmondsw has quit IRC23:12
*** edmondsw has joined #openstack-nova23:13
mriedemedmondsw: fyi, your bug is also a problem for resize reschedules23:14
*** shaohe_feng has quit IRC23:15
*** shaohe_feng has joined #openstack-nova23:16
*** edmondsw has quit IRC23:17
openstackgerritMatt Riedemann proposed openstack/nova master: Add another up-call to the cells v2 caveats list  https://review.openstack.org/58191023:23
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (6)  https://review.openstack.org/57411323:24
oomichitakashin: hi, can you check my comment on https://review.openstack.org/#/c/570842 ?23:24
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (7)  https://review.openstack.org/57497423:25
takashinoomichi: okay. Thanks.23:25
*** shaohe_feng has quit IRC23:26
mriedemha, just found another one23:26
mriedemduring a resize reschedule, compute casts to cell conductor's migrate_server method,23:27
mriedemwhich has the targets_cell decorator on it which gets an instance mapping,23:27
mriedemif you don't have access to the api db, kablammo23:27
mriedemhttp://logs.openstack.org/27/581727/1/check/tempest-full-py3/15d7fdc/controller/logs/screen-n-cpu.txt#_Jul_11_13_32_54_82299623:27
*** shaohe_feng has joined #openstack-nova23:27
* melwitt keels over23:27
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (5)  https://review.openstack.org/57084223:29
*** chyka has quit IRC23:30
mriedemhttps://bugs.launchpad.net/nova/+bug/178130023:31
openstackLaunchpad bug 1781300 in OpenStack Compute (nova) "resize reschedule results in CantStartEngineError during up-call to InstanceMappings table" [Medium,Triaged]23:31
*** shaohe_feng has quit IRC23:36
*** shaohe_feng has joined #openstack-nova23:36
*** shaohe_feng has quit IRC23:46
*** yamamoto has joined #openstack-nova23:47
*** med_ has joined #openstack-nova23:47
*** shaohe_feng has joined #openstack-nova23:47
*** gyee has quit IRC23:48
*** yamamoto has quit IRC23:52
*** shaohe_feng has quit IRC23:56
openstackgerritMatt Riedemann proposed openstack/nova master: Short-circuit targets_cell if already targeted  https://review.openstack.org/58191223:57
mriedemmelwitt: dansmith: ^23:57
*** shaohe_feng has joined #openstack-nova23:57
openstackgerritMatt Riedemann proposed openstack/nova master: Short-circuit targets_cell if already targeted  https://review.openstack.org/58191223:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!