Tuesday, 2023-12-05

opendevreviewmelanie witt proposed openstack/nova master: Use split kernel/initramfs Cirros UEC image by default in jobs  https://review.opendev.org/c/openstack/nova/+/90221703:37
opendevreviewmelanie witt proposed openstack/nova master: Use split kernel/initramfs Cirros UEC image by default in jobs  https://review.opendev.org/c/openstack/nova/+/90221707:25
sean-k-mooney[m]gibi:  melwitt can either of ye take a look at https://review.opendev.org/c/openstack/nova/+/88368208:47
sean-k-mooney[m]its too late to pull that into our next downstream release but it would still be nice to get that fixed before the end of the year if we can08:48
sean-k-mooney[m]melwitt:  oh i see you have some comments since i last reviewd ill read over them now08:50
sean-k-mooney[m]ok dropped back to +1 based on your comments09:03
sean-k-mooney[m]bauzas:  can you review this functional repoducer test quickly. gibi and i are +2 on the actual fix https://review.opendev.org/c/openstack/nova/+/891290/1  gibi helped write the repoducer so we need another core to review 09:45
opendevreviewMerged openstack/nova stable/2023.1: Do not manage CPU0's state  https://review.opendev.org/c/openstack/nova/+/90156710:11
opendevreviewMerged openstack/nova stable/2023.1: Allow enabling cpu_power_management with 0 dedicated CPUs  https://review.opendev.org/c/openstack/nova/+/90166010:31
bauzassean-k-mooney: sorry I was off due to some issue with my car, I'm back now10:36
stephenfinJayF: If it's okay, I'd rather someone else take that. I think it's reasonably complete10:40
sean-k-mooneybauzas: no worries10:45
bauzasI needed to go to the policy just because the folk that hitted my car with his skis wasn't replied to me :(10:46
bauzasreplying*10:46
bauzaspolice*10:47
bauzasshit, I need to remember how to run a specific stestr worker 10:58
* bauzas goes back at https://stestr.readthedocs.io/en/latest/MANUAL.html10:59
bauzasyeah found $ stestr last --subunit | subunit-filter -s --xfail --with-tag=worker-3 | subunit-ls > slave-3.list11:01
bauzashah11:09
bauzasonce you have a list of tests from the same worker, stestr run --analyze-isolation does the job for you 11:10
bauzasnaïce11:10
bauzashttps://paste.opendev.org/show/bDpwRC0njzsbM1PmxTKr/11:10
bauzas\o/11:10
bauzasnot sure auniyal6 and Uggla know about this ^11:11
dvo-plvHello, sean-k-mooney gibi your comments for glance were resolved and zull verification pass successful11:15
dvo-plvhttps://review.opendev.org/c/openstack/glance/+/88213311:15
*** tobias-urdin-pto is now known as tobias-urdin11:15
elodillesbauzas: fyi, i need to take off the rest of the day, so i'm almost certain that i won't be around at the nova meeting :S i'll update the stable release patches when i'll be back so that we can release them finally11:21
*** elodilles is now known as elodilles_pto11:21
bauzas++11:23
sean-k-mooneydvo-plv: cool just bare in mind gibi and i dont have core rights on glance11:24
sean-k-mooneybut ill re review and add my +1 so they know nova are fine with the patch11:24
dvo-plvokay, thanks, I saw Abhishek in the reviewers, but I need one more person, whom I should to ask for review ?11:25
sean-k-mooneyi added dansmith to the review list but im not sure if they are on PTO for the rest of they year yet. i think tehy were here this week but not sure11:26
dvo-plvI see, thank you11:27
opendevreviewMerged openstack/nova master: Remove workaround for old python-ironicclient  https://review.opendev.org/c/openstack/nova/+/82324711:38
opendevreviewMerged openstack/nova master: Add retry on attachment_update  https://review.opendev.org/c/openstack/nova/+/90181511:38
opendevreviewMerged openstack/nova master: [libvirt]Add migration_inbound_addr  https://review.opendev.org/c/openstack/nova/+/90020311:43
opendevreviewMerged openstack/nova master: Reproduce bug #2025480 in a functional test  https://review.opendev.org/c/openstack/nova/+/89129011:43
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for add/remove instance info from node  https://review.opendev.org/c/openstack/nova/+/65969112:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for getting network metadata from node  https://review.opendev.org/c/openstack/nova/+/67021312:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for volume_target.*  https://review.opendev.org/c/openstack/nova/+/86779012:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for node.set_provision_state  https://review.opendev.org/c/openstack/nova/+/86779112:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for node.set_power_state  https://review.opendev.org/c/openstack/nova/+/86779212:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for node.vif*  https://review.opendev.org/c/openstack/nova/+/86779312:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for remaining network operations  https://review.opendev.org/c/openstack/nova/+/86779412:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for remaining ironic driver calls  https://review.opendev.org/c/openstack/nova/+/86779512:10
opendevreviewStephen Finucane proposed openstack/nova master: Use SDK for deprecated baremetal proxy API  https://review.opendev.org/c/openstack/nova/+/86779612:10
opendevreviewStephen Finucane proposed openstack/nova master: Remove Ironic client wrapper  https://review.opendev.org/c/openstack/nova/+/86779712:10
opendevreviewStephen Finucane proposed openstack/nova master: Remove unnecessary aliasing  https://review.opendev.org/c/openstack/nova/+/86779812:10
opendevreviewStephen Finucane proposed openstack/nova master: Use real SDK objects in tests  https://review.opendev.org/c/openstack/nova/+/86779912:10
stephenfinJayF: Actually, it was a trivial enough rebase12:10
stephenfinI thought it would have rotted more12:10
stephenfinsean-k-mooney: Could you look at https://review.opendev.org/c/openstack/nova/+/896227/ and https://review.opendev.org/c/openstack/nova/+/896228/ again, when you get a chance?12:29
sean-k-mooneyi just got the email pings so yes12:29
stephenfinta12:31
sean-k-mooneydone, im happy with both of those and all the commits below12:38
opendevreviewMerged openstack/nova master: Do not untrack resources of a server being unshelved  https://review.opendev.org/c/openstack/nova/+/89040612:56
opendevreviewMerged openstack/nova master: doc: Remove crud from conf.py file  https://review.opendev.org/c/openstack/nova/+/89622412:56
opendevreviewStephen Finucane proposed openstack/nova master: Remove nova.wsgi module  https://review.opendev.org/c/openstack/nova/+/90268614:48
opendevreviewStephen Finucane proposed openstack/nova master: Add new nova.wsgi module  https://review.opendev.org/c/openstack/nova/+/90268714:48
opendevreviewStephen Finucane proposed openstack/nova master: setup: Remove pbr's wsgi_scripts  https://review.opendev.org/c/openstack/nova/+/90268814:48
stephenfinsean-k-mooney: I think that's where we are now ^14:48
sean-k-mooneyi started looking at makeing pbr work but i didnt have time to actully fix the wsgi_sripts functionatly14:51
sean-k-mooneybut im ok with also takeign the approch of avoiding the extion if others are14:52
sean-k-mooneyill try an review them again shortly14:52
sean-k-mooneyi know rougly where to make the change just havnt figureoud out exactly how too.14:52
opendevreviewMerged openstack/nova master: pre-commit: Update plugin versions  https://review.opendev.org/c/openstack/nova/+/89622614:53
sahido/ quick question guys14:54
sahidregarding https://review.opendev.org/c/openstack/nova/+/89651214:54
sahidI'm glad that may consider it14:54
sahidjust, naything else is needed?14:54
opendevreviewMerged openstack/nova master: pre-commit: Use native hacking pre-commit hook  https://review.opendev.org/c/openstack/nova/+/89700514:55
sean-k-mooneysahid: procedurelly jsut need to be approved in the team meeting later14:56
sean-k-mooneyi have withdrawn my objections so im fine with tracking this as a specless blueprint14:56
jkulikon the topic of WSGI: there's this bug open https://bugs.launchpad.net/nova/+bug/1930406 which talks about 2 things: 1) dropping the rpc-call to nova-compute on volume-attach to reserve_block_device_name() and 2) running Nova with threads instead of eventlet - where could I look for the state/updates on these topics?15:06
sean-k-mooneyjkulik: im not sure that anyone is working on them currecnly i will remove lees name form it as they nolonger work on openstack15:11
sean-k-mooneythreads>1 is not somethign we are likely to supprot going forward15:12
sean-k-mooneyat least not in the short term15:13
sean-k-mooneywe may howeveer do that indirecly fi we end up removing our use of eventlets entirly for the nova-api when run under wsgi15:13
sean-k-mooneyi know melwitt had partly looked at changign to using treadpools at one point15:13
sean-k-mooneyi cant recall if we merged that15:13
jkulikok. maybe I can find a review-request for that. I'll take a look15:14
jkulikthank you15:14
sahidsean-k-mooney: ack thank you15:16
sean-k-mooneyjkulik: https://review.opendev.org/c/openstack/nova/+/65017215:16
sahiddansmith: regarding your reviews on it, thanks a lot. There are some that I'm not sure about, I have kept them as unrosolved, if you can have a look15:17
sean-k-mooneyjkulik: i dont reacall if there was anything preventing that form movign forword other then the fact i and dansmith have been busy and melwitt hasnt pinged use to review15:17
sean-k-mooneybut the last patch passed ci 15:17
sean-k-mooneyso perhaps we can move forward with that.15:17
jkuliksounds good. I'll read through it, too.15:18
bauzasgibi: given my next internal meeting, I can't really do the upstream meeting15:19
sean-k-mooneyi just rebased https://review.opendev.org/c/openstack/devstack/+/893302 to get new results too15:19
bauzasgibi: could you please chair it ?15:20
jkuliksean-k-mooney: FYI, I found an abandoned spec https://review.opendev.org/c/openstack/nova-specs/+/765097 for "Remove device from attach_volume request response"15:28
sean-k-mooneyjkulik: ya so this feel off our radar since we dint have anyoen working on it15:30
jkulikyeah, I'll put it into the bug report to maybe re-propose it later on15:30
bauzasman the global CONF object is killing me15:55
bauzasif some test runs before some other, then a group is persistent between them15:56
bauzas#startmeeting nova16:00
opendevmeetMeeting started Tue Dec  5 16:00:53 2023 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
opendevmeetThe meeting name has been set to 'nova'16:00
bauzasas I said earlier, I have a lot of things to do now16:01
bauzasso, please, let's do a very short meeting (like 10 mins)16:01
bauzas#link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting16:01
bauzasI can start16:02
bauzas#topic Bugs (stuck/critical) 16:02
bauzas#info No Critical bug16:02
bauzas#link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 38 new untriaged bugs (+0 since the last meeting)16:02
bauzas#info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster16:02
bauzas#info bug baton is gibi16:02
bauzaslet's move on16:02
bauzas#topic Gate status 16:02
bauzas#link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:02
bauzas#link https://etherpad.opendev.org/p/nova-ci-failures-minimal16:02
bauzasthanks to melwitt ^16:03
bauzas#link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status16:03
bauzasperiodics are green for the master branch16:03
bauzas#info Please look at the gate failures and file a bug report with the gate-failure tag.16:03
bauzasmoving on again16:03
bauzas#topic Release Planning 16:03
bauzas#link https://releases.openstack.org/caracal/schedule.html#nova16:03
bauzas#info Caracal-2 (and spec freeze) milestone in 5 weeks16:03
bauzasgiven both of some personal issues this morning + some hard work on bugfixes, I forgot about our today's spec freeze day16:04
bauzasso,16:04
bauzas#info Spec review day tomorrow16:04
bauzas#action bauzas to send an email accordingly16:04
* gibi joining late16:04
bauzasfor the records, next cycle, I'll propose our review days to be on Wednesdays16:04
bauzaseasier to not forget about them16:04
bauzasand my personal action is to add the ical in my gcal16:05
bauzasmoving on16:05
bauzas#topic Review priorities 16:05
bauzas#link https://etherpad.opendev.org/p/nova-caracal-status16:05
bauzasI've moved two bugfixes series ^16:05
bauzasyou can also review them16:05
bauzasI'll lookup the feature series once I have time16:05
bauzas(hopefully before 2024)16:06
bauzas#topic Stable Branches 16:06
bauzaselodilles_pto is on pto16:06
* gibi taking the bug baton without promises16:06
bauzascopying the notes16:06
bauzas#info stable gates don't seem blocked16:06
bauzas#info stable release patches still open for review: https://review.opendev.org/q/project:openstack/releases+is:open+intopic:nova16:06
bauzas+1: yoga is going to be unmaintained, so final stable/yoga release should happen ASAP - https://etherpad.opendev.org/p/nova-stable-yoga-eom16:07
bauzas#info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci16:07
bauzasI'll ask elodilles_pto about yoga tomrrow16:07
bauzasfwiw, I +1d the antelope release patch with a note16:07
bauzasasking whether we need a .y version16:07
bauzasmoving on16:07
bauzas#topic vmwareapi 3rd-party CI efforts Highlights 16:07
bauzascopying the notes too16:07
Ugglao/16:08
bauzas#Info A productive 768GB host reserved for CI and "outer"  hypervisor has been being installed. Test VMs are going to be nested.  Main challenge: Network setup for nested hypervisor, testing in lab  environment.16:08
bauzasbravo fwiesel16:08
bauzasand thanks for the update16:08
bauzaslast topic16:08
bauzas#topic Open discussion 16:08
bauzas(artom) Specless blueprint for persistent mdevs 16:08
bauzas#link https://blueprints.launchpad.net/nova/+spec/persistent-mdevs16:08
bauzasno upgrade concerns I can see16:09
bauzasthe libvirt version needed for this is older than Antelope, but I need to doublecheck16:09
bauzasgiven the lack of quorum, I propose to punt the approval until next week16:10
bauzasand hopefully, I'd have more time to doublecheck the upgrade concerns16:10
JayFI have a couple items for open discussion, if that's OK?16:11
bauzaslike I said at the beginning I have a very constrained agenda for today16:12
bauzashence the very short meeting16:12
bauzasso either other folks are joining or we punt that to next week16:12
JayFWell, one is a question can be fielded outside, the other is a specless blueprint you'd likely punt for quorum anyway, so we can hit it next week.16:12
bauzasJayF: please add it in https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 16:13
bauzasso we won't forget :)16:13
bauzas(last open discussion topic)16:13
JayFI will once you clear it for this one :D 16:14
bauzasokay, thanks folks16:15
bauzasthat was a short one16:15
bauzassorry for the rush16:15
bauzas#endmeeting16:15
opendevmeetMeeting ended Tue Dec  5 16:15:31 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:15
opendevmeetMinutes:        https://meetings.opendev.org/meetings/nova/2023/nova.2023-12-05-16.00.html16:15
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/nova/2023/nova.2023-12-05-16.00.txt16:15
opendevmeetLog:            https://meetings.opendev.org/meetings/nova/2023/nova.2023-12-05-16.00.log.html16:15
JayFbauzas: the question was, do I need to resubmit https://specs.openstack.org/openstack/nova-specs/specs/2023.2/approved/ironic-shards.html for caracal; I think I answered my question as "yes" 16:15
JayFand am working on that right now16:15
bauzasJayF: please do indeed16:15
fwieselthanks, bauzas16:15
opendevreviewJay Faulkner proposed openstack/nova-specs master: Re-submit Ironic-shards for Caracal  https://review.opendev.org/c/openstack/nova-specs/+/90269816:17
bauzasJayF: right in time for the spec review day tomorrow16:19
JayFthat's what I saw that gave me the moment of "I need to do this" :D16:21
JayFI'm going to go review all of stephen's changes to put SDK support in nova<>ironic driver for now though16:21
JayFtrying to get that in first... remove the footgun that caused the failure last cycle rather than just charging forward16:21
bauzasthe global CONF object is getting me nuts16:22
bauzashow the fuck can I ensure that it's cleaned up in my test ?16:22
bauzasI fucking need to reset the whole blob16:23
sean-k-mooneybauzas: we have a fixture for that16:30
bauzasohohoh16:31
bauzasmy brain fscked16:31
bauzasI remember we have a solution but I forgot the pattern16:31
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/test.py#L214-L21916:32
sean-k-mooneyits done in the base test case16:32
bauzascontext : https://review.opendev.org/c/openstack/nova/+/902084/3/nova/tests/unit/virt/libvirt/test_driver.py#26492 fails because the CONF object is persistent across tests16:32
sean-k-mooneythe base nova one16:32
sean-k-mooney it  shoudl not be unless you did somethign odd16:32
bauzasonce the dynamic group is registered in some test, it's kept on the global CONF obj16:32
bauzassean-k-mooney: I can prove it16:33
sean-k-mooneyoh well i guess then the fixture is incomplete16:33
sean-k-mooneyand needs to be updated to supprot that16:33
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/test.py#L21916:33
sean-k-mooneywe do use the fixture in any test case that inherits form nova.test.TestCase16:34
sean-k-mooneyits defiend here https://github.com/openstack/nova/blob/master/nova/tests/fixtures/conf.py#L2616:34
bauzasI see it 16:35
sean-k-mooneyhttps://github.com/openstack/nova/blob/master/nova/tests/fixtures/conf.py#L6916:35
sean-k-mooneythat is likely relevent16:35
bauzasthis is normal if you did set the flags before16:35
sean-k-mooneythe flags are not ment to exscape the current test case16:36
bauzaswhat's not normal is that I still have the groups registered in some test before I set them16:36
bauzasanyway, I need to prepare some internal meetup16:36
bauzasso I'll have to hardstop16:36
sean-k-mooneyok16:36
opendevreviewMerged openstack/nova stable/2023.1: Translate VF network capabilities to port binding  https://review.opendev.org/c/openstack/nova/+/89894516:49
melwittsean-k-mooney: https://review.opendev.org/c/openstack/nova/+/650172 is ready for review iirc, I don't remember if there are unaddressed comments so if there are, just lmk17:05
melwittsean-k-mooney: and the nova patch with uec images passed finally! https://review.opendev.org/c/openstack/nova/+/90221717:06
sean-k-mooneymelwitt: woot17:57
sean-k-mooneyi noticed you working on it quite late last night considering it was early in my morning :)17:58
melwittsean-k-mooney: the patch was so close to working 😁18:16
clarkbmelwitt: fwiw we should have 30 day retention of job logs in our swift backends18:16
clarkbthat range originated with our old 10TB fileserver for logs. That was basically as much as we could fit at the time into that space with some headroom for maintenance and we carried it over when we moved to swift18:17
melwittclarkb: ah ok. good to know. I always use "last 30 days" but I feel like I haven't seen results that go back further than around 10 days. I'll check again18:18
clarkbhttps://zuul.opendev.org/t/zuul/build/19dae94a70524ae28db9cd8ed4257136/logs these are almost 30 days old18:19
melwitthrrmm ok, thanks18:21
melwittdpawlik: do you know if there is a time limit of some kind on the opensearch log indexing? I generally see around 10 days worth of search results but the logs are actually kept for 30 days? ^18:31
clarkboh you are talking about retention in opensearch18:35
clarkbin the old elasticsearch system we had between 7-10 days of retention due to indexing the data causing it to explode in size and that is after filtering out the debug logs18:35
melwittyes, sorry, I thought your comment was in reference to my ML post about the gate issues in nova and my mention of 10 days worth of opensearch results :)18:39
clarkbit was and I misread it to say you are giving opensearch links because the static log storage doesn't have more than 10 days of retention18:43
melwittahhh gotcha18:43
sean-k-mooneythe static log storage used to be 30 in the past right18:44
sean-k-mooneythen it was 30 days subject to storage not being full18:44
sean-k-mooneywhen did it drop to 10 days for zuul logs out of interest18:44
clarkbsean-k-mooney: when we moved to swift it has always been 30 days since we weren't limited by filesystems aynmore. The 10 day thing is opensearch specific I think18:45
melwittthe opensearch share links were sometimes not working at all, for lack of a better way to describe it. I would copy paste or click the share icon and get the link, then to test it I pasted it into a new browser tab and after the first visit, the link no longer worked and would not bring up the search results and would just load the page with no search query at all18:45
sean-k-mooneyah ok18:46
sean-k-mooneymelwitt: i alwasy put the opensarch link thorugh tiny url by the way18:46
sean-k-mooneyhttps://tinyurl.com/app18:46
melwittI'll try that. I tried the built in url shortener in opensearch but it doesn't work or isn't enabled or something and I didn't go to another one18:47
melwittgmann: you might be interested in these, some patches intended to improve our CI stability in nova https://review.opendev.org/q/topic:%22nova-ci-improvements%22 one patch is for tempest to support UEC images18:58
gmannmelwitt: ack, thanks19:13
gmannmelwitt: commented in https://review.opendev.org/c/openstack/nova/+/90221720:04
gmannmelwitt: also I did not get how tempest change is dependency of it20:04
gmannmelwitt: btw I like the idea of using UEC image, maybe we can try in a few of the tempest jobs also20:05
melwittgmann: thanks for looking! the tempest change is needed bc there are a couple of places where tempest creates an image itself, and those couple of places currently don't support UEC images. without the tempest change, when tempest creates an image based on the default image, it doesn't set the kernel and ramdisk image IDs in the create request that are needed by the AMI image, so when it tries to create a server from that image, it22:16
melwitt won't work. the server won't boot. similarly when tempest creates a fresh image from the tarball, currently it can't support that, it won't extract the tarball and create the kernel and ramdisk images in glance, and associate the AMI image it creates with them22:16
gmannmelwitt: ah i see, tempest uses img_file in create, got it thanks https://review.opendev.org/c/openstack/tempest/+/902609/3/tempest/scenario/manager.py#77122:24
gmannmelwitt: +2 on tempest change, on nova change did you see my comment about CIRROS_VERSION 22:25
gmann setting https://review.opendev.org/c/openstack/nova/+/902217/9/.zuul.yaml#51222:25
gmannmelwitt: as it has one +2 and work in current devstack default. I am ok to convert my vote and ok to do it 22:25
gmann that in follow up22:25
melwittgmann: yes, just replied. you are right about not needing to set those default image vars for nova-next. setting those is residue from my original PS that was setting defaults in project.vars and trying to inherit from that. that ultimately couldn't work bc devstack-tempest sets IMAGE_URLS and overrides project.vars, so I had to change it around22:27
melwittgmann: ok sure, sounds good. I'll upload a follow up22:28
gmannmelwitt: thanks, +A22:31
opendevreviewmelanie witt proposed openstack/nova master: Remove redundant setting of DEFAULT_IMAGE_NAME(_FILE)  https://review.opendev.org/c/openstack/nova/+/90273422:35
melwitt^ thanks gmann 22:36

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!