| kashyap | dansmith: I was off yesterday (public holiday). Looking at the scrollback now about "tempest.api.compute.admin.test_volumes_negative.VolumesAdminNegativeTest" | 08:07 |
|---|---|---|
| opendevreview | Elod Illes proposed openstack/nova master: Add nova-tox-functional-py310 to gate jobs https://review.opendev.org/c/openstack/nova/+/881339 | 09:38 |
| sfinucan | bauzas: addressed your comments on https://review.opendev.org/c/openstack/python-openstackclient/+/881822 lemme know if they work | 11:19 |
| sean-k-mooney | sfinucan: im ok wiht osc generating keypairs client side | 11:57 |
| sean-k-mooney | althoug it proably should be in its own command but i understand why we might wasnt to proxy it form the nova one | 11:58 |
| sean-k-mooney | sfinucan: can i suggest we do both. proceed with bauzas patch as is and then add your change on top to add client side generation | 11:59 |
| sean-k-mooney | the keypair type would be changing so they are not really interchangable | 11:59 |
| sean-k-mooney | they kind of are but old operating systsms wont supprot ssh-ed25519 | 12:00 |
| sean-k-mooney | i.e. centos 7 and newer OSs wont support RSA i.e. centos 9 | 12:00 |
| sean-k-mooney | so i see you are proposing taking the approch of always generating htem client side isntaed fo doing this based on the microversion | 12:02 |
| sean-k-mooney | that actully has some benifits in that the behaivor wont be "incorrect" for the new microverion | 12:03 |
| sean-k-mooney | i.e. the client wont be giving the perception of the api generating the key for the latest microversion | 12:03 |
| sean-k-mooney | since now it will be alwasy client side. | 12:04 |
| sfinucan | sean-k-mooney: yeah, I'd rather not have different behaviour depending on the microversion in use | 12:56 |
| bauzas | sfinucan: sean-k-mooney: I'm OK with generating private keys by default in OSC even if the API microversion is older than 2.92 | 12:58 |
| bauzas | since the API was able to provide a private key by passing it, I'm ok | 12:58 |
| sean-k-mooney | ok | 13:03 |
| sean-k-mooney | so how do you want to proceed | 13:03 |
| sean-k-mooney | just go with sfinucan patch | 13:03 |
| sean-k-mooney | or merge both? | 13:03 |
| sfinucan | I think it's a case of one or the other | 13:04 |
| *** sfinucan is now known as stephenfin | 13:04 | |
| stephenfin | sean-k-mooney: if we start doing stuff client-side, there's no reason not to do it for all microversions. If we stick server-side, we're obliged to drop key generation functionality for newer or all microversions | 13:06 |
| stephenfin | sean-k-mooney: bauzas: btw, do you have +2 on OSC now? You should? | 13:11 |
| bauzas | stephenfin: my only concern if we merge your patch is that enduser wouldn' longer see that they need to create their own keypairs | 13:12 |
| bauzas | stephenfin: so if they use openstacksdk directly, they would see then | 13:13 |
| sean-k-mooney | i can check | 13:13 |
| stephenfin | wdym? They don't. We're still doing it for them, only on the client rather than the server | 13:13 |
| stephenfin | Ah | 13:13 |
| stephenfin | I mean, I think that's okay. We do a whole load of helpful things in OSC that don't happen in SDK | 13:14 |
| stephenfin | Like the 'server ssh' command. There's no equivalent for that in SDK, mainly because it's not needed. We could add a docstring in SDK noting that the private key field must be explicitly provided in newer microversions | 13:15 |
| sean-k-mooney | stephenfin: for osc i do not have +2 currently | 13:18 |
| sean-k-mooney | likely the same for sdk i have not actuly looked | 13:18 |
| bauzas | ditto, I'm not osc-core (and not sure I want it :) ) | 13:26 |
| dansmith | kashyap: thanks for looking..that looks like a kernel bug to me and so it'd be good to know if it's been fixed as it's not uncommon to see it | 13:27 |
| stephenfin | Ah, okay, it seems I'd misunderstood what 'python-openstackclient-service-core' was being used for. I'd thought it would contain all the service core groups by default. Evidently not. | 13:27 |
| stephenfin | bauzas: https://review.opendev.org/c/openstack/openstacksdk/+/881965 | 13:28 |
| kashyap | dansmith: Yeah, it looks like "somehow" the loading of the kernel modules itself is failing: "failed loading these modules: nls_ascii nls_iso8859-1 nls_utf8 ip_tables ahci" | 13:29 |
| kashyap | Also, I hate to say this, but I wonder if it's CirrOS-specific | 13:30 |
| kashyap | dansmith: I take it that this segfault is consistently reproducible | 13:30 |
| dansmith | kashyap: I can't push a button and make it happen, but I see it fairly often | 13:31 |
| kashyap | dansmith: Noted; I'm trying to wade through the logs to also find the QEMU command-line of this. And the host kernel version | 13:32 |
| dansmith | kashyap: entirely possible that it's cirros related, but since this there is not much going on in userland and this is a kernel crash, I imagine it's not the actual fault of cirros | 13:32 |
| kashyap | Have you got the instance name / UUID, per chance? | 13:32 |
| dansmith | kashyap: if you look right above the top of the console dump you'll see requests tempest was making to watch the status and you should find the uuid there | 13:35 |
| kashyap | Okido, this seems the UUID: e0dc4c98-95e2-4421-a6ba-ae07213914a7 | 13:35 |
| kashyap | Thanks! | 13:36 |
| * dansmith teaches a man to fish | 13:40 | |
| dansmith | :) | 13:40 |
| dansmith | bauzas: FYI, I've got a few good passing runs on the nova patch with new ceph on jammy, after *much* work on the ceph devstack plugin, cinder tempest plugin, and tempest itself | 13:53 |
| dansmith | so I'm hoping we can start to land those things soon and flip that job over, with your blessing | 13:53 |
| opendevreview | yatin proposed openstack/nova master: Add config option to configure TB cache size https://review.opendev.org/c/openstack/nova/+/868419 | 13:53 |
| dansmith | but let me know if you want to examine results closely | 13:53 |
| bauzas | dansmith: oh that's excellent news, I got distracted from following the series | 14:03 |
| dansmith | gouthamr: I think I need to make the devtack-ceph-plugin dependent on the cinder test plugin, which is dependent on the tempest stack.. are you okay with that? and do you want me to do anything else to the devstack plugin before we merge? or just merge and clean it up after? | 14:03 |
| dansmith | bauzas: not expecting you to follow it, just giving the boss the executive summary :) | 14:03 |
| bauzas | dansmith: don't call me like this, man, I'm just a cat herder :) | 14:03 |
| dansmith | heh | 14:04 |
| bauzas | dansmith: so, what's the plan ? merging the tempest bits first, and then the ceph devstack plugin one ? | 14:04 |
| dansmith | bauzas: well, I was just saying above that I think it needs to be devstack-plugin -> cinder tempest -> tempest (so they merge in reverse order of this) | 14:05 |
| bauzas | sorry missed your line | 14:06 |
| dansmith | and probably change my DNM nova patch to stop commenting out all the jobs to make sure they're all still happy, other than just the ceph one | 14:06 |
| bauzas | yeah | 14:06 |
| bauzas | dansmith: at least that's a good thing to tell in the nova meeting today :) | 14:08 |
| dansmith | yeah | 14:08 |
| sean-k-mooney | dansmith: by the way i pushed this yesterday https://review.opendev.org/c/openstack/nova/+/881912 the image it pulled does not have resize2fs installed | 14:18 |
| sean-k-mooney | so that is why some of the tests failed | 14:19 |
| dansmith | sean-k-mooney: because it can't resize itself to fill the root on boot? | 14:19 |
| sean-k-mooney | yep | 14:20 |
| sean-k-mooney | i can fix that pretty simply i need to look and see if any other packages are missing | 14:20 |
| dansmith | ack, cool | 14:21 |
| sean-k-mooney | in at least some of the case the inablity to resize the root fs prevent the ssh hostkeys from being genreated which prevented ssh access to the vm | 14:27 |
| dansmith | it'll prevent other things too, like writing the timestamp files | 14:28 |
| dansmith | (assuming it's really that constrained) | 14:28 |
| sean-k-mooney | virtual size: 292 MiB (305659904 bytes) | 14:29 |
| sean-k-mooney | disk size: 83 MiB | 14:30 |
| opendevreview | Fabian Wiesel proposed openstack/nova master: Add more password generation options https://review.opendev.org/c/openstack/nova/+/865669 | 14:50 |
| bauzas | reminder : nova meeting in one hour-ish | 15:03 |
| enriquetaso | thanks bauzas | 15:04 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/868419 is almost ready to merge just one last issues to adress if people want to review | 15:16 |
| *** Guest74 is now known as atmark | 15:22 | |
| bauzas | sean-k-mooney: I plan to review this feature by tomorrow | 15:41 |
| bauzas | (the tb cache size one) | 15:41 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Revert "Debug Nova APIs call failures" https://review.opendev.org/c/openstack/nova/+/882052 | 15:51 |
| bauzas | dansmith: melwitt: you may appreciate a quick review of the logging leak ^ | 15:56 |
| bauzas | (did it on purpose before the meeting) | 15:56 |
| bauzas | #startmeeting nova | 16:00 |
| opendevmeet | Meeting started Tue May 2 16:00:02 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'nova' | 16:00 |
| bauzas | welcome everyone | 16:00 |
| elodilles | o/ | 16:00 |
| bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | 16:00 |
| gmann | o/ | 16:00 |
| gibi | o/ | 16:00 |
| gibi | (on a spotty connection) | 16:00 |
| dansmith | bauzas: got it | 16:01 |
| bauzas | coolio, let's start | 16:02 |
| bauzas | #topic Bugs (stuck/critical) | 16:02 |
| bauzas | #info One Critical bug | 16:02 |
| bauzas | #link https://bugs.launchpad.net/nova/+bug/2012993 | 16:02 |
| bauzas | shall be quickly reverted | 16:02 |
| bauzas | and I'll propose backports as soon as it lands | 16:02 |
| bauzas | not sure we need to discuss on it by now | 16:03 |
| bauzas | so we can move on, unless people wanna know more about it | 16:03 |
| sean-k-mooney | i have +w'ed it so its on its way | 16:04 |
| bauzas | yup, and as you wrote, two backports are planned | 16:04 |
| bauzas | down to 2023.1 and Zed | 16:04 |
| bauzas | anyway, moving on | 16:04 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 20 new untriaged bugs (+3 since the last meeting) | 16:04 |
| bauzas | artom: I assume you didn't had a lot of time for spinning around those bugs ? | 16:05 |
| bauzas | again, no worries | 16:05 |
| Uggla | o/ | 16:05 |
| bauzas | #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | 16:05 |
| bauzas | Uggla: you're the next on the list, happy to axe some of the bugs ? | 16:06 |
| artom | bauzas, trying to do a few now | 16:06 |
| bauzas | cool, no rush | 16:06 |
| Uggla | bauzas, yep ok for me. | 16:06 |
| bauzas | thanbks | 16:06 |
| artom | For https://bugs.launchpad.net/nova/+bug/2018172 I think we need kashyap to weigh in, 'virtio' may not be a thing in RHEL 9.1? | 16:06 |
| bauzas | #info bug baton is being passed to Uggla | 16:06 |
| bauzas | artom: looking | 16:06 |
| dansmith | uh, what | 16:07 |
| artom | Or maybe just close immediately and direct them to a RHEL... thing | 16:07 |
| kashyap | artom: Wait; it's impossible "virtio" can't be a thing in RHEL9.1. /me looks | 16:07 |
| dansmith | hah, yeah | 16:07 |
| dansmith | but looks like maybe just a particular virtio video model/ | 16:07 |
| bauzas | that's a libvirt exception | 16:07 |
| bauzas | so, IMHO invalid | 16:07 |
| bauzas | (for the project) | 16:07 |
| artom | Right, but before that I'd like to confirm that Nova is doing the right thing | 16:08 |
| dansmith | is the video model an extra spec/ | 16:08 |
| opendevreview | Rafael Weingartner proposed openstack/nova master: Nova to honor "cross_az_attach" during server(VM) migrations https://review.opendev.org/c/openstack/nova/+/864760 | 16:08 |
| * dansmith curses his ? key | 16:08 | |
| artom | I guess I'm going about it backwards - it's just an easier thing to check initially, RHEL 9.1 support for virtio video | 16:09 |
| artom | dansmith, yeah | 16:09 |
| sean-k-mooney | its an image property | 16:09 |
| bauzas | yeah | 16:09 |
| dansmith | is virtio the only option there or is it like virtio plus a device model name/ | 16:09 |
| sean-k-mooney | virtio shoudl be a thing in rhel9 | 16:09 |
| bauzas | we just try to generate a domain which is badly incorrect given the image metadata | 16:09 |
| dansmith | just wondering if they're specifying the wrong thing | 16:09 |
| sean-k-mooney | technially it maps to virtio-gpu | 16:09 |
| bauzas | that rings a bell to me | 16:10 |
| bauzas | anyway, I'd propose to punt this bug today by asking the reporter its flavor and image | 16:10 |
| opendevreview | Rafael Weingartner proposed openstack/nova master: Nova to honor "cross_az_attach" during server(VM) migrations https://review.opendev.org/c/openstack/nova/+/864760 | 16:10 |
| dansmith | why are we even discussing this on the meeting? | 16:11 |
| bauzas | I guess because artom wanted to triage it | 16:11 |
| bauzas | so please => incomplete it | 16:11 |
| artom | I just wanted to ping kashyap on it, is all | 16:11 |
| sean-k-mooney | i guess it was a bug artom tought whoudl be brougt to our attention based on the triage | 16:11 |
| bauzas | and ask for the image metadata | 16:11 |
| kashyap | artom: We can sort it out off-meeting :) Thx! | 16:12 |
| sean-k-mooney | artom: i have been using hw_video_model=virtio for years so its valid for sure | 16:12 |
| sean-k-mooney | but ya we can move on | 16:12 |
| kashyap | (Yeah; plain "virtio" must work) | 16:12 |
| bauzas | moving on so | 16:13 |
| kashyap | Is it still on-topic to discuss a CirrOS seg-fault that dansmith pointed out earlier? | 16:13 |
| bauzas | #topic Gate status | 16:13 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:13 |
| bauzas | #link https://etherpad.opendev.org/p/nova-ci-failures | 16:13 |
| bauzas | there it is time to discuss gate failures now | 16:14 |
| bauzas | dansmith: shoot the good news and the bad ones if you have | 16:14 |
| dansmith | tldr is that I have got the ceph job moved to jammy and cephadm and it's working finally, and: | 16:14 |
| dansmith | that I found a bunch of places where we thought we were requiring SSHABLE before volume activities where it was being silently ignored | 16:15 |
| dansmith | basically what gibi asserted | 16:15 |
| dansmith | so I've got a stack of tempest patches (and cinder-tempest-plugin) to not only fix those, | 16:15 |
| dansmith | but also sanity check and raise an error if someone asks for SSHABLE but without the requisite extra stuff to actually honor it | 16:15 |
| dansmith | that helps us pass the ceph job in the new config but will also likely help improve volume failures in the other jobs | 16:16 |
| bauzas | cool | 16:16 |
| bauzas | thanks for the janitorisation | 16:16 |
| gibi | dansmith: awesome | 16:16 |
| gibi | thank you | 16:17 |
| bauzas | which patches shall be looked at for people who care ? | 16:17 |
| dansmith | it's been a lot of work and frustration, but happy to see it becoming fruitful | 16:17 |
| dansmith | well, no patches to nova, but I can get you a link, hang on | 16:17 |
| gmann | this one #link https://review.opendev.org/q/topic:sshable-volume-tests | 16:17 |
| dansmith | this stack on tempest: https://review.opendev.org/c/openstack/tempest/+/881925 | 16:18 |
| dansmith | and this one against cinder-tempest https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/881764 | 16:18 |
| dansmith | the nova patch I have up is a DNM just to test our job with that full stack, but we don't need to merge anything | 16:18 |
| bauzas | #link https://review.opendev.org/q/topic:sshable-volume-tests Tempest patches that add more ssh checks for volume-related tests | 16:18 |
| bauzas | cool excellent, thanks a lot dansmith for this hard work | 16:19 |
| * dansmith nods | 16:19 | |
| bauzas | that's noted. | 16:19 |
| bauzas | any other CI failure to relate or mention ? | 16:19 |
| bauzas | looks not, excellent | 16:20 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status | 16:20 |
| bauzas | the fips job run had a node failure, but should be green next week | 16:21 |
| bauzas | on a side note, I'm trying to add some kind of specific vgpu job that would use the mtty sample framework for validating the usage we have on a periodic basis | 16:22 |
| bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:22 |
| bauzas | #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures | 16:23 |
| bauzas | that's it for me on that topic | 16:23 |
| bauzas | moving on | 16:23 |
| bauzas | #topic Release Planning | 16:23 |
| bauzas | #link https://releases.openstack.org/bobcat/schedule.html | 16:23 |
| bauzas | #info Nova deadlines are set in the above schedule | 16:23 |
| bauzas | #info Bobcat-1 is in 1 weeks | 16:23 |
| bauzas | #info Next Tuesday 9th is stable branches review day https://releases.openstack.org/bobcat/schedule.html#b-nova-stable-review-day | 16:23 |
| bauzas | I'll communicate on that review day by emailing -discuss | 16:24 |
| bauzas | but let's discuss that more in the stable topic we have later | 16:24 |
| bauzas | #topic Review priorities | 16:25 |
| bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+(label:Review-Priority%252B1+OR+label:Review-Priority%252B2) | 16:25 |
| bauzas | #info As a reminder, cores eager to review changes can +1 to indicate their interest, +2 for committing to the review | 16:25 |
| bauzas | #topic Stable Branches | 16:25 |
| bauzas | elodilles: go for it | 16:25 |
| elodilles | #info stable nova versions were released for zed (26.1.1) and for yoga (25.1.1) last week | 16:25 |
| elodilles | otherwise not much happened | 16:25 |
| bauzas | rigt | 16:25 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:25 |
| elodilles | that's all from me | 16:25 |
| bauzas | excellent thanks | 16:27 |
| bauzas | and as I said just before, we shall round about some patches next week hopefully | 16:27 |
| elodilles | \o/ | 16:27 |
| bauzas | #topic Open discussion | 16:28 |
| bauzas | (enriquetaso) Discuss the blueprint: NFS Encryption Support for qemu | 16:28 |
| bauzas | enriquetaso: shoot | 16:28 |
| enriquetaso | hi | 16:28 |
| enriquetaso | sure | 16:28 |
| enriquetaso | As discussed on the PTG a couple weeks ago. I’ve proposed the blueprint. | 16:28 |
| enriquetaso | #link https://blueprints.launchpad.net/nova/+spec/nfs-encryption-support | 16:28 |
| enriquetaso | Summary: Cinder is working on supporting encryption on NFS volumes. To do this NFS driver uses LUKS inside qcow2 for this. | 16:28 |
| enriquetaso | This affects Nova because Nova cannot handle qemu + LUKS inside qcow2 disk format at the moment. | 16:29 |
| enriquetaso | What are your thoughts? | 16:29 |
| enriquetaso | should I mention the rolling upgrades would be a problem on the bp ? | 16:29 |
| bauzas | lemme reopen the ptg notes | 16:30 |
| bauzas | right | 16:30 |
| bauzas | we basically said we were quite okay with the proposed design but we were wondering if it was worth not scheduling to old computes | 16:31 |
| bauzas | we have three options here : | 16:32 |
| bauzas | 1/ avoid scheduling to old computes (by adding a prefilter) | 16:32 |
| bauzas | 2/ preventing this feature on the API level by checking the compute service versions | 16:33 |
| bauzas | 3/ do some compute check that would prevent the volume to be encryped on some preconditionsq | 16:33 |
| bauzas | #2 seems a bit harsh to me | 16:34 |
| sean-k-mooney | dind we discusss addign a trait | 16:35 |
| bauzas | we did it | 16:35 |
| sean-k-mooney | so 1 | 16:35 |
| bauzas | without having full quorum, hence me restating the options | 16:35 |
| sean-k-mooney | well i vote 1 or file a spec | 16:35 |
| bauzas | then I tend to say option #1 and specless blueprint as we basically went down | 16:36 |
| gibi | I'm OK with 1/ | 16:36 |
| sean-k-mooney | becasue if we dont just use a trait/prefileter then i think we need a spec to expalin why that is not sufficent and descirbe it in detail | 16:36 |
| dansmith | so a trait of "this is newer than X"? | 16:37 |
| sean-k-mooney | no | 16:37 |
| dansmith | that's kinda fundamentally wrong, so you need to expose it as a feature flag | 16:37 |
| sean-k-mooney | COMPUTE_somehting | 16:37 |
| bauzas | sean-k-mooney: do we all agree now that we accept a new trait saying something like "I_CAN_ENCRYPT_YOUR-STUFF" | 16:37 |
| dansmith | is there any compute config that needs to be enabled? if so, ideally that would control the exposure (or not) of the trait | 16:37 |
| sean-k-mooney | to report the hypervers capablity to supprot lux in qcow | 16:37 |
| sean-k-mooney | dansmith: i think this just need to check the qemu/libvirt version | 16:37 |
| bauzas | my only concern is the traits inflation but that's a string | 16:38 |
| sean-k-mooney | and report it statically if we are above that | 16:38 |
| sean-k-mooney | i dont think we need a cofnig option | 16:38 |
| bauzas | sean-k-mooney: that's why I was considering option 3 | 16:38 |
| dansmith | sean-k-mooney: yeah, that's just a little annoying I think | 16:38 |
| bauzas | which would be "meh dude, you don't have what I need, I'll just do what I can do" | 16:38 |
| sean-k-mooney | bauzas: i dont think optional encypting is accpetable | 16:38 |
| dansmith | because it becomes not as much a feature flag but a shadow version number | 16:38 |
| bauzas | sean-k-mooney: true | 16:39 |
| sean-k-mooney | if you asked for the storage to be enypeed we either need to do it or raise an error | 16:39 |
| bauzas | sounds then reasonable to ERROR the instance | 16:40 |
| enriquetaso | what a `new trait` involves? | 16:40 |
| sean-k-mooney | dansmith: im not agaisn a min compute service version check in the api as well by the way | 16:40 |
| bauzas | that's option 3 | 16:40 |
| sean-k-mooney | i dont really think 2 is harsh | 16:40 |
| bauzas | s/3/2 | 16:40 |
| sean-k-mooney | we normally dont enabel feature untill the cloud is fully upgraded | 16:40 |
| dansmith | I'm not saying that's how it needs to be, I'm just saying it feels like we're bordering on trait abuse here | 16:40 |
| dansmith | so FWIW, | 16:40 |
| sean-k-mooney | we have in the past done this on a per compute host bassis | 16:41 |
| dansmith | we could also have a scheduler filter that requires a service version at or above a number | 16:41 |
| sean-k-mooney | but in generall i think a min comptue version check is preferable | 16:41 |
| dansmith | and we could add hints/advice to the scheduler for this sort of thing | 16:41 |
| dansmith | which would be nice for this and other things I imagine | 16:41 |
| bauzas | yeah this sounds quite a reasonable tradeoff | 16:41 |
| sean-k-mooney | this being? | 16:42 |
| bauzas | I'm just wondering whether we expose the service version on the scheduling side | 16:42 |
| sean-k-mooney | i think you can already schedul based on the comptue service version | 16:42 |
| sean-k-mooney | with either the json of comptue capablity filter | 16:42 |
| sean-k-mooney | but in any case we want this to work without any configuration requried | 16:42 |
| sean-k-mooney | bauzas: why not just do 2? | 16:43 |
| gibi | I might miss something but if this feature nees compute code + libirt/qemu version then as simple compute version check is not enough | 16:43 |
| dansmith | sean-k-mooney: Im saying make it integrated | 16:43 |
| dansmith | sean-k-mooney: kinda like a prefilter | 16:43 |
| bauzas | wait https://github.com/openstack/nova/blob/master/nova/scheduler/request_filter.py#L399 | 16:43 |
| dansmith | gibi: yeah, you need service version and libvirt version and qemu version right? | 16:44 |
| sean-k-mooney | so if feature x requries minv version y have a prefilter or simialr that will request a host with that min version | 16:44 |
| bauzas | that's ephemeral encryption | 16:44 |
| sean-k-mooney | ya thats seperate | 16:44 |
| sean-k-mooney | we are talkign about encyption for nfs | 16:44 |
| sean-k-mooney | for cinder volume backend that use nfs | 16:45 |
| dansmith | gibi: that's kinda why I just think this is trait pollution because we end up with all these feature flags for any possible combination of several versions | 16:45 |
| dansmith | and especially in three years, that trait is useless as everything exposes it all the time | 16:45 |
| dansmith | so I'm not sure what the best plan is here, to be clear, I'm just saying none of these simple things feels right | 16:45 |
| sean-k-mooney | dansmith: not really usign a trait we report an abstract capblity. but in general i would prefer both a trait and a compute version bump | 16:46 |
| dansmith | that's the thing though: | 16:46 |
| sean-k-mooney | enriquetaso: how is this feature requested on teh volume by the way? | 16:47 |
| dansmith | just exposing a "can do nfs encryption" because all the versions are new enough just feels like we could have a thousand of those things | 16:47 |
| enriquetaso | when attaching the volume sean-k-mooney | 16:47 |
| sean-k-mooney | how exactly a atribute on the volume | 16:47 |
| sean-k-mooney | enriquetaso: we have two distict but related problems | 16:47 |
| sean-k-mooney | for attachemtn we know the host and have to check if the host suprot this | 16:47 |
| sean-k-mooney | for new boots or move operattions | 16:48 |
| sean-k-mooney | we need to find another host that also supports it | 16:48 |
| bauzas | that's why I tend to lean on option 2 | 16:48 |
| sean-k-mooney | and we need this to work without any operator configurign of aggreates ectra | 16:48 |
| bauzas | it's an interim solution | 16:48 |
| dansmith | bauzas: me too, but 2 is not enough right? | 16:48 |
| sean-k-mooney | option 2 works if an only if our min libvirt/qemu version are new enough | 16:49 |
| dansmith | because you could be running an older libvirt/qemu | 16:49 |
| enriquetaso | mmh, the volume attr says it's encrypted and the volume image format is qcow2: https://review.opendev.org/c/openstack/nova/+/854030/6/nova/virt/libvirt/utils.py | 16:49 |
| dansmith | and I suspect it also depends on brick versions? | 16:49 |
| enriquetaso | not sure to understand the questions | 16:49 |
| sean-k-mooney | enriquetaso: i was asking because if we are booting a new vm we would need to look that up | 16:49 |
| sean-k-mooney | and then include it as an input to placemnt and the schduler in some way | 16:49 |
| bauzas | dansmith: hah, true, I was considering an API check, but that would require the libvirt versions, so nevermind my foolness | 16:50 |
| bauzas | yeah, so that'a tuple (libvirt, compute) for accepted versions | 16:50 |
| sean-k-mooney | well it may or may not again it depend on if our min version is above or below the ersion that intoduced it | 16:50 |
| bauzas | you need libvirt AND compute versions to be recent enough | 16:50 |
| enriquetaso | oh, the volume doesnt have anything special, it just volume type=nfs and encrypted=true sean-k-mooney | 16:50 |
| dansmith | sean-k-mooney: yeah, well, that's a good reason not to just add the compute+libvirt+qemu into a trait IMHO | 16:51 |
| sean-k-mooney | dansmith: right which is not what i suggested | 16:51 |
| dansmith | I know :) | 16:51 |
| sean-k-mooney | i very explictly said have a triat for the capablity to supprot lux in qcow | 16:51 |
| gibi | dansmith: on the trait pollution: either we encode a list of versions as a capability on the compute side, or we expose those versions to the scheduler / placement and code up a similar mapping of capability - versions in the sceduler side. We just move around similar logic | 16:52 |
| dansmith | gibi: yeah, understand, it's just that traits are supposed to be timeless right? | 16:52 |
| bauzas | we have 5 mins to find a solution or defer to a spec, honestly | 16:52 |
| dansmith | I'm not saying I know which solution (combination) is best, I'm just saying nothing feels particularly natural to me | 16:53 |
| sean-k-mooney | so i was suggestign have the driver check the requirements and report COMPUTE_LUKS_IN_QCOW if it supprots it | 16:53 |
| sean-k-mooney | and have a prefilter requesst that if the voluem was nfs and qcow and encrypted=true | 16:53 |
| gibi | dansmith: we will have a bunch of traits yes, but I don't see what problem that causes. We never remove min compute version checks from the code either | 16:54 |
| sean-k-mooney | and addtional have a min compute service check for the rooling upgrade case | 16:54 |
| sean-k-mooney | the other thing about traits is they are ment to be virt driver independent | 16:54 |
| gibi | LUKS_IN_QCOW does not seem to be libvirt dependent | 16:55 |
| sean-k-mooney | to dans timeless point i.e. if we add one it shoudl be resuable by other virt driver | 16:55 |
| sean-k-mooney | ya i was just thinking that | 16:55 |
| sean-k-mooney | its the qcow bit but | 16:55 |
| sean-k-mooney | we have that alredy | 16:55 |
| sean-k-mooney | https://github.com/openstack/os-traits/blob/master/os_traits/compute/ephemeral.py#L18 and https://github.com/openstack/os-traits/blob/master/os_traits/compute/image.py#L28 | 16:56 |
| sean-k-mooney | almost would work togather | 16:56 |
| sean-k-mooney | btu we cant assume that epmeral encryption supprot for lux means nfs also works | 16:56 |
| enriquetaso | LUKS_IN_QCOW is already a config option on Nova gibi ? | 16:56 |
| sean-k-mooney | enriquetaso: not really | 16:57 |
| sean-k-mooney | or not that im aware of | 16:57 |
| sean-k-mooney | in the context of cinder | 16:57 |
| gibi | enriquetaso: we are discussing creating a new placement trait to represent if a compute supports luks in qcow | 16:57 |
| enriquetaso | gibi++ thanks | 16:57 |
| bauzas | I honestly think we need to settle the dust | 16:58 |
| bauzas | given the very short time we have left I hereby propose enriquetaso to create a spec and describe the feature | 16:58 |
| bauzas | we could then chime on the upgrade concerns | 16:58 |
| enriquetaso | okay u.u | 16:59 |
| bauzas | enriquetaso: are you familiar with the spec process ? | 16:59 |
| enriquetaso | is it too different from the cinder one? | 16:59 |
| enriquetaso | bauzas, do you a have a doc? :P | 16:59 |
| bauzas | enriquetaso: I can provide you pointers and more than that : guidance | 16:59 |
| enriquetaso | sure bauzas | 17:00 |
| bauzas | #agreed enriquetaso to provide a spec for this feature | 17:00 |
| bauzas | #action bauzas to provide enriquetaso details on the spec process | 17:00 |
| bauzas | we're on time | 17:00 |
| bauzas | the agenda is done | 17:00 |
| bauzas | so thanks all and see you next week | 17:00 |
| bauzas | #endmeeting | 17:01 |
| opendevmeet | Meeting ended Tue May 2 17:01:01 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:01 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2023/nova.2023-05-02-16.00.html | 17:01 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2023/nova.2023-05-02-16.00.txt | 17:01 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2023/nova.2023-05-02-16.00.log.html | 17:01 |
| elodilles | thanks o/ | 17:01 |
| enriquetaso | thansk!! | 17:01 |
| bauzas | enriquetaso: gimme a sec | 17:01 |
| enriquetaso | sure bauzas | 17:01 |
| bauzas | enriquetaso: so the spec process is quite simple : it's a doc | 17:01 |
| bauzas | enriquetaso: we have a template you can reuse https://specs.openstack.org/openstack/nova-specs/specs/2023.2/approved/2023.2-template.html | 17:01 |
| enriquetaso | looks good | 17:02 |
| bauzas | enriquetaso: what you have to do is to write your own rst file based on that template and propose it for the approved/ directory | 17:02 |
| bauzas | like https://review.opendev.org/q/project:openstack/nova-specs+is:open | 17:02 |
| enriquetaso | bauzas, do you remember when is the spec freeze? | 17:03 |
| sean-k-mooney | its milestone 2 | 17:03 |
| sean-k-mooney | so after the physical ptg at the end of july | 17:03 |
| bauzas | enriquetaso: sure, look https://releases.openstack.org/bobcat/schedule.html#b-nova-spec-review-day | 17:03 |
| bauzas | damn | 17:04 |
| sean-k-mooney | july 6th actullly so start of july | 17:04 |
| sean-k-mooney | https://releases.openstack.org/bobcat/schedule.html#b-nova-spec-freeze | 17:04 |
| bauzas | yup, my link was wrong | 17:04 |
| bauzas | enriquetaso: honestly, as you understood, most of the review process on that file will consist on us to agree on the upgrade concerns | 17:05 |
| bauzas | which are not only the rolling upgrade concerns but also the fact that we need a recent libvirt | 17:05 |
| bauzas | enriquetaso: you know our minimum libvirt versions we support, right? | 17:05 |
| enriquetaso | sorry | 17:06 |
| enriquetaso | it desconnect | 17:06 |
| enriquetaso | i'm back | 17:06 |
| bauzas | enriquetaso: no worries | 17:06 |
| enriquetaso | but I think I have all the info i need sean-k-mooney bauzas | 17:06 |
| bauzas | I was mentioning that most of the spec review will be about the upgrade concerns | 17:06 |
| bauzas | enriquetaso: our current minimums for libvirt are https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L219-L222 | 17:07 |
| bauzas | sean-k-mooney: have you already proposed the libvirt min bump ? | 17:07 |
| sean-k-mooney | no i asked kashyap to do that | 17:08 |
| sean-k-mooney | i dont think they have had time to propose any patches | 17:08 |
| kashyap | sean-k-mooney: Yes; it's on my list for this week | 17:09 |
| bauzas | ack | 17:09 |
| sean-k-mooney | if our currnt mins are enough this is simple as its just a min comptue service bump. if we have a min os-brick we can do that by raising or min os-brick | 17:09 |
| kashyap | Right now I'm chasing down the kernel crash Dan pointed out earlier | 17:09 |
| dansmith | can't do the min bump until I finish the ceph job stuff I think | 17:10 |
| sean-k-mooney | it might be possibel but im ok waiting to m2 i guess | 17:10 |
| sean-k-mooney | i would have preferd to do it at m1 | 17:10 |
| sean-k-mooney | but i think ceph is a higher piority | 17:11 |
| bauzas | I was under the impression that the nfs encryption was requiring a recent libvirt version but I could be wrong and mixing two things | 17:12 |
| enriquetaso | mmmh, I'm not sure | 17:15 |
| sean-k-mooney | i donthing think libvirt matters | 17:15 |
| sean-k-mooney | but qemu-img might | 17:15 |
| sean-k-mooney | and qemu | 17:16 |
| enriquetaso | I can check and add this information on the spec if the feature need a min version of libvirt or qemu | 17:17 |
| enriquetaso | okay, I'm going to disconnect now | 17:18 |
| enriquetaso | thank you so much again | 17:19 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!