| noonedeadpunk | not sure what ceph_config | 08:15 |
|---|---|---|
| noonedeadpunk | mornings | 08:18 |
| noonedeadpunk | sorry, had to take a day off yesterday | 08:18 |
| noonedeadpunk | jrosser: no idea what ceph_config is doing though? I guess injecting conf into ceph mons directly? | 08:19 |
| noonedeadpunk | aka what cephadm does? | 08:19 |
| noonedeadpunk | that would kinda make sense then I guess... | 08:19 |
| jrosser | I think so yes | 08:28 |
| jrosser | though as far as I can see only from the [global] section perhaps | 08:28 |
| jrosser | noonedeadpunk: also what is this?! https://github.com/NVIDIA/open-gpu-kernel-modules/commit/476bd34534a9389eedff73464d3f2fa5912f09ae | 09:11 |
| noonedeadpunk | o_O | 09:12 |
| noonedeadpunk | so this shouldn't be from application hub anymore and finally part of opensource drivers? | 09:13 |
| noonedeadpunk | as actually, all nvidia drivers, except ones for vgpu, were opensourced for a while now | 09:14 |
| noonedeadpunk | so if you'd use mig or pci-passthrough without vgpu - they could be used. | 09:14 |
| noonedeadpunk | so that's kinda sweet. | 09:14 |
| jrosser | idk where README.vgpu is though | 09:15 |
| noonedeadpunk | hm... they've also updated application hub? as my account doesn/t work anymore..... | 09:16 |
| noonedeadpunk | it suggests creating an account now o_) | 09:17 |
| jrosser | there is also this https://github.com/NVIDIA/vgpu-device-manager | 09:17 |
| jrosser | oh well nvidia have some strange definition of SSO where you actually have multiple accounts and they keep merging/changing the auth backend | 09:18 |
| jrosser | its like multiple-single-sign-on | 09:18 |
| noonedeadpunk | but my email that used to work now suggests creating a new account... anyway | 09:19 |
| noonedeadpunk | vgpu-device-manager is also super interesting actually | 09:19 |
| jrosser | whilst it says k8s thats not really what the readme describes | 09:19 |
| jrosser | looks totally usable on normal hosts too | 09:19 |
| noonedeadpunk | yup, it is. I guess it still does echo to /sys though. but in a way more usable fashion kinda | 09:20 |
| noonedeadpunk | I wonder if they do have packaging for it.... | 09:21 |
| jrosser | we are just doing planning for removing vCS licences, which is a total mess | 09:21 |
| noonedeadpunk | Yeah... | 09:22 |
| jrosser | so if the open driver can do similar then that is very very interesting | 09:22 |
| noonedeadpunk | btw, do you know where they've moved doc on slicing vgpu for enterprise ai? | 09:22 |
| noonedeadpunk | as they've removed vCS from vGPU page completely | 09:22 |
| noonedeadpunk | (which is fair) | 09:22 |
| noonedeadpunk | but didn't add this enterprise ai | 09:23 |
| noonedeadpunk | (which is not) | 09:23 |
| noonedeadpunk | Meaning this: https://docs.nvidia.com/grid/16.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid-reference | 09:24 |
| jrosser | like this? https://docs.nvidia.com/ai-enterprise/latest/user-guide/index.html#supported-gpus-grid-vgpu | 09:25 |
| jrosser | they even call it grid :) | 09:26 |
| jrosser | so - what do you think about this https://review.opendev.org/c/openstack/openstack-ansible/+/910220 | 09:28 |
| jrosser | ^ easy fix | 09:28 |
| jrosser | or difficult fix is to make slurp jobs handle branch names that will randomly change from stable/* to unmaintained/* | 09:28 |
| jrosser | becasue currently we are totally broken on 2023.1 | 09:29 |
| ThiagoCMC | So, a bit about this Ceph situation. I am unable to use `ceph-ansible` `stable-8.0` in an isolated (Ceph-only) lab. I swear it worked days ago, not anymore. :-/ | 09:57 |
| jrosser | ThiagoCMC: they are continuously changing the code on that branch, i am not surprised at all | 10:02 |
| jrosser | like i said yesterday i think that you will be able to deploy Reef using stable-7.0 | 10:02 |
| jrosser | you just need to override the variable that defines the version | 10:02 |
| ThiagoCMC | Cool, I haven't tried that. Thanks for reminding me! I'll give it a try. | 10:03 |
| jrosser | if it doesnt work, then the problem is likely to be small compared to stable-8.0 imho | 10:03 |
| ThiagoCMC | True lol | 10:04 |
| jrosser | damiandabrowski: andrew is away this week so if you are able to look at reviews it would be helpful | 12:04 |
| damiandabrowski | okay! I'll have a look during the evening | 12:08 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova stable/2023.1: Evaluate my_ip address once https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/908699 | 12:19 |
| jrosser | noonedeadpunk: at some point it would be good to look at the magnum stuff again | 12:48 |
| jrosser | i am really unsure about what to do with tempest and the resources creation as eventually it seems to always end up "refactor even more old stuff" and it's kind of never ending | 12:48 |
| jrosser | for example, i could make a new tidier tempest role somewhere in plugins collection or something, but that feels like taking on yet another large project | 12:50 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:01 |
| opendevmeet | Meeting started Tue Feb 27 15:01:02 2024 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:01 |
| noonedeadpunk | #topic rollcall | 15:01 |
| noonedeadpunk | o/ | 15:01 |
| damiandabrowski | hi! | 15:01 |
| jrosser | o/ hello | 15:01 |
| noonedeadpunk | #topic office hours | 15:03 |
| noonedeadpunk | so, it feels it's really high time for new point releases | 15:03 |
| noonedeadpunk | though I saw some "blockers" which would be nice to handle first | 15:03 |
| noonedeadpunk | seems https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/909868 was quite important, for instance | 15:04 |
| jrosser | 2023.1 is totally blocked i think | 15:04 |
| noonedeadpunk | yep, by Yoga upgrade | 15:05 |
| noonedeadpunk | So we need to land Yoga upgrade disablement first: https://review.opendev.org/c/openstack/openstack-ansible/+/910220 | 15:05 |
| jrosser | i looked at how to handle stable|unmaintainted but that was just /o\ complicated | 15:05 |
| noonedeadpunk | Yeah, I also failed to get us access to unmaintained. | 15:06 |
| noonedeadpunk | And frankly - this branch removal/adding is quite confusing... | 15:06 |
| NeilHanlon | o/ sorry i'm late | 15:09 |
| noonedeadpunk | I also didn't check neither on docs for ops repo, nor for octavia and ovn scenario in AIO | 15:10 |
| jrosser | i need some direction on the magnum patches | 15:11 |
| jrosser | well not so much magnum, but the fixing * else that seems to be also involved :( | 15:11 |
| jrosser | specifically tempest resource creation, it's just gigantic mess now | 15:11 |
| noonedeadpunk | yup | 15:12 |
| noonedeadpunk | I know... | 15:12 |
| jrosser | i think that i can make time this week to just strip everythig to do with resource creation out of os_tempest | 15:13 |
| jrosser | and port it to openstack_resources | 15:13 |
| jrosser | but we should decide if that is a good idea or not | 15:13 |
| noonedeadpunk | that is very good question | 15:14 |
| noonedeadpunk | as problematic part - that plenty of logic and weirdness lies in tempest role itself | 15:14 |
| jrosser | i am wondering if that is just historical accumulation | 15:14 |
| noonedeadpunk | and I guess end-goal of all that would be to just skip tempest, but do have some resources? | 15:14 |
| jrosser | yes thats right | 15:15 |
| noonedeadpunk | And basically only public network is needed iirc | 15:15 |
| jrosser | but you can't do that just now without making the logic in tempest role even more complicated | 15:15 |
| jrosser | ultimately there is actually not much needed in tempest.conf | 15:16 |
| jrosser | flavor / image id * 2, network id | 15:16 |
| jrosser | maybe one more | 15:16 |
| jrosser | so i was thinking to make it possible to pass in name -> os_tempest looks up the id | 15:17 |
| jrosser | or pass the id directly | 15:17 |
| jrosser | and move all the creation stuff out of the role completely | 15:17 |
| jrosser | as even if we use openstack_resources that doesnt return the id really to re-use later | 15:18 |
| noonedeadpunk | but why you still try to install it at all instead of just disabling it as a whole and including openstack_resources just here https://review.opendev.org/c/openstack/openstack-ansible-ops/+/906363/14/mcapi_vexxhost/playbooks/install_and_test.yml#14 ? | 15:18 |
| noonedeadpunk | yeah, output of openstack_resources result is actually a good topic on it's own | 15:18 |
| noonedeadpunk | and if that should be covered | 15:18 |
| noonedeadpunk | maybe registering results or output to some local facts might be useful... | 15:19 |
| jrosser | well maybe you are right and i was trying too hard to make a general solution | 15:19 |
| noonedeadpunk | I mean - doing general solution is perfect scenario | 15:19 |
| noonedeadpunk | But given amount of overhead... | 15:20 |
| noonedeadpunk | Maybe it should not be a blocker and we just need to iterate over things | 15:20 |
| jrosser | yes tbh this is a better way to look at it | 15:20 |
| noonedeadpunk | I still think we should do smth with tempest. | 15:20 |
| jrosser | seems everone is busy++ so need to take a tractible path | 15:21 |
| noonedeadpunk | but this should not really block capi from my perspective. Or at least if there's a way to unblock - better do that | 15:21 |
| noonedeadpunk | Yes, until end of March I'm really just /o\ | 15:21 |
| noonedeadpunk | So is damiandabrowski | 15:21 |
| noonedeadpunk | I do hope to be able to catch-up though once thing we're working on is done. | 15:22 |
| noonedeadpunk | Also, I guess it's time to start populating PTG etherpad.... | 15:24 |
| noonedeadpunk | Let it be the link | 15:24 |
| noonedeadpunk | #link https://etherpad.opendev.org/p/osa-dalmatian-ptg | 15:24 |
| NeilHanlon | 🥳 | 15:24 |
| noonedeadpunk | and I'm adding ceph-ansible right away. | 15:25 |
| NeilHanlon | yes. | 15:25 |
| noonedeadpunk | Will populate it with leftovers from caracal ptg as well | 15:26 |
| noonedeadpunk | but also - we probably should pick up a timeframe for the PTG | 15:26 |
| noonedeadpunk | We can do "as usual" Tuesday - 14 - 17 UTC? | 15:27 |
| noonedeadpunk | or 15 - 18 | 15:28 |
| noonedeadpunk | or should I make some kind of poll to vote on it? | 15:28 |
| jrosser | what actual date is this? | 15:29 |
| noonedeadpunk | good question | 15:30 |
| noonedeadpunk | April 9 | 15:30 |
| NeilHanlon | April 8-12, 2024 | 15:30 |
| NeilHanlon | yep, so the 9th | 15:30 |
| NeilHanlon | i'm flexible, but will be traveling to Texas for a conference on 4/11 | 15:31 |
| jrosser | hmm that is during school holidays for me so 50/50 at best for the whole week | 15:33 |
| noonedeadpunk | ouch | 15:33 |
| noonedeadpunk | that's defenitely a bad timing for PTG then... | 15:33 |
| noonedeadpunk | but eventually, looking at scope for Caracal, it slightly feels that not much will be delivered out of it | 15:34 |
| noonedeadpunk | like - incus for sure won't be done | 15:34 |
| jrosser | tbh i think this is a large job | 15:34 |
| noonedeadpunk | yeah... | 15:34 |
| jrosser | and requires some pretty good thinking, as it is an opportunity to modernise things rather than just drop-in replacement | 15:35 |
| noonedeadpunk | I close to never used LXD at scale, so hard to judge on what's best practise would be | 15:35 |
| jrosser | i think that personally i can only commit to smaller things than that for maybe the next cycle or two | 15:36 |
| noonedeadpunk | But also I guess it should be not drop-in but indeed smth modern which can be done as an option to old legacy | 15:36 |
| jrosser | my hunch is that we can collapse many many ansible tasks into native things in LXD/incus | 15:36 |
| NeilHanlon | I think incus is reasonable for next cycle, fwiw (on the Fedora/EL side) | 15:37 |
| noonedeadpunk | well, will see about time/prios for that | 15:44 |
| noonedeadpunk | as that is totally would be very-very appealing to have and quite logical evolution of what we have today | 15:44 |
| noonedeadpunk | with LXC | 15:44 |
| jrosser | are there any bugs to look at? | 15:44 |
| ThiagoCMC | I have experience with LXD, I am currently running part of my OSA (Compute, Network, and OSDs) on top of LXD Containers. I want to help! | 15:44 |
| jrosser | i had a report from hamburgler3 yesterday which i have just put into launchpad | 15:45 |
| noonedeadpunk | well, I mean, we have also an etherpad from bug triage day that needs to be looked at | 15:45 |
| noonedeadpunk | #link https://bugs.launchpad.net/openstack-ansible/+bug/2055178 | 15:46 |
| noonedeadpunk | ok, I had very simmilar lately | 15:46 |
| noonedeadpunk | I didn't get to the point of finding out wtf is going on | 15:46 |
| noonedeadpunk | eventually, /var/lib/haproxy/dev/log is a "chroot" | 15:47 |
| noonedeadpunk | And actually... not being idempotent might be the root cause | 15:48 |
| noonedeadpunk | so that is potentially good catch | 15:48 |
| jrosser | my thoughts were why we needed to do any of this | 15:48 |
| jrosser | as i would expect the distro packages to do the necessary stuff when haproxy is installed | 15:49 |
| noonedeadpunk | well... there's your note there.... | 15:49 |
| jrosser | well indeed, but it has been a while and that might no longer be true | 15:49 |
| noonedeadpunk | Yep, we had this exact issue being reproduced, so I for sure can look there with some priority | 15:50 |
| jrosser | even needing to make the bind mount surprises me, as haproxy does this chroot thing as part of it's own functionality | 15:50 |
| jrosser | but i kind of feel i miss something important here | 15:51 |
| noonedeadpunk | yep, true, I did just rmdir and it was created is proper permissions on restart | 15:51 |
| noonedeadpunk | and well, after systemd-journald restart as well | 15:51 |
| noonedeadpunk | but again - that was all on ubuntu | 15:52 |
| noonedeadpunk | worth trying dropping all that for sure | 15:53 |
| jrosser | maybe it is as simple as boot a centos / ubuntu vm and chdeck that haproxy can log to the journal out of the box | 15:53 |
| jrosser | if so we can delete all of this | 15:53 |
| noonedeadpunk | ++ | 15:54 |
| noonedeadpunk | btw, we've also tested and slightly adopted andrew's patch to keystone: https://review.opendev.org/c/openstack/keystone/+/910337 | 15:55 |
| noonedeadpunk | so if you can check if it works for you still - would be great :) | 15:55 |
| noonedeadpunk | but yes | 15:55 |
| jrosser | oh i did see that yes | 15:57 |
| jrosser | we can look at that maybe next week | 15:57 |
| noonedeadpunk | #endmeeting | 15:59 |
| opendevmeet | Meeting ended Tue Feb 27 15:59:08 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:59 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-02-27-15.01.html | 15:59 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-02-27-15.01.txt | 15:59 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-02-27-15.01.log.html | 15:59 |
| noonedeadpunk | ah, snap | 15:59 |
| noonedeadpunk | I fully forgot about 1 big thing.... | 15:59 |
| noonedeadpunk | as part of ovn-bgp-agent, deployment of FRRouting is needed | 15:59 |
| noonedeadpunk | With that I'm intentind to move ansible-role-frrouting from vexxhost namespace under osa governance | 16:00 |
| NeilHanlon | 👍 makes sense | 16:00 |
| jrosser | sure, looks like nice new capability | 16:01 |
| noonedeadpunk | And I'm practising there with Molecule right now. Hopefully this will trigger me to add coverage to other "standalone" roles in a good way | 16:01 |
| jrosser | i also did some zuul error cleanup | 16:02 |
| noonedeadpunk | ah, yes, thanks a lot for that! | 16:02 |
| noonedeadpunk | I guess Zed is slightly broken now from what I saw... | 16:02 |
| jrosser | these i think are maybe only useful if someone is amenable to force-merge them on older branches | 16:02 |
| noonedeadpunk | actually ovng-bgp-agent has soooo many ways to be deployed/configured.... | 16:06 |
| noonedeadpunk | and then quite some changes in logic might be needed. Like one way requires a standalone local ovn cluster on top | 16:07 |
| noonedeadpunk | another just absent connectivity of provider network to ovs (don't add port to briidge) | 16:08 |
| jrosser | NeilHanlon: if you are around could you take a look at https://review.opendev.org/c/openstack/openstack-ansible/+/910220 | 16:23 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ops master: Add hook playbook install and test magnum capi driver https://review.opendev.org/c/openstack/openstack-ansible-ops/+/906363 | 16:36 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_magnum master: Add job to test Vexxhost cluster API driver https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/905199 | 16:37 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Bump ansible version to 2.15.9 https://review.opendev.org/c/openstack/openstack-ansible/+/905619 | 16:43 |
| spatel | Folks, do you know how to not allow end user to release/disassociate floating IP in horizon or commandline? | 17:02 |
| spatel | currently my customer has permission to remove or disassociate floating IP that is painful sometime. I want to stop this behavior | 17:03 |
| noonedeadpunk | policy? | 17:03 |
| spatel | reading policy file and found - delete_floatingips_tags | 17:07 |
| noonedeadpunk | ok, just in case - this bind mount is needed for centos at least... | 18:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Use correct permissions for haproxy log mount https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/910384 | 18:46 |
| noonedeadpunk | actually this seems fixing it ^ | 18:46 |
| noonedeadpunk | hamburgler3: pinging you so you could check as well :) | 18:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Update upstream SHAs https://review.opendev.org/c/openstack/openstack-ansible/+/910386 | 18:52 |
| hamburgler | noonedeadpunk: haproxy fix looks good :) | 19:01 |
| noonedeadpunk | awesome | 19:01 |
| spatel | Folks, Did you try to use NFS based cinder-volume ? | 20:28 |
| spatel | Do I need to mount NFS directory to all the controller + compute nodes ? | 20:29 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Do not log contents of installed keypairs by default https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/908838 | 23:07 |
| opendevreview | Merged openstack/openstack-ansible-os_tempest master: Switch default external network name to 'physnet1' https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/908768 | 23:26 |
| opendevreview | Merged openstack/ansible-role-uwsgi stable/2023.2: Remove undefined bionic linters job https://review.opendev.org/c/openstack/ansible-role-uwsgi/+/910191 | 23:41 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!