| opendevreview | Merged openstack/kolla-ansible master: Fix bad openstack command while registering IDP https://review.opendev.org/c/openstack/kolla-ansible/+/826322 | 07:14 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/xena: Fix bad openstack command while registering IDP https://review.opendev.org/c/openstack/kolla-ansible/+/826375 | 07:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/wallaby: Fix bad openstack command while registering IDP https://review.opendev.org/c/openstack/kolla-ansible/+/826376 | 07:28 |
| jingvar | \o | 07:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/wallaby: Fix bad openstack command while registering IDP https://review.opendev.org/c/openstack/kolla-ansible/+/826376 | 07:53 |
| opendevreview | Kevin Rasmussen proposed openstack/kolla-ansible stable/xena: Fix bad openstack command while registering IDP https://review.opendev.org/c/openstack/kolla-ansible/+/826375 | 09:32 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/train: CI: make it work again https://review.opendev.org/c/openstack/kolla/+/826033 | 09:55 |
| opendevreview | likui proposed openstack/kolla-ansible stable/wallaby: update the default value of node_custom_config https://review.opendev.org/c/openstack/kolla-ansible/+/825941 | 10:45 |
| opendevreview | likui proposed openstack/kolla-ansible stable/wallaby: update the default value of node_custom_config https://review.opendev.org/c/openstack/kolla-ansible/+/825941 | 10:48 |
| opendevreview | likui proposed openstack/kolla-ansible stable/victoria: update the default value of node_custom_config https://review.opendev.org/c/openstack/kolla-ansible/+/825940 | 10:50 |
| opendevreview | likui proposed openstack/kolla-ansible stable/xena: update the default value of node_custom_config https://review.opendev.org/c/openstack/kolla-ansible/+/825942 | 10:53 |
| opendevreview | Merged openstack/kolla-ansible master: openvswitch: add option to set hw offload https://review.opendev.org/c/openstack/kolla-ansible/+/819254 | 10:55 |
| opendevreview | Merged openstack/kolla-ansible stable/xena: Revert "Use friendly target names in Prometheus" https://review.opendev.org/c/openstack/kolla-ansible/+/826216 | 10:55 |
| opendevreview | Merged openstack/kolla-ansible stable/train: CI: make it work again (k-a side) https://review.opendev.org/c/openstack/kolla-ansible/+/826269 | 10:55 |
| opendevreview | Merged openstack/kolla-ansible stable/xena: Fix bad openstack command while registering IDP https://review.opendev.org/c/openstack/kolla-ansible/+/826375 | 12:29 |
| jingvar | I've added a few new compute nodes, after container image pull, I've lost containers on working nodes | 12:59 |
| jingvar | images from local registry | 13:00 |
| jingvar | Is it expected? | 13:00 |
| mnasiadka | mgoddard: might be a bit late for the meeting, would you be able to start it if I'm not around? | 13:19 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla-ansible stable/train: CI: trust Let's Encrypt certificates on CentOS https://review.opendev.org/c/openstack/kolla-ansible/+/826464 | 13:24 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/train: CI: make it work again https://review.opendev.org/c/openstack/kolla/+/826033 | 13:25 |
| dswebb | hey all, are there any plans to support Rocky Linux as the baseOS going forward? Or are you sticking with CentOS streams? | 13:30 |
| dswebb | (the reason I'm asking is we're planning a new DC and trying to figure out what path we want go from our normal CentOS 8 installation) | 13:32 |
| hrw | dswebb: there is one patch for RL as host OS. | 13:37 |
| hrw | dswebb: there are no plans for RL as in-container-os | 13:38 |
| dswebb | re: container-os, that is fine. We've made our peace with the debian only containers, but it would require a bunch of retooling to move away from a rhel derivative for our host OS setup. | 13:39 |
| hrw | dswebb: so run C8S? | 13:39 |
| dswebb | I was just wondering if stream was going to be a long term commitment from kolla. Last I had heard (arguably at the beginning of the stream debacle) it was a wait and see game. What we don't want to do is go down a path that requires us at a later date to rejig everything to stay in a supported setup. | 13:43 |
| dswebb | if that makes sense | 13:43 |
| hrw | we plan to limit in-container-os choices and get rid of binary images. k-a will allow to deploy on c8s/debian/rockylinux/ubuntu (and if someone provide patches for other rhel rebuilds then we may consider) | 13:45 |
| hrw | c8 is eol already and we moved c8->c8s already | 13:45 |
| dswebb | perfect, thanks for that. | 13:47 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/train: CI: make it work again https://review.opendev.org/c/openstack/kolla/+/826033 | 14:06 |
| mnasiadka | mgoddard: actually I'm back ;) | 14:07 |
| hrw | upgrade jobs are no longer in train | 14:10 |
| mnasiadka | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt frickler adrian-a - meeting in 3 minutes | 14:57 |
| jingvar | I've added a few new compute nodes, after container image pull, I've lost containers on working nodes - is it expected? | 14:58 |
| mnasiadka | #startmeeting kolla | 15:00 |
| opendevmeet | Meeting started Wed Jan 26 15:00:09 2022 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 15:00 |
| mnasiadka | #topic rollcall | 15:00 |
| yoctozepto | o/ | 15:00 |
| mnasiadka | o/ | 15:00 |
| mgoddard | O\ | 15:04 |
| mnasiadka | oh, and I wanted to write two person meeting ;-) | 15:04 |
| hrw | ]o[ | 15:04 |
| mnasiadka | four, hooray | 15:04 |
| mnasiadka | #topic agenda | 15:04 |
| mnasiadka | * Announcements | 15:04 |
| mnasiadka | * Review action items from the last meeting | 15:04 |
| mnasiadka | * CI status | 15:04 |
| mnasiadka | * Release tasks | 15:04 |
| mnasiadka | * Current cycle planning | 15:04 |
| mnasiadka | * Additional agenda (from whiteboard) | 15:04 |
| mnasiadka | * Open discussion | 15:04 |
| mnasiadka | #topic Announcements | 15:05 |
| mnasiadka | None from me | 15:05 |
| mnasiadka | #topic Review action items from the last meeting | 15:06 |
| mnasiadka | 1. mnasiadka to triage security bugs and update them with resolution plan (if needed) | 15:06 |
| mnasiadka | 2. mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle | 15:06 |
| mnasiadka | 3. kevko to let frickler know whether osism's solution is fine for his use case | 15:06 |
| mnasiadka | 4. yoctozepto to review going-podman patches | 15:06 |
| mnasiadka | 5. mgoddard to review going-podman patches | 15:06 |
| mnasiadka | 6. halomiva/hinermar propose change for podman | 15:06 |
| mnasiadka | 1. is in progress - haproxy tls ciphers bug doesn't seem to be a bug, I have a WIP change for using testssl.sh on our ciphers to detect issues earlier | 15:07 |
| mgoddard | I reviewed the systemd patch | 15:07 |
| mnasiadka | 2. still to be done | 15:07 |
| yoctozepto | I did not have time to review | 15:07 |
| mnasiadka | Ok then | 15:08 |
| mnasiadka | #action mnasiadka to triage security bugs and update them with resolution plan (if needed) | 15:09 |
| mnasiadka | #action mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle | 15:09 |
| mnasiadka | #action kevko to let frickler know whether osism's solution is fine for his use case | 15:09 |
| mnasiadka | #action yoctozepto to review going-podman patches | 15:09 |
| mnasiadka | did halomiva/hinermar propose changes for podman? | 15:10 |
| mnasiadka | (I guess not only the systemd ones) | 15:10 |
| mnasiadka | #action halomiva/hinermar propose changes for podman | 15:11 |
| mnasiadka | They are not here, so let's add it back as well. | 15:12 |
| mnasiadka | #topic CI status | 15:12 |
| * hrw | 15:13 | |
| mgoddard | they did push the podman changes, although they need a rebase | 15:14 |
| hrw | I hope that current train fixes make it pass. | 15:14 |
| hrw | once they merge I do not want to touch train again | 15:14 |
| yoctozepto | hrw: nobody does | 15:14 |
| mnasiadka | Ok, Train situation is clear more or less | 15:15 |
| mnasiadka | What about CentOS Linux 8 retirement in CI - is it done on Kolla/Kolla-Ansible side? | 15:15 |
| yoctozepto | we have some cephadm issues on ubuntu | 15:15 |
| yoctozepto | mnasiadka: pending at least due to bifrost | 15:15 |
| yoctozepto | but I have green light from dtantsur to drop the offending CI config | 15:16 |
| mnasiadka | Ok, I also need to take a look in the Kayobe patches. | 15:16 |
| mgoddard | mnasiadka: I think the ussuri kayobe patch is stuck | 15:16 |
| mgoddard | possibly we need to drop upgrade jobs, or backport cs8 to train | 15:17 |
| mnasiadka | mgoddard: I noticed it's failing on post upgrade vm tests, but cirros seems to get up - so I don't really know what is the issue | 15:17 |
| mnasiadka | But yes, dropping upgrade jobs seems like a good solution (and an easy one). | 15:18 |
| mnasiadka | Anyway, let's keep an eye on the CI related work that needs to be done... | 15:18 |
| mnasiadka | #topic Release tasks | 15:19 |
| mnasiadka | It's R-9 week, | 15:19 |
| yoctozepto | so close to R-8 | 15:20 |
| mnasiadka | R-8: Switch binary images to current release¶ | 15:20 |
| hrw | easy and worked last time I checked | 15:21 |
| mnasiadka | So that's next week - any volunteer? | 15:21 |
| hrw | can prepare patch for it | 15:21 |
| mnasiadka | thanks | 15:21 |
| mnasiadka | #action hrw to prepare patches for R-8 Switch binary images to current release | 15:21 |
| mnasiadka | #topic Current cycle planning | 15:22 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/826033 - train is +1 on zuul | 15:23 |
| mnasiadka | So, we have some features here and there, I see Podman has gained some traction, so I guess we're on the right track. | 15:23 |
| mnasiadka | At some point last cycle we promised to look at VMware NSXP support patches - we all know there's no CI for that, but I guess we can't leave those patches hanging there. | 15:25 |
| mnasiadka | Add support for VMware NSXP https://review.opendev.org/c/openstack/kolla-ansible/+/807404 | 15:25 |
| mnasiadka | Add support for VMware First Class Disk (FCD) https://review.opendev.org/c/openstack/kolla-ansible/+/808760 | 15:25 |
| mnasiadka | Is there any volunteer to look into those? | 15:25 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Switch to use Yoga binary packages https://review.opendev.org/c/openstack/kolla/+/826488 | 15:26 |
| mgoddard | I could take a look | 15:27 |
| mgoddard | looks like you already did mnasiadka | 15:27 |
| mnasiadka | Yes, I already did some time ago, I'll sign us both into that. | 15:27 |
| mnasiadka | Ansible core 2.12 - I think I saw a change around bumping it up | 15:28 |
| mnasiadka | Yes, merged already, crossing it out. | 15:28 |
| mgoddard | do we have one for kayobe? | 15:28 |
| yoctozepto | mnasiadka: yeah, done | 15:29 |
| yoctozepto | works perfectly | 15:29 |
| yoctozepto | I love such changes | 15:29 |
| hrw | +1 for FCD | 15:29 |
| mgoddard | didn't 2.12 come with the requirement of python 3.8+? | 15:29 |
| yoctozepto | hrw: FCD? | 15:29 |
| yoctozepto | ah | 15:29 |
| yoctozepto | vmware | 15:29 |
| mnasiadka | mgoddard: they claimed in the docs - that it will require it, but it seems to work... | 15:30 |
| mgoddard | we do test max & min versions in CI | 15:30 |
| yoctozepto | we test max only on ubuntu | 15:30 |
| mgoddard | I think centos gets min | 15:30 |
| yoctozepto | where it is 3.8 | 15:30 |
| yoctozepto | yeah | 15:30 |
| mgoddard | ok | 15:30 |
| mgoddard | probably we could do it using version-specific deps in requirements.txt for kayobe | 15:31 |
| mnasiadka | Ok, so Kayobe part is to be done | 15:31 |
| mnasiadka | Added to the Kayobe list | 15:32 |
| mnasiadka | What about Let's Encrypt? It would be nice to get that merged, but it seems we would need to discuss that again - I personally don't like the supervisor approach (and we probably don't need it anymore - since we bumped haproxy to 2.2) | 15:33 |
| mgoddard | is that part still present in the current patch? | 15:33 |
| mnasiadka | It's in the Kolla patch to add supervisor packages | 15:34 |
| mgoddard | I thought we dropped/split out the contentious haproxy bits | 15:34 |
| mnasiadka | It would be nice to discuss if headphoneJames would have time to put cycles in it | 15:34 |
| mnasiadka | or is he more interested in Keystone System Scope | 15:35 |
| mnasiadka | I'll try to reach out to him outside of this meeting. | 15:35 |
| mgoddard | I think he's put some time into it recently | 15:35 |
| mgoddard | secure RBAC work is in fairly good shape for yoga I think | 15:36 |
| mnasiadka | That's nice. | 15:36 |
| mnasiadka | I haven't seen any progress in the AWS OpenSearch topic - I'll bug parallax later about that. | 15:37 |
| mnasiadka | Ok, that's probably enough on the planning side. | 15:37 |
| mnasiadka | No additional topics on the whiteboard (for the meeting today) | 15:38 |
| mnasiadka | #topic Open discussion | 15:38 |
| mnasiadka | Any topic anyone? ;-) | 15:39 |
| mgoddard | Libvirt on the host | 15:39 |
| mgoddard | I've been working on some kolla-ansible & kayobe patches to run libvirt on the host, rather than in a container | 15:39 |
| mgoddard | we see this as a step towards decoupling the host & container OS | 15:40 |
| mgoddard | https://review.opendev.org/c/openstack/kayobe/+/825359 | 15:40 |
| mgoddard | https://review.opendev.org/c/openstack/kolla-ansible/+/825357 | 15:40 |
| yoctozepto | yet we rely on host's libvirt then | 15:40 |
| yoctozepto | why is it better? | 15:40 |
| mgoddard | the client-server relationship is loose | 15:41 |
| mgoddard | libvirt-kernel less so | 15:41 |
| mgoddard | we have seen some issues even between CentOS minor version mismatches | 15:42 |
| yoctozepto | interesting | 15:42 |
| yoctozepto | I thought it was kernel-independent | 15:42 |
| yoctozepto | as libvirt only manages the qemu processes | 15:42 |
| mnasiadka | I'm not personally convinced it's a better way than libvirt in container, maybe an alternative for Kayobe users | 15:42 |
| yoctozepto | perhaps it's actually the issue with qemu | 15:42 |
| yoctozepto | i.e., the thing we should be using from the host is qemu | 15:43 |
| mnasiadka | Around kernel mismatches - yes, we've had one or two occurences, but maybe because we haven't been pinning images to CentOS minor release | 15:43 |
| mgoddard | most likely | 15:43 |
| mgoddard | mnasiadka: but doesn't that suggest that a good match is required? | 15:43 |
| yoctozepto | I guess I don't mind it this way, newer libvirt+qemu rarely offer enough if the kernel is old | 15:43 |
| yoctozepto | and people preferring less stress will run homogenous hosts | 15:44 |
| yoctozepto | so no need to homogenise at the container level | 15:44 |
| mgoddard | if we were to consider mixing container & OS distros, it seems more sensible to keep libvirt on the host, right? | 15:45 |
| mgoddard | another change that is coming down the line is modular libvirt daemons | 15:45 |
| mgoddard | https://libvirt.org/daemons.html | 15:45 |
| mgoddard | at some point libvirt will make this mandatory | 15:46 |
| mnasiadka | Well, I would be happy to see at least proper logging in the containerised libvirt (currently we don't support virtlogd) | 15:46 |
| mgoddard | so you'd need a few virt*d containers | 15:46 |
| yoctozepto | yeah, I agree going host is smarter | 15:47 |
| yoctozepto | more so that we are likely the only project containerising libvirt | 15:48 |
| yoctozepto | khekhe | 15:48 |
| yoctozepto | so all dog food there is our dog food | 15:48 |
| mgoddard | my current approach makes the libvirt container optional in kolla-ansible | 15:48 |
| yoctozepto | and it's usually nice to share this kind of bowl with others | 15:48 |
| mgoddard | switching to host paths (/var/lib/libvirt etc) when disabled | 15:49 |
| mgoddard | kayobe handles config & deploy of libvirt | 15:49 |
| mgoddard | but it uses the stackhpc.libvirt-host ansible role, which could easily be picked up by kolla-ansible if it so desired | 15:49 |
| yoctozepto | makes sense | 15:49 |
| yoctozepto | I don't mind | 15:50 |
| mgoddard | just thought I'd share what will be coming down the pipe | 15:50 |
| yoctozepto | but let's make this optional anyway | 15:50 |
| mgoddard | +1 | 15:50 |
| mgoddard | </libvirt on host> | 15:52 |
| DK4 | it is currently possible to place the haproxy on separate nodes or do i need to turn off haproxy completely and do the lbs on my own? | 15:55 |
| mnasiadka | ok, let's finish the meeting | 15:56 |
| mnasiadka | thanks guys for attending | 15:56 |
| yoctozepto | thanks mnasiadka for chairing | 15:56 |
| mnasiadka | #endmeeting | 15:56 |
| opendevmeet | Meeting ended Wed Jan 26 15:56:30 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:56 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2022/kolla.2022-01-26-15.00.html | 15:56 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2022/kolla.2022-01-26-15.00.txt | 15:56 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2022/kolla.2022-01-26-15.00.log.html | 15:56 |
| mnasiadka | DK4: just edit the inventory and change members of the haproxy group? | 15:56 |
| hrw | now, cores: +2+W train fixes (both) | 15:56 |
| mgoddard | DK4: possible, via the loadbalancer group (or haproxy group in wallaby & earlier) | 15:57 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/826033https://review.opendev.org/c/openstack/kolla-ansible/+/826464 | 15:57 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/826033 https://review.opendev.org/c/openstack/kolla-ansible/+/826464 | 15:57 |
| DK4 | Sorry - didnt want to crash into the meeting. Thanks i was looking for haproxy in the inventory but it is indeed loadbalancer | 15:58 |
| * yoctozepto off | 15:58 | |
| * hrw off | 15:58 | |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: [DNM] [CI] Test Zun with Ceph https://review.opendev.org/c/openstack/kolla-ansible/+/825520 | 16:03 |
| jingvar | I've added a few new compute nodes, after container image pull, I've lost containers on working nodes - is it expected? | 16:24 |
| mgoddard | jingvar: not expected | 16:26 |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: Deploy Zun with Cinder Ceph support https://review.opendev.org/c/openstack/kolla-ansible/+/824722 | 16:28 |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: [DNM] [CI] Test Zun with Ceph https://review.opendev.org/c/openstack/kolla-ansible/+/825520 | 16:28 |
| jingvar | mgoddard: thaks , will reproduce | 16:29 |
| opendevreview | Merged openstack/kolla-ansible master: fluentd: remove legacy parsing config https://review.opendev.org/c/openstack/kolla-ansible/+/823161 | 17:07 |
| opendevreview | Merged openstack/kolla-ansible stable/train: CI: trust Let's Encrypt certificates on CentOS https://review.opendev.org/c/openstack/kolla-ansible/+/826464 | 17:07 |
| opendevreview | Merged openstack/kolla stable/train: CI: make it work again https://review.opendev.org/c/openstack/kolla/+/826033 | 17:16 |
| opendevreview | Radosław Piliszek proposed openstack/kolla master: Use distro provided GRUB efi https://review.opendev.org/c/openstack/kolla/+/724630 | 19:48 |
| opendevreview | Merged openstack/kolla master: Use distro provided GRUB efi https://review.opendev.org/c/openstack/kolla/+/724630 | 23:33 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!