| mnasiadka | yoctozepto, hrw: Do we have an idea when debian will be fixed? I don't want to wait weeks to get stream9 merged | 07:31 |
|---|---|---|
| hrw | let mark debian as non-voting and we continue with stuff? | 07:31 |
| mnasiadka | yes, that's what I was thinking of | 07:32 |
| mnasiadka | or pin to older qemu? | 07:32 |
| hrw | that's huge downgrade | 07:33 |
| mnasiadka | so then non voting | 07:33 |
| hrw | will send | 07:35 |
| yoctozepto | hrw, mnasiadka: let's not waste resources for the time being, comment out the jobs | 07:36 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: CI: mark Debian as non-voting due to qemu bug https://review.opendev.org/c/openstack/kolla/+/859384 | 07:37 |
| hrw | yoctozepto: ok, feel free to edit patch | 07:37 |
| opendevreview | Radosław Piliszek proposed openstack/kolla master: [CI] Disable Debian jobs https://review.opendev.org/c/openstack/kolla/+/859385 | 07:38 |
| yoctozepto | hrw: went parallelly | 07:38 |
| yoctozepto | let's merge quickly ^ | 07:39 |
| opendevreview | Radosław Piliszek proposed openstack/kolla master: Move to CentOS Stream 9 https://review.opendev.org/c/openstack/kolla/+/836664 | 07:40 |
| hrw | ok | 07:40 |
| opendevreview | Alex Welsh proposed openstack/kayobe master: added support for ANSIBLE_VAULT_PASSWORD_FILE https://review.opendev.org/c/openstack/kayobe/+/858703 | 07:41 |
| yoctozepto | mnasiadka: need your votes on both and we are gtg | 07:45 |
| mnasiadka | ok then, let me finally upload the Rocky Linux 9 patch | 07:53 |
| mnasiadka | because stream9 is not going to get published | 07:53 |
| hrw | mnasiadka: RL9 patch is like +2 lines? | 07:58 |
| mnasiadka | hrw: sort of, rather like +8 :) | 07:58 |
| hrw | +3 even | 07:58 |
| hrw | u. a lot | 07:58 |
| mnasiadka | or maybe not | 07:58 |
| mnasiadka | will push in an hour, need to attend something else first | 07:58 |
| mnasiadka | most of the lines will be in zuul config to get a publish :) | 07:59 |
| hrw | image, os-release, python version make it 3 lines | 07:59 |
| hrw | ah, right. zuul | 07:59 |
| yoctozepto | looking forward to the rocky patch | 07:59 |
| opendevreview | Merged openstack/kolla master: [CI] Disable Debian jobs https://review.opendev.org/c/openstack/kolla/+/859385 | 08:00 |
| yoctozepto | \o/ | 08:01 |
| mnasiadka | yay | 08:13 |
| opendevreview | Radosław Piliszek proposed openstack/kolla master: grafana: ignore SHA1 gpg key on CentOS Stream 9 (and family) https://review.opendev.org/c/openstack/kolla/+/858738 | 08:17 |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: Do not enable nova_legacy service by default https://review.opendev.org/c/openstack/kolla-ansible/+/852271 | 08:37 |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: [DNM] Test Glance Ceph via Cinder https://review.opendev.org/c/openstack/kolla-ansible/+/797238 | 08:47 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: Bump Ceph and EPEL versions https://review.opendev.org/c/openstack/ansible-collection-kolla/+/859395 | 08:56 |
| frickler | has anyone else seen this error? rmq containers all fail to start on ubuntu/yoga for me https://paste.opendev.org/show/bKG2l5yCBxA8TXsjBkih/ | 09:00 |
| yoctozepto | me never | 09:06 |
| yoctozepto | "Store listener records in ETS instead of Mnesia" sounds cryptic as all rmq stuff | 09:06 |
| yoctozepto | https://www.rabbitmq.com/memory-use.html | 09:07 |
| yoctozepto | https://www.erlang.org/doc/man/ets.html | 09:07 |
| yoctozepto | i.e. looks like an erlang internal issue | 09:08 |
| opendevreview | Alex Welsh proposed openstack/kayobe master: added support for ANSIBLE_VAULT_PASSWORD_FILE https://review.opendev.org/c/openstack/kayobe/+/858703 | 09:24 |
| hrw | for upgrade from CentOS/yoga we will recommend Rocky Linux 9/zed, right? | 09:51 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Remove support for Ironic boot option https://review.opendev.org/c/openstack/kayobe/+/859398 | 09:52 |
| mnasiadka | hrw: I guess so | 09:55 |
| hrw | ok | 09:57 |
| opendevreview | Merged openstack/kolla master: Move to CentOS Stream 9 https://review.opendev.org/c/openstack/kolla/+/836664 | 10:00 |
| hrw | YAY! | 10:00 |
| yoctozepto | \o/ | 10:02 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: nova-libvirt: provide TPM emulation https://review.opendev.org/c/openstack/kolla/+/857883 | 10:02 |
| hrw | rebased | 10:02 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Don't ignore the remaining apt-get update errors https://review.opendev.org/c/openstack/kolla/+/856970 | 10:04 |
| hrw | rebased | 10:04 |
| hrw | as those two are fine to merge even without debian | 10:05 |
| hrw | bbl | 10:09 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 10:18 |
| mnasiadka | wip for now | 10:18 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 10:20 |
| yoctozepto | mnasiadka: comment out aarch64 nodes as they break zuul config and we will get no results | 10:20 |
| mnasiadka | no arm64, because we're waiting for infra change | 10:20 |
| mnasiadka | yes, forgot to comment out one more | 10:20 |
| yoctozepto | well, that was quick | 10:20 |
| yoctozepto | :D | 10:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 10:22 |
| mnasiadka | changed to voting :D | 10:23 |
| mnasiadka | hrw, kevko, frickler: willing to +w https://review.opendev.org/c/openstack/kolla-ansible/+/859056 ? it's a dependency for some Kayobe all-in-one work | 10:24 |
| mnasiadka | ravlew: ^^ | 10:24 |
| kevko | will check ..now i have meeting | 10:24 |
| mnasiadka | you can rework your Kayobe workarounds :D | 10:24 |
| mnasiadka | kevko: thanks | 10:24 |
| kevko | mnasiadka: btw, we have to test rabbitmq policy ha-all ...i still didn't have a time to test it but i think it's problem | 10:27 |
| kevko | big problem | 10:27 |
| mnasiadka | but I think we moved to non-ha? :) | 10:28 |
| yoctozepto | mnasiadka: I think kevko is broken by that for some reason | 10:28 |
| kevko | yes, we moved :D | 10:28 |
| kevko | and did you try to turn off one rabbit and verify eveything is ok ? :D | 10:28 |
| kevko | so try it :D | 10:29 |
| mnasiadka | I'm not going into RMQ dark hole just for pure fun, no thanks :) | 10:30 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 10:32 |
| mnasiadka | Maybe we should rework base_centos_* blocks to only apply on centos, now it's sort of a mess (it applies on rocky as well) | 10:32 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 10:50 |
| opendevreview | Merged openstack/kolla master: grafana: ignore SHA1 gpg key on CentOS Stream 9 (and family) https://review.opendev.org/c/openstack/kolla/+/858738 | 10:55 |
| kevko | mnasiadka: ok, i will verify RMQ ... because reason why I think that it is broken is that few days before some guy was asking why his services is not working even if he turn off just one node .. | 11:05 |
| kevko | mnasiadka: for me RMQ is also little dark hole :P ..but I remember that when I found that kolla dropped ha-all policy ...my colleguage said that this will not work ... | 11:06 |
| kevko | mnasiadka: https://review.opendev.org/c/openstack/kolla-ansible/+/859056 << let's replace regex by path | 11:09 |
| opendevreview | Verification of a change to openstack/ansible-collection-kolla master failed: Bump Ceph and EPEL versions https://review.opendev.org/c/openstack/ansible-collection-kolla/+/859395 | 11:10 |
| kevko | as yoctozepto suggested, and then we can merge | 11:10 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Change clouds.yaml location https://review.opendev.org/c/openstack/kolla-ansible/+/859056 | 11:10 |
| mnasiadka | done | 11:10 |
| kevko | +2 done | 11:11 |
| kevko | thanks | 11:11 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 11:38 |
| opendevreview | Konstantin Yarovoy proposed openstack/kolla master: Refactor image module https://review.opendev.org/c/openstack/kolla/+/828716 | 11:42 |
| opendevreview | Konstantin Yarovoy proposed openstack/kolla master: Refactor image module https://review.opendev.org/c/openstack/kolla/+/828716 | 11:50 |
| ravlew | mnasiadka yes, it's a dependency for my Kayobe Zed tests | 11:53 |
| frickler | that rmq thing seems to be 3.11.0, which officially isn't even released yet. very nice to publish it in the repo anyway. time to pin again. (this is for osism only, upstream the pin to 3.10.* is still in place and working) | 12:09 |
| hrw | mnasiadka: +1 for rocky + comments | 12:22 |
| opendevreview | Merged openstack/kolla master: Don't ignore the remaining apt-get update errors https://review.opendev.org/c/openstack/kolla/+/856970 | 12:37 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Refactor image module https://review.opendev.org/c/openstack/kolla/+/828716 | 12:58 |
| hrw | pep8 fix | 12:58 |
| hrw | I have 'tox -e pep8' as commit hook | 12:59 |
| kevko | anyone ? https://review.opendev.org/c/openstack/kolla-ansible/+/859325 ? | 13:18 |
| hrw | +2 | 13:22 |
| opendevreview | Merged openstack/kolla-ansible master: Change clouds.yaml location https://review.opendev.org/c/openstack/kolla-ansible/+/859056 | 13:56 |
| opendevreview | Verification of a change to openstack/ansible-collection-kolla master failed: Bump Ceph and EPEL versions https://review.opendev.org/c/openstack/ansible-collection-kolla/+/859395 | 13:56 |
| mnasiadka | hmm, seems we broke something in Kayobe with changing clouds.yaml location :) | 14:45 |
| kevko | hrw thanks | 14:53 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: [DNM]: debug clouds.yaml issues https://review.opendev.org/c/openstack/kayobe/+/859442 | 14:57 |
| opendevreview | Merged openstack/kolla-ansible master: Specify number of threads for designate bind9 backend https://review.opendev.org/c/openstack/kolla-ansible/+/859325 | 15:01 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/yoga: Specify number of threads for designate bind9 backend https://review.opendev.org/c/openstack/kolla-ansible/+/859412 | 15:03 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 15:03 |
| mnasiadka | yoctozepto: I'm thinking if it doesn't make sense to add kayobe in experimental pipeline for kolla-ansible... | 15:05 |
| yoctozepto | mnasiadka: ok from me | 15:06 |
| mnasiadka | but let me find first what did we break now | 15:08 |
| yoctozepto | mnasiadka: clouds.yaml | 15:09 |
| mnasiadka | yoctozepto: thank you captain obvious | 15:12 |
| opendevreview | Jakub Darmach proposed openstack/kolla-ansible master: Keystone OIDC fix for Azure integration https://review.opendev.org/c/openstack/kolla-ansible/+/858698 | 15:12 |
| yoctozepto | mnasiadka: you are very welcome | 15:18 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 15:33 |
| hrw | let's see what zuul say | 15:36 |
| mnasiadka | the template override j2 is some nightmare | 15:37 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add RockyLinux 9 support https://review.opendev.org/c/openstack/kolla/+/859400 | 15:43 |
| hrw | always was | 15:48 |
| hrw | raw/endraw suxx | 15:48 |
| mnasiadka | yes, and if base_distro should not be raw, because then it's useless (Kolla does not parse that) | 15:49 |
| mnasiadka | the question now is, should we rename base_centos_* to base_redhat_* | 15:50 |
| mnasiadka | but that's a minor thing | 15:50 |
| mnasiadka | rocky build is obviously slower - no mirrors... | 15:51 |
| hrw | and infra told 'sorry, no space'? | 15:51 |
| mnasiadka | fungi said they don't see a lot of usage on Rocky - but if they do - they'll add a mirror | 15:54 |
| hrw | mnasiadka: ah. I see that we also need to adapt cs9 mirror template... | 15:54 |
| mnasiadka | so one month of Kolla builds and probably they'll see the usage ;) | 15:54 |
| hrw | CentOS-OpsTools.repo:#baseurl=http://mirror.stream.centos.org/SIGs/$stream/opstools/$basearch/collectd-5/ | 15:54 |
| hrw | mirror.stream.centos.org | 15:55 |
| mnasiadka | well, we have now the opportunity to clean the overrides a bit | 15:55 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: [DNM]: debug clouds.yaml issues https://review.opendev.org/c/openstack/kayobe/+/859442 | 16:00 |
| mnasiadka | ok, it's building, let's see what fails | 16:02 |
| hrw | we do not use mirror for cs9 | 16:02 |
| mnasiadka | ok then, maybe we should (if there's one) | 16:02 |
| hrw | working on it | 16:02 |
| hrw | mirror exists: https://mirror.mtl01.iweb.opendev.org/centos-stream/9-stream/ | 16:03 |
| hrw | no #baseurl makes it trickier | 16:06 |
| mnasiadka | oh boy, those distro folks know how to make our life easier | 16:06 |
| hrw | metalink=https://mirrors.centos.org/metalink?repo=centos-crb-$stream&arch=$basearch&protocol=https,http | 16:06 |
| hrw | that's all we have | 16:06 |
| mnasiadka | but probably we can add baseurl and everyone is happy? | 16:07 |
| hrw | we only have basic tools to make it | 16:07 |
| hrw | nope. there is python ;D | 16:07 |
| opendevreview | Merged openstack/kolla-ansible master: Do not enable nova_legacy service by default https://review.opendev.org/c/openstack/kolla-ansible/+/852271 | 16:09 |
| opendevreview | Merged openstack/kolla-ansible master: Fix Ironic API healthcheck with backend TLS https://review.opendev.org/c/openstack/kolla-ansible/+/859222 | 16:15 |
| opendevreview | Verification of a change to openstack/ansible-collection-kolla master failed: Do not create kolla_user by default https://review.opendev.org/c/openstack/ansible-collection-kolla/+/854985 | 16:17 |
| *** mloza2 is now known as mloza | 16:24 | |
| admin1 | hi .. i enabled octavia .. the playbooks ran ok, but haproxy is not listening to the port . how do i restart services in kolla-ansible ? | 16:25 |
| hrw | mnasiadka: I am thinking of moving CI repo config into docker/base/Dockerfile.j2 | 16:25 |
| admin1 | docker restart haproxy does not make it listen on the new ports | 16:25 |
| admin1 | on octavia port | 16:25 |
| hrw | mnasiadka: as we need to write own centos.repo file. | 16:26 |
| hrw | mnasiadka: or we create tests/docker-extra-files-for-ci/ and use it as extra --docker-dir | 16:27 |
| admin1 | i do not see octavia.cfg inside haproxy services | 16:27 |
| admin1 | how do I rebuild/tell kolla-ansible to add this file | 16:27 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible stable/yoga: Fix Ironic API healthcheck with backend TLS https://review.opendev.org/c/openstack/kolla-ansible/+/859451 | 16:28 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible stable/xena: Fix Ironic API healthcheck with backend TLS https://review.opendev.org/c/openstack/kolla-ansible/+/859452 | 16:28 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible stable/wallaby: Fix Ironic API healthcheck with backend TLS https://review.opendev.org/c/openstack/kolla-ansible/+/859453 | 16:29 |
| admin1 | octavia_auto_configure: yes is set | 16:30 |
| *** mloza is now known as atmark | 16:35 | |
| *** atmark is now known as mloza | 16:39 | |
| *** mloza is now known as atmark | 16:39 | |
| *** atmark is now known as mloza | 16:41 | |
| *** mloza is now known as atmark | 16:41 | |
| hrw | (kolla) 18:47 marcin@puchatek:kolla$ wc -l docker/base/centos-ci.repo | 16:50 |
| hrw | 37 docker/base/centos-ci.repo | 16:50 |
| hrw | and it will be longer... | 16:50 |
| hrw | mnasiadka: look at this: | 16:54 |
| hrw | +COPY ci-centos.repo /etc/yum.repos.d/ | 16:54 |
| hrw | +RUN cd /etc/yum.repos.d/; mkdir not-for-ci/; mv (centos|CentOS|epel)*.repo not-for-ci/; \ | 16:54 |
| hrw | + && sed -i -e "s/MIRROR/{{ nodepool_mirror_host }}/g" /etc/yum.repos.d/ci-centos.repo | 16:54 |
| hrw | mnasiadka: ci-centos.repo has definition of all centos, epel and sigs repos which are on mirror, all repos provided by packages (other than delorean) are ignored. | 16:56 |
| hrw | will see how it works | 16:56 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: repos: in CentOS Stream 9 haproxy is in appstream repo https://review.opendev.org/c/openstack/kolla/+/859459 | 17:08 |
| opendevreview | Merged openstack/kolla-ansible master: Introduce variables for cinder backend names https://review.opendev.org/c/openstack/kolla-ansible/+/853006 | 17:13 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: use repos from opendev mirror https://review.opendev.org/c/openstack/kolla/+/859463 | 17:25 |
| hrw | let's see | 17:25 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: use repos from opendev mirror https://review.opendev.org/c/openstack/kolla/+/859463 | 17:39 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: use repos from opendev mirror https://review.opendev.org/c/openstack/kolla/+/859463 | 17:49 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: use repos from opendev mirror https://review.opendev.org/c/openstack/kolla/+/859463 | 18:10 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: use repos from opendev mirror https://review.opendev.org/c/openstack/kolla/+/859463 | 18:46 |
| hrw | mnasiadka, yoctozepto: Debian will be broken for atleast week or two | 18:53 |
| yoctozepto | hrw: lol, why | 18:53 |
| hrw | I'll fix this (with a 3-char change in d/control) when I'll return, | 18:54 |
| hrw | hopefully next week. | 18:54 |
| yoctozepto | ah | 18:54 |
| hrw | from bugreport | 18:54 |
| hrw | in meantime: CentOS Stream 9 builds use CI mirrors now ;D | 18:55 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: [WIP] centos: use repos from opendev mirror https://review.opendev.org/c/openstack/kolla/+/859463 | 18:58 |
| hrw | and final touch | 18:58 |
| hrw | https://0c68218832dc7cac70c7-9752c849fa19cb3b4ae0f2b2e19d3d65.ssl.cf2.rackcdn.com/859463/6/check/kolla-build-centos9s/15b9e8d/kolla/build/nova-libvirt.log | 19:32 |
| hrw | I like how it is visible that it uses CI mirror now ;) | 19:32 |
| hrw | INFO:kolla.common.utils.nova-libvirt:(OpenDev mirror) CentOS Stream 9 - NFV OpenvSwi 6.0 MB/s | 70 kB 00:00 | 19:33 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!