| frickler | centos8 + stable/xena seems to be a particularly difficult combination | 05:24 |
|---|---|---|
| mnasiadka | the same on kayobe side | 06:52 |
| mnasiadka | but branch irrelevant :) | 06:52 |
| yoctozepto | doh | 06:53 |
| yoctozepto | maybe centos is getting worse nodes or has higher reqs than ubuntu and debian ;d | 06:53 |
| mnasiadka | deprecate centos 8 | 06:57 |
| mnasiadka | oh well, we already did | 06:57 |
| priteau | mnasiadka: I believe kayobe ussuri CI may be broken beyond rechecks, it wasn't working several weeks ago already | 07:00 |
| mnasiadka | yoctozepto: it seems that dhcp stops working after upgrade :) | 07:12 |
| yoctozepto | mnasiadka: what dhcp? where? | 07:12 |
| mnasiadka | https://1b04b4aa4abf462f6a8f-6a9f39b64727978767e48024ae991439.ssl.cf1.rackcdn.com/858049/2/check/kolla-ansible-centos8s-source-upgrade/d93ca53/primary/logs/ansible/test-core-openstack-upgrade | 07:13 |
| mnasiadka | this | 07:13 |
| mnasiadka | and it was the same before recheck | 07:14 |
| yoctozepto | ah, you mean the neutron dhcp agent does not provide addressing... | 07:14 |
| mnasiadka | yup | 07:16 |
| mnasiadka | https://1b04b4aa4abf462f6a8f-6a9f39b64727978767e48024ae991439.ssl.cf1.rackcdn.com/858049/2/check/kolla-ansible-centos8s-source-upgrade/d93ca53/primary/logs/kolla/all-CRITICAL.txt | 07:18 |
| mnasiadka | this is a bit interesting | 07:18 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe-config-dev stable/ussuri: [CI] Move queue setting to project level https://review.opendev.org/c/openstack/kayobe-config-dev/+/858328 | 07:26 |
| yoctozepto | so ovs decided to go awry? very interesting | 07:32 |
| yoctozepto | we upgrade ovs first then neutron so it should generally work | 07:33 |
| yoctozepto | I mean, as long as the deploy works | 07:34 |
| yoctozepto | but it seems worky | 07:34 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/ussuri: Remove centos8s tag suffix https://review.opendev.org/c/openstack/kolla-ansible/+/858490 | 07:36 |
| mnasiadka | and this will fix Kayobe's Ussuri ;) | 07:36 |
| yoctozepto | mnasiadka: but it worked in k-a, the images are there | 07:40 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 07:46 |
| yoctozepto | mnasiadka: kayobe needs to be moved to source | 07:54 |
| yoctozepto | it is using binary | 07:54 |
| mnasiadka | I think it's not using them in master | 07:55 |
| mnasiadka | for some reason kayobe-config-dev CI job fails on pulling centos-binary-cron:ussuri-centos8s | 07:55 |
| yoctozepto | *binary* | 07:55 |
| mnasiadka | so that default in kolla-ansible is kicking in | 07:55 |
| yoctozepto | > binary < | 07:55 |
| yoctozepto | binary / | 07:55 |
| mnasiadka | ah right | 07:55 |
| yoctozepto | \ binary \ | 07:55 |
| mnasiadka | so source is there | 07:55 |
| mnasiadka | just checked | 07:55 |
| mnasiadka | you're right | 07:55 |
| yoctozepto | :P | 07:55 |
| mnasiadka | let's abandon that change :) | 07:56 |
| yoctozepto | I believe you have overdone the quay cleanup of binary images ;p | 07:56 |
| yoctozepto | not that I mind | 07:56 |
| mnasiadka | fixing kayobe | 07:56 |
| mnasiadka | overdone? I don't think so | 07:56 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 07:56 |
| mnasiadka | when we moved to centos8s - we only did source | 07:56 |
| mnasiadka | we should basically have a precheck saying stop using binary | 07:57 |
| yoctozepto | but then kayobe would be broken much longer than priteau suggests | 07:57 |
| yoctozepto | well, whatever, just use source in the CI | 07:57 |
| yoctozepto | it's EM, for kayobe, for ussuri that is soon going EOL | 07:57 |
| yoctozepto | :D | 07:57 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe-config-dev stable/ussuri: [CI] Move queue setting to project level https://review.opendev.org/c/openstack/kayobe-config-dev/+/858328 | 08:13 |
| mnasiadka | yoctozepto: so the only question is what the heck with the upgrade job ;-) | 08:14 |
| congnt | mnasiadka: Hello, I have a trouble with rabbitmq. I deploy openstack yoga with kolla-ansible (3 nodes controller). After stop one rabbitmq, all services openstack can't communicate with each other. | 08:59 |
| congnt | Log openstack services: ERROR oslo.messaging._drivers.impl_rabbit [-] [eec0d85e-ab92-4968-801d-a1a155d782cd] AMQP server on 10.5.8.103:5672 is unreachable: <RecoverableConnectionError: unknown error>. Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: <RecoverableConnectionError: unknown error> | 08:59 |
| congnt | mnasiadka: Do you know this issue of rabbitmq in kolla-ansible? | 08:59 |
| congnt | I can't create volume, server after stop only one rabbit | 09:01 |
| opendevreview | Merged openstack/kolla-ansible stable/yoga: Fix prometheus-msteams image name https://review.opendev.org/c/openstack/kolla-ansible/+/858270 | 10:00 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe master: Move to CentOS Stream 9 / Rocky Linux 9 https://review.opendev.org/c/openstack/kayobe/+/855656 | 10:02 |
| mnasiadka | congnt: no, I don't think it's kolla-ansible doing, but feel free to raise a bug. | 10:06 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add Opensearch image(s) https://review.opendev.org/c/openstack/kolla/+/830373 | 10:09 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 10:22 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 10:26 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe stable/ussuri: Update dnf templates to https://review.opendev.org/c/openstack/kayobe/+/858513 | 10:31 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe stable/ussuri: Update dnf templates to $stream https://review.opendev.org/c/openstack/kayobe/+/858513 | 10:36 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe-config-dev stable/ussuri: [CI] Move queue setting to project level https://review.opendev.org/c/openstack/kayobe-config-dev/+/858328 | 10:36 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 10:39 |
| opendevreview | Merged openstack/kayobe-config-dev stable/yoga: [CI] Move queue setting to project level https://review.opendev.org/c/openstack/kayobe-config-dev/+/858324 | 10:42 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add Opensearch image(s) https://review.opendev.org/c/openstack/kolla/+/830373 | 10:59 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Pass through kolla inventories as is https://review.opendev.org/c/openstack/kayobe/+/802864 | 11:32 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe master: Move to CentOS Stream 9 / Rocky Linux 9 https://review.opendev.org/c/openstack/kayobe/+/855656 | 11:39 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe master: Move to CentOS Stream 9 / Rocky Linux 9 https://review.opendev.org/c/openstack/kayobe/+/855656 | 11:39 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: veryWIP: Add Opensearch support https://review.opendev.org/c/openstack/kolla-ansible/+/856610 | 12:10 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: veryWIP: Add Opensearch support https://review.opendev.org/c/openstack/kolla-ansible/+/856610 | 12:11 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe-config-dev stable/ussuri: [CI] Move queue setting to project level https://review.opendev.org/c/openstack/kayobe-config-dev/+/858328 | 12:51 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 12:57 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: veryWIP: Add Opensearch support https://review.opendev.org/c/openstack/kolla-ansible/+/856610 | 12:57 |
| hrw | [Tue Sep 20 15:19:43.395235 2022] [wsgi:error] [pid 53:tid 116] [client 192.168.66.149:35772] Target WSGI script not found or unable to stat: /var/lib/kolla/venv/lib/python3.6 | 13:20 |
| hrw | hm. somewhere we forgot change? | 13:20 |
| hrw | ah. CI forces 3.9, local run did not | 13:22 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe master: Move to CentOS Stream 9 / Rocky Linux 9 https://review.opendev.org/c/openstack/kayobe/+/855656 | 13:24 |
| kevko__ | congnt, mnasiadka I guess I know what is rabbitmq problem :) | 14:30 |
| kevko__ | and it is kolla thing .. | 14:30 |
| opendevreview | Mathias Fechner proposed openstack/kolla-ansible master: Add ovn-exporter https://review.opendev.org/c/openstack/kolla-ansible/+/855498 | 14:48 |
| mnasiadka | kevko: bug with a bigger writeup than a bold statement? :) | 15:54 |
| mnasiadka | hrw: raised a patch for rl9 arm64 nodes, once that merges will push patch for rl9 support | 16:46 |
| hrw | mnasiadka: ok. they will fail anyway due to rmq situation | 16:46 |
| mnasiadka | oh well, rmq team not going to build packages? | 16:46 |
| mnasiadka | https://github.com/rabbitmq/erlang-rpm/issues/88 | 16:48 |
| mnasiadka | ah, not really | 16:48 |
| opendevreview | Ivan Halomi proposed openstack/kolla-ansible master: Add support of podman deployment https://review.opendev.org/c/openstack/kolla-ansible/+/799229 | 16:48 |
| mnasiadka | hrw: what about the messaging SIG? http://mirror.stream.centos.org/SIGs/9-stream/messaging/aarch64/rabbitmq-38/Packages/e/ | 16:49 |
| frickler | linting exploded again https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-linters&project=openstack/kolla-ansible , I can check tomorrow if noone else wants to | 17:03 |
| hrw | mnasiadka: rmq team? nope. | 17:37 |
| hrw | mnasiadka: messaging sig? 3.9.x with erlang 24.x | 17:37 |
| mnasiadka | hrw: better that than nothing I guess... maybe it's worth to ask messaging sig if they plan to build 3.10 | 17:46 |
| hrw | mnasiadka: will ask. | 17:47 |
| hrw | mnasiadka: can be done on top of cs9 patch | 17:47 |
| mnasiadka | hrw: agree, let's try to merge what we have now | 17:47 |
| hrw | frickler: looking at | 17:53 |
| hrw | 25 files changed, 54 insertions(+), 56 deletions(-) so far | 18:21 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla-ansible master: Fix some issues reported by ansible-lint https://review.opendev.org/c/openstack/kolla-ansible/+/858571 | 18:36 |
| hrw | WIP | 18:36 |
| hrw | Finished with 9 failure(s), 0 warning(s) on 1770 files. | 18:36 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla-ansible master: Fix some issues reported by ansible-lint https://review.opendev.org/c/openstack/kolla-ansible/+/858571 | 19:11 |
| mjknoxie | Hi there, is centos for kolla-ansibe not being maintained? Deploying with yoga throws an error " line 31, in create_api_error_from_http_exception\\n raise cls(e, response=response, explanation=explanation)\\ndocker.errors.NotFound: 404 Client Error: Not Found (\"manifest for quay.io/openstack.kolla/centos-source-tgtd:yoga not found: manifest unknown: manifest unknown\")\\n'"}" | 19:38 |
| yoctozepto | mjknoxie: centos does not offer tgtd, they require lio | 20:37 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!