Thursday, 2023-05-04

mnasiadkamorning06:14
mnasiadkahrw: seems TC is looking for you - https://governance.openstack.org/tc/reference/upstream-investment-opportunities/2023/ironic-arm-contibutions.html ;-)06:33
hrwmnasiadka: ;D07:37
hrwmnasiadka: Linaro is out of Openstack07:37
SvenKieskethat's sad07:53
opendevreviewMaksim Malchuk proposed openstack/kolla master: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla/+/88223808:21
opendevreviewMaksim Malchuk proposed openstack/kolla stable/zed: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla/+/88223908:22
opendevreviewMaksim Malchuk proposed openstack/kolla stable/yoga: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla/+/88172908:23
opendevreviewMaksim Malchuk proposed openstack/kolla stable/xena: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla/+/88224008:26
opendevreviewMaksim Malchuk proposed openstack/kolla-ansible master: Abort on bad passwords.yml permissions  https://review.opendev.org/c/openstack/kolla-ansible/+/88217708:38
opendevreviewMaksim Malchuk proposed openstack/kolla-ansible master: Abort on bad passwords.yml permissions  https://review.opendev.org/c/openstack/kolla-ansible/+/88217708:39
hrwSvenKieske: we did "cloud" before AWS and others went there. Now, with AWS, GCP, Azure and others offering Arm machines there is no need to spend out limited resources there.09:08
hrwSvenKieske: we made several things working in a cloud. More and more projects builds arm versions of their artifacts nowadays09:09
hrwSvenKieske: so we can move to other areas09:09
SvenKieskehrw: sure, but the big three don't open source their cloud infra :) 09:19
hrwSvenKieske: the opensource infra (openstack) is foss and works on arm. out job is done09:21
SvenKieskehrw: so what's your new focus?09:26
opendevreviewMaksim Malchuk proposed openstack/kolla-ansible master: Fix passwords.yml permissions  https://review.opendev.org/c/openstack/kolla-ansible/+/88210009:27
opendevreviewMichal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file  https://review.opendev.org/c/openstack/kolla/+/88224709:33
opendevreviewMatt Crees proposed openstack/kolla-ansible master: Correct ovn-ctl --db-nb-pidfile usage in templates  https://review.opendev.org/c/openstack/kolla-ansible/+/88224809:37
hrwSvenKieske: SBSA Reference Platform in QEMU. We want to be able to emulate Arm server09:40
hrwSvenKieske: there is work ongoing in QEMU to be able to emulate complex systems. Think "server with bmc" in one09:41
SvenKieskehrw: that would be very useful! I only know virtualbmc for now. it's always a pain point in virtual openstack testing imho.09:44
hrwSvenKieske: what you need virtual bmc for in openstack case? you already have power of VM's power/screen/etc09:55
SvenKiesketo test bmc workflows, e.g. to test ironic workflows involving bmcs via redfish impi et al? "if it's not tested it does not work" is my experience/mantra :)09:57
SvenKieskethat's also the downside of virtual bmcs: they tend to behave differently than real world bmcs. oh how I wish this would be standardized floss software. one can dream.09:58
hrw;)10:13
opendevreviewMaksim Malchuk proposed openstack/kolla stable/yoga: https: remove dhparams.pem before running gencerts  https://review.opendev.org/c/openstack/kolla/+/88146710:17
opendevreviewMerged openstack/kolla master: CI: Ensure quay repo is public on publish  https://review.opendev.org/c/openstack/kolla/+/87486010:38
opendevreviewMichal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file  https://review.opendev.org/c/openstack/kolla/+/88224710:53
SvenKieskecould someone maybe take a look at https://review.opendev.org/c/openstack/kolla-ansible/+/839269 ? It's a rather good docs only change11:40
opendevreviewMaksim Malchuk proposed openstack/kolla-ansible master: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla-ansible/+/88228011:48
mmalchukmnasiadka please review master https://review.opendev.org/c/openstack/kolla/+/88223811:49
opendevreviewMichal Nasiadka proposed openstack/kolla master: Transition to DockerClient and docker 3.0.0+  https://review.opendev.org/c/openstack/kolla/+/82578611:55
opendevreviewMichal Nasiadka proposed openstack/kolla master: Transition to DockerClient and docker 3.0.0+  https://review.opendev.org/c/openstack/kolla/+/82578611:57
mmalchukmnasiadka may be we should wait with docker3+ until https://review.opendev.org/q/topic:upgrade-pip-for-tests merged ?12:01
mnasiadkammalchuk: we're not merging it yet, just wanted to make it work12:04
mmalchukmnasiadka there are the same issues with dependencies docker/urlib3/requests12:04
mnasiadkaand please, don't make backports before the patch in master gets merged12:04
mmalchukmnasiadka I've proposed the related fix in k-a too12:04
mmalchukyou can rebase on it to test12:05
mmalchukmnasiadka I know about backports, all of them is just for test. and the one from xena show an error with k-a I've proposed12:05
mnasiadkaare the docs also pointing users to update setuptools?12:05
mnasiadkatoday we're testing what they usually get, with your patches - we won't12:06
mmalchuklooks like we should, I'do it as a followup12:06
mnasiadkaI'd prefer to have this in the original patchset in master, less patches to backport12:06
mmalchukthese patchsets all for tests, the same code should be used in playbooks and updated documentation - this would be another huge work as a separate patchset12:08
mnasiadkaI thought you meant followup == separate patch in Gerrit12:11
hrwbtw openstack/requirements lists docker 6.0.0 already12:12
opendevreviewMaksim Malchuk proposed openstack/kolla-ansible master: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla-ansible/+/88228012:13
hrwmmalchuk: can you bump to some set-in-stone versions?12:14
hrwpip==23.* setuptools=67.* etc?12:14
hrwno to get 'shit, new pip, out CI exploded!!!@#!#!@$@#%$@%'12:15
hrws/out/our12:15
mmalchukhrw I think about this, but can't come up with idea how to deal with everywhere the same way12:17
hrwmmalchuk: tests/ci-requirements.txt with versions inside?12:17
hrwor sth like that?12:18
mnasiadkaideally when we bump those versions - they get bumped in docs as well12:18
mmalchukhrw also this is not the panacea, because another one new docker/urlib3/etc lib can be released and the old pinned version breaks again12:18
mnasiadkathis is what we call the usual firefighting ;)12:19
mmalchukyep12:19
SvenKieskeit's a pretty intertangled mess :)12:21
mmalchukmnasiadka what do you think? should I change to pip==XX.Y everywhere?12:21
mnasiadkaFirst of all, is there any breakage because of old pip/setuptools?12:21
mmalchukyes12:21
mnasiadkawhere?12:22
mmalchukall stable branches affected12:22
mmalchukkolla and kolla-ansible12:22
mmalchukand master too12:23
mnasiadkaany example?12:23
mmalchukhttps://zuul.opendev.org/t/openstack/build/44c2eeb6e4f44d458704bd93376357b812:24
mmalchukkolla-ansible12:24
opendevreviewMichal Nasiadka proposed openstack/kolla master: Revert "Pin setuptools=67.2.*"  https://review.opendev.org/c/openstack/kolla/+/87646012:24
mmalchukkolla-ansible test in kolla job12:24
mnasiadkabut that's in your patch ;-)12:26
mmalchukyep, thereis no active patches in xena for now12:27
mmalchukand filed not my changes but kolla install12:28
mmalchukthe same for yoga, you know, from yoga it comes to solve by me12:29
mmalchukthe problem actually in Ubuntu 20.04 with old Python and old PIP with bad dependency solver12:30
mmalchukthis way Debian Bullseye not failed, because more recent Python and PIP12:31
mmalchukin pip12:32
mmalchuk--use-feature=2020-resolver - the new feature needed12:32
mmalchukhttps://pip.pypa.io/en/stable/news/#v20-212:32
mmalchukhere introduced12:32
mmalchukhttps://pip.pypa.io/en/stable/news/#v22-312:32
mmalchukhere this new resolver used12:33
mmalchukso we can pin pip==22.*12:34
mnasiadkaok, so breakage is in xena (and probably yoga) with Ubuntu Focal12:44
opendevreviewMichal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file  https://review.opendev.org/c/openstack/kolla/+/88224712:45
mmalchukmnasiadka everywhere in Ubuntu Focal due old pip12:48
mmalchukpython3-pip is already the newest version (20.0.2-5ubuntu1.8).12:49
mmalchukin focal ^^^12:49
mnasiadkano Focal in Zed/Antelope, so either we work out what hrw mentioned in master and backport (which I don't think is a really good idea, we shouldn't change our testing in stable branches) - or we just do a one off in xena/yoga for focal - documenting how to fix the issue on Focal for users and fix that in our CI12:50
mnasiadkado we have a bug raised?12:50
opendevreviewMichal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file  https://review.opendev.org/c/openstack/kolla/+/88224712:53
mmalchuknope12:53
opendevreviewMichal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file  https://review.opendev.org/c/openstack/kolla/+/88224712:53
mmalchukin Jammy Jellyfis12:55
mmalchukpython3-pip is already the newest version (22.0.2+dfsg-1ubuntu0.2).12:55
opendevreviewMerged openstack/kolla-ansible master: Correct ovn-ctl --db-nb-pidfile usage in templates  https://review.opendev.org/c/openstack/kolla-ansible/+/88224812:56
mmalchukso its fine to pin pip==22.* in master and do backports12:56
mmalchukso we merge https://review.opendev.org/c/openstack/kolla/+/882238 as is? or I should pin pip==22.* at least?13:00
mmalchuksetuptools==59.* can be pinned in master and backports it stable enough for kolla install13:05
mmalchukwheel no need to pin it still 0.x13:05
opendevreviewMatt Crees proposed openstack/kolla-ansible stable/zed: Correct ovn-ctl --db-nb-pidfile usage in templates  https://review.opendev.org/c/openstack/kolla-ansible/+/88229013:07
opendevreviewMatt Crees proposed openstack/kolla-ansible stable/yoga: Correct ovn-ctl --db-nb-pidfile usage in templates  https://review.opendev.org/c/openstack/kolla-ansible/+/88229113:08
opendevreviewMichal Nasiadka proposed openstack/kolla stable/yoga: fluentd: cap elasticsearch again  https://review.opendev.org/c/openstack/kolla/+/88228913:10
mmalchukmnasiadka https://bugs.launchpad.net/kolla/+bug/201850613:29
opendevreviewMaksim Malchuk proposed openstack/kolla master: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla/+/88223813:30
opendevreviewMaksim Malchuk proposed openstack/kolla-ansible master: Upgrade pip/setuptools/wheel for tests  https://review.opendev.org/c/openstack/kolla-ansible/+/88228013:34
SvenKieskebtw, what happened to the resurrection of openeuler? did it succeed?14:24
mmalchukdead14:25
SvenKieskeso should we pick up https://review.opendev.org/c/openstack/kolla-ansible/+/879129 again? maybe a good topic for the next meeting?14:26
opendevreviewMichal Nasiadka proposed openstack/kolla stable/yoga: DNM: pin urllib3<2.0 to fix docker  https://review.opendev.org/c/openstack/kolla/+/88233315:42
kevkoFolks, i found cinder-volume container was not working correctly ...15:43
kevkoon my deployment 15:43
kevkoopenstack volume service list reported some cinder-volumes are down ...15:43
kevkoif i checked docker healthcheck for cinder volume on reported host ...it reported healthy 15:44
kevkocinder-volume has check if local port connect to 5672     localhost:xxxx -> rabbitmq:567215:45
kevkoso docker healthcheck didn't work ..but openstack api knew that it is down ...15:46
kevkowouldn't it be better to check inside cinder-volume (and other containers) to just check cinder-manage service status for that node ? 15:47
kevko(if i restarted manually everything started to work --- but this can be done by docker automatically because of healthcheck )15:47
mnasiadkakevko: basically the healthchecks are extremely basic, anything to improve them is welcome15:52
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug openEuler CI  https://review.opendev.org/c/openstack/kolla-ansible/+/87773915:53
mnasiadkaSvenKieske: let's see if that fails again15:53
kevkomnasiadka: cool, i will work on it 15:55
SvenKieskekevko: assumming all your info is correct, seems reasonable :)16:03
SvenKieskemnasiadka: ah I overlooked that changeset it seems16:12
kevkoSvenKieske: if you have for example ceph and you are using cinder-volume ...just try to corrupt rbd backend config and restart cinder-volume 16:13
kevkodocker healthcheck will be ok 16:13
kevkoopenstack will report down 16:13
SvenKieskekevko: yeah, as mnasiadka said, most checks are pretty basic, happy to review any improvements :)16:14
kevkohmm, and we should also implement nova copy production entries to shadov tables16:29
kevkonova-manage db archive_deleted_rows16:29
mnasiadkafantastic, docker from 6.0.0 started returning '<module 'docker.version' from 'site-packages/docker/version.py'> instead of a string for docker.version ;-)16:51
opendevreviewMichal Nasiadka proposed openstack/kolla master: Fix docker version check after docker-py>=6.0.0  https://review.opendev.org/c/openstack/kolla/+/88234317:00
opendevreviewMichal Nasiadka proposed openstack/ansible-collection-kolla master: docker_sdk: bump version  https://review.opendev.org/c/openstack/ansible-collection-kolla/+/88234417:08
opendevreviewMaksim Malchuk proposed openstack/kolla master: [DNM] test-pip  https://review.opendev.org/c/openstack/kolla/+/88234917:55
opendevreviewMaksim Malchuk proposed openstack/kolla master: [DNM] test-pip  https://review.opendev.org/c/openstack/kolla/+/88234917:56
BobZAnnapolis_simple question - upon installation, openstack-ansible puts a file called openstack-release in /etc (or at least it used to" so you could tell what version is running;does kolla have something similar ?18:17
opendevreviewMaksim Malchuk proposed openstack/kolla stable/xena: [DNM] test-pip  https://review.opendev.org/c/openstack/kolla/+/88235218:50
opendevreviewMaksim Malchuk proposed openstack/kolla stable/yoga: https: remove dhparams.pem before running gencerts  https://review.opendev.org/c/openstack/kolla/+/88146719:36
opendevreviewMaksim Malchuk proposed openstack/kayobe master: Add custom user_data content configuration for Bifrost  https://review.opendev.org/c/openstack/kayobe/+/87955419:40
mmalchuknew requests 2.30.0 released, 'pip install' issue fixed, but docker broken on the all OSes20:29
mmalchukdebian: https://zuul.opendev.org/t/openstack/build/6c3232e917404e8991bc6350e85915fd20:29
mmalchukrocky9: https://zuul.opendev.org/t/openstack/build/523343481f654abbbdb66534dfc4d3f620:29
mmalchukubuntu: https://zuul.opendev.org/t/openstack/build/f4f1a2ed03f24e94a9f604ce1b24428b20:29
mmalchukat least stable/yoga20:30
mmalchukhttps://github.com/docker/docker-py/pull/311620:34
mmalchukis still opened, and they know about 2.30.020:34

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!