| mnasiadka | morning | 06:14 |
|---|---|---|
| mnasiadka | hrw: seems TC is looking for you - https://governance.openstack.org/tc/reference/upstream-investment-opportunities/2023/ironic-arm-contibutions.html ;-) | 06:33 |
| hrw | mnasiadka: ;D | 07:37 |
| hrw | mnasiadka: Linaro is out of Openstack | 07:37 |
| SvenKieske | that's sad | 07:53 |
| opendevreview | Maksim Malchuk proposed openstack/kolla master: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla/+/882238 | 08:21 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/zed: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla/+/882239 | 08:22 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/yoga: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla/+/881729 | 08:23 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/xena: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla/+/882240 | 08:26 |
| opendevreview | Maksim Malchuk proposed openstack/kolla-ansible master: Abort on bad passwords.yml permissions https://review.opendev.org/c/openstack/kolla-ansible/+/882177 | 08:38 |
| opendevreview | Maksim Malchuk proposed openstack/kolla-ansible master: Abort on bad passwords.yml permissions https://review.opendev.org/c/openstack/kolla-ansible/+/882177 | 08:39 |
| hrw | SvenKieske: we did "cloud" before AWS and others went there. Now, with AWS, GCP, Azure and others offering Arm machines there is no need to spend out limited resources there. | 09:08 |
| hrw | SvenKieske: we made several things working in a cloud. More and more projects builds arm versions of their artifacts nowadays | 09:09 |
| hrw | SvenKieske: so we can move to other areas | 09:09 |
| SvenKieske | hrw: sure, but the big three don't open source their cloud infra :) | 09:19 |
| hrw | SvenKieske: the opensource infra (openstack) is foss and works on arm. out job is done | 09:21 |
| SvenKieske | hrw: so what's your new focus? | 09:26 |
| opendevreview | Maksim Malchuk proposed openstack/kolla-ansible master: Fix passwords.yml permissions https://review.opendev.org/c/openstack/kolla-ansible/+/882100 | 09:27 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file https://review.opendev.org/c/openstack/kolla/+/882247 | 09:33 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible master: Correct ovn-ctl --db-nb-pidfile usage in templates https://review.opendev.org/c/openstack/kolla-ansible/+/882248 | 09:37 |
| hrw | SvenKieske: SBSA Reference Platform in QEMU. We want to be able to emulate Arm server | 09:40 |
| hrw | SvenKieske: there is work ongoing in QEMU to be able to emulate complex systems. Think "server with bmc" in one | 09:41 |
| SvenKieske | hrw: that would be very useful! I only know virtualbmc for now. it's always a pain point in virtual openstack testing imho. | 09:44 |
| hrw | SvenKieske: what you need virtual bmc for in openstack case? you already have power of VM's power/screen/etc | 09:55 |
| SvenKieske | to test bmc workflows, e.g. to test ironic workflows involving bmcs via redfish impi et al? "if it's not tested it does not work" is my experience/mantra :) | 09:57 |
| SvenKieske | that's also the downside of virtual bmcs: they tend to behave differently than real world bmcs. oh how I wish this would be standardized floss software. one can dream. | 09:58 |
| hrw | ;) | 10:13 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/yoga: https: remove dhparams.pem before running gencerts https://review.opendev.org/c/openstack/kolla/+/881467 | 10:17 |
| opendevreview | Merged openstack/kolla master: CI: Ensure quay repo is public on publish https://review.opendev.org/c/openstack/kolla/+/874860 | 10:38 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file https://review.opendev.org/c/openstack/kolla/+/882247 | 10:53 |
| SvenKieske | could someone maybe take a look at https://review.opendev.org/c/openstack/kolla-ansible/+/839269 ? It's a rather good docs only change | 11:40 |
| opendevreview | Maksim Malchuk proposed openstack/kolla-ansible master: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla-ansible/+/882280 | 11:48 |
| mmalchuk | mnasiadka please review master https://review.opendev.org/c/openstack/kolla/+/882238 | 11:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Transition to DockerClient and docker 3.0.0+ https://review.opendev.org/c/openstack/kolla/+/825786 | 11:55 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Transition to DockerClient and docker 3.0.0+ https://review.opendev.org/c/openstack/kolla/+/825786 | 11:57 |
| mmalchuk | mnasiadka may be we should wait with docker3+ until https://review.opendev.org/q/topic:upgrade-pip-for-tests merged ? | 12:01 |
| mnasiadka | mmalchuk: we're not merging it yet, just wanted to make it work | 12:04 |
| mmalchuk | mnasiadka there are the same issues with dependencies docker/urlib3/requests | 12:04 |
| mnasiadka | and please, don't make backports before the patch in master gets merged | 12:04 |
| mmalchuk | mnasiadka I've proposed the related fix in k-a too | 12:04 |
| mmalchuk | you can rebase on it to test | 12:05 |
| mmalchuk | mnasiadka I know about backports, all of them is just for test. and the one from xena show an error with k-a I've proposed | 12:05 |
| mnasiadka | are the docs also pointing users to update setuptools? | 12:05 |
| mnasiadka | today we're testing what they usually get, with your patches - we won't | 12:06 |
| mmalchuk | looks like we should, I'do it as a followup | 12:06 |
| mnasiadka | I'd prefer to have this in the original patchset in master, less patches to backport | 12:06 |
| mmalchuk | these patchsets all for tests, the same code should be used in playbooks and updated documentation - this would be another huge work as a separate patchset | 12:08 |
| mnasiadka | I thought you meant followup == separate patch in Gerrit | 12:11 |
| hrw | btw openstack/requirements lists docker 6.0.0 already | 12:12 |
| opendevreview | Maksim Malchuk proposed openstack/kolla-ansible master: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla-ansible/+/882280 | 12:13 |
| hrw | mmalchuk: can you bump to some set-in-stone versions? | 12:14 |
| hrw | pip==23.* setuptools=67.* etc? | 12:14 |
| hrw | no to get 'shit, new pip, out CI exploded!!!@#!#!@$@#%$@%' | 12:15 |
| hrw | s/out/our | 12:15 |
| mmalchuk | hrw I think about this, but can't come up with idea how to deal with everywhere the same way | 12:17 |
| hrw | mmalchuk: tests/ci-requirements.txt with versions inside? | 12:17 |
| hrw | or sth like that? | 12:18 |
| mnasiadka | ideally when we bump those versions - they get bumped in docs as well | 12:18 |
| mmalchuk | hrw also this is not the panacea, because another one new docker/urlib3/etc lib can be released and the old pinned version breaks again | 12:18 |
| mnasiadka | this is what we call the usual firefighting ;) | 12:19 |
| mmalchuk | yep | 12:19 |
| SvenKieske | it's a pretty intertangled mess :) | 12:21 |
| mmalchuk | mnasiadka what do you think? should I change to pip==XX.Y everywhere? | 12:21 |
| mnasiadka | First of all, is there any breakage because of old pip/setuptools? | 12:21 |
| mmalchuk | yes | 12:21 |
| mnasiadka | where? | 12:22 |
| mmalchuk | all stable branches affected | 12:22 |
| mmalchuk | kolla and kolla-ansible | 12:22 |
| mmalchuk | and master too | 12:23 |
| mnasiadka | any example? | 12:23 |
| mmalchuk | https://zuul.opendev.org/t/openstack/build/44c2eeb6e4f44d458704bd93376357b8 | 12:24 |
| mmalchuk | kolla-ansible | 12:24 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Revert "Pin setuptools=67.2.*" https://review.opendev.org/c/openstack/kolla/+/876460 | 12:24 |
| mmalchuk | kolla-ansible test in kolla job | 12:24 |
| mnasiadka | but that's in your patch ;-) | 12:26 |
| mmalchuk | yep, thereis no active patches in xena for now | 12:27 |
| mmalchuk | and filed not my changes but kolla install | 12:28 |
| mmalchuk | the same for yoga, you know, from yoga it comes to solve by me | 12:29 |
| mmalchuk | the problem actually in Ubuntu 20.04 with old Python and old PIP with bad dependency solver | 12:30 |
| mmalchuk | this way Debian Bullseye not failed, because more recent Python and PIP | 12:31 |
| mmalchuk | in pip | 12:32 |
| mmalchuk | --use-feature=2020-resolver - the new feature needed | 12:32 |
| mmalchuk | https://pip.pypa.io/en/stable/news/#v20-2 | 12:32 |
| mmalchuk | here introduced | 12:32 |
| mmalchuk | https://pip.pypa.io/en/stable/news/#v22-3 | 12:32 |
| mmalchuk | here this new resolver used | 12:33 |
| mmalchuk | so we can pin pip==22.* | 12:34 |
| mnasiadka | ok, so breakage is in xena (and probably yoga) with Ubuntu Focal | 12:44 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file https://review.opendev.org/c/openstack/kolla/+/882247 | 12:45 |
| mmalchuk | mnasiadka everywhere in Ubuntu Focal due old pip | 12:48 |
| mmalchuk | python3-pip is already the newest version (20.0.2-5ubuntu1.8). | 12:49 |
| mmalchuk | in focal ^^^ | 12:49 |
| mnasiadka | no Focal in Zed/Antelope, so either we work out what hrw mentioned in master and backport (which I don't think is a really good idea, we shouldn't change our testing in stable branches) - or we just do a one off in xena/yoga for focal - documenting how to fix the issue on Focal for users and fix that in our CI | 12:50 |
| mnasiadka | do we have a bug raised? | 12:50 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file https://review.opendev.org/c/openstack/kolla/+/882247 | 12:53 |
| mmalchuk | nope | 12:53 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: log bootstrap output to file https://review.opendev.org/c/openstack/kolla/+/882247 | 12:53 |
| mmalchuk | in Jammy Jellyfis | 12:55 |
| mmalchuk | python3-pip is already the newest version (22.0.2+dfsg-1ubuntu0.2). | 12:55 |
| opendevreview | Merged openstack/kolla-ansible master: Correct ovn-ctl --db-nb-pidfile usage in templates https://review.opendev.org/c/openstack/kolla-ansible/+/882248 | 12:56 |
| mmalchuk | so its fine to pin pip==22.* in master and do backports | 12:56 |
| mmalchuk | so we merge https://review.opendev.org/c/openstack/kolla/+/882238 as is? or I should pin pip==22.* at least? | 13:00 |
| mmalchuk | setuptools==59.* can be pinned in master and backports it stable enough for kolla install | 13:05 |
| mmalchuk | wheel no need to pin it still 0.x | 13:05 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/zed: Correct ovn-ctl --db-nb-pidfile usage in templates https://review.opendev.org/c/openstack/kolla-ansible/+/882290 | 13:07 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/yoga: Correct ovn-ctl --db-nb-pidfile usage in templates https://review.opendev.org/c/openstack/kolla-ansible/+/882291 | 13:08 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/yoga: fluentd: cap elasticsearch again https://review.opendev.org/c/openstack/kolla/+/882289 | 13:10 |
| mmalchuk | mnasiadka https://bugs.launchpad.net/kolla/+bug/2018506 | 13:29 |
| opendevreview | Maksim Malchuk proposed openstack/kolla master: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla/+/882238 | 13:30 |
| opendevreview | Maksim Malchuk proposed openstack/kolla-ansible master: Upgrade pip/setuptools/wheel for tests https://review.opendev.org/c/openstack/kolla-ansible/+/882280 | 13:34 |
| SvenKieske | btw, what happened to the resurrection of openeuler? did it succeed? | 14:24 |
| mmalchuk | dead | 14:25 |
| SvenKieske | so should we pick up https://review.opendev.org/c/openstack/kolla-ansible/+/879129 again? maybe a good topic for the next meeting? | 14:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/yoga: DNM: pin urllib3<2.0 to fix docker https://review.opendev.org/c/openstack/kolla/+/882333 | 15:42 |
| kevko | Folks, i found cinder-volume container was not working correctly ... | 15:43 |
| kevko | on my deployment | 15:43 |
| kevko | openstack volume service list reported some cinder-volumes are down ... | 15:43 |
| kevko | if i checked docker healthcheck for cinder volume on reported host ...it reported healthy | 15:44 |
| kevko | cinder-volume has check if local port connect to 5672 localhost:xxxx -> rabbitmq:5672 | 15:45 |
| kevko | so docker healthcheck didn't work ..but openstack api knew that it is down ... | 15:46 |
| kevko | wouldn't it be better to check inside cinder-volume (and other containers) to just check cinder-manage service status for that node ? | 15:47 |
| kevko | (if i restarted manually everything started to work --- but this can be done by docker automatically because of healthcheck ) | 15:47 |
| mnasiadka | kevko: basically the healthchecks are extremely basic, anything to improve them is welcome | 15:52 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug openEuler CI https://review.opendev.org/c/openstack/kolla-ansible/+/877739 | 15:53 |
| mnasiadka | SvenKieske: let's see if that fails again | 15:53 |
| kevko | mnasiadka: cool, i will work on it | 15:55 |
| SvenKieske | kevko: assumming all your info is correct, seems reasonable :) | 16:03 |
| SvenKieske | mnasiadka: ah I overlooked that changeset it seems | 16:12 |
| kevko | SvenKieske: if you have for example ceph and you are using cinder-volume ...just try to corrupt rbd backend config and restart cinder-volume | 16:13 |
| kevko | docker healthcheck will be ok | 16:13 |
| kevko | openstack will report down | 16:13 |
| SvenKieske | kevko: yeah, as mnasiadka said, most checks are pretty basic, happy to review any improvements :) | 16:14 |
| kevko | hmm, and we should also implement nova copy production entries to shadov tables | 16:29 |
| kevko | nova-manage db archive_deleted_rows | 16:29 |
| mnasiadka | fantastic, docker from 6.0.0 started returning '<module 'docker.version' from 'site-packages/docker/version.py'> instead of a string for docker.version ;-) | 16:51 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Fix docker version check after docker-py>=6.0.0 https://review.opendev.org/c/openstack/kolla/+/882343 | 17:00 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker_sdk: bump version https://review.opendev.org/c/openstack/ansible-collection-kolla/+/882344 | 17:08 |
| opendevreview | Maksim Malchuk proposed openstack/kolla master: [DNM] test-pip https://review.opendev.org/c/openstack/kolla/+/882349 | 17:55 |
| opendevreview | Maksim Malchuk proposed openstack/kolla master: [DNM] test-pip https://review.opendev.org/c/openstack/kolla/+/882349 | 17:56 |
| BobZAnnapolis_ | simple question - upon installation, openstack-ansible puts a file called openstack-release in /etc (or at least it used to" so you could tell what version is running;does kolla have something similar ? | 18:17 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/xena: [DNM] test-pip https://review.opendev.org/c/openstack/kolla/+/882352 | 18:50 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/yoga: https: remove dhparams.pem before running gencerts https://review.opendev.org/c/openstack/kolla/+/881467 | 19:36 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe master: Add custom user_data content configuration for Bifrost https://review.opendev.org/c/openstack/kayobe/+/879554 | 19:40 |
| mmalchuk | new requests 2.30.0 released, 'pip install' issue fixed, but docker broken on the all OSes | 20:29 |
| mmalchuk | debian: https://zuul.opendev.org/t/openstack/build/6c3232e917404e8991bc6350e85915fd | 20:29 |
| mmalchuk | rocky9: https://zuul.opendev.org/t/openstack/build/523343481f654abbbdb66534dfc4d3f6 | 20:29 |
| mmalchuk | ubuntu: https://zuul.opendev.org/t/openstack/build/f4f1a2ed03f24e94a9f604ce1b24428b | 20:29 |
| mmalchuk | at least stable/yoga | 20:30 |
| mmalchuk | https://github.com/docker/docker-py/pull/3116 | 20:34 |
| mmalchuk | is still opened, and they know about 2.30.0 | 20:34 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!