Tuesday, 2021-04-06

*** zijlboot has joined #openstack-kolla00:01
*** zijlboot has quit IRC00:06
*** zijlboot has joined #openstack-kolla00:10
*** zijlboot has quit IRC00:15
*** zijlboot has joined #openstack-kolla00:19
*** zijlboot has quit IRC00:23
*** zijlboot has joined #openstack-kolla00:28
*** LinPeiWen has joined #openstack-kolla00:31
*** zijlboot has quit IRC00:33
*** zijlboot has joined #openstack-kolla00:38
*** zijlboot has quit IRC00:42
*** zijlboot has joined #openstack-kolla00:47
*** zijlboot has quit IRC00:51
*** ivan_lin has quit IRC00:52
*** ivan_lin has joined #openstack-kolla00:53
*** zijlboot has joined #openstack-kolla00:56
*** zijlboot has quit IRC01:01
*** zijlboot has joined #openstack-kolla01:05
*** zijlboot has quit IRC01:10
*** zijlboot has joined #openstack-kolla01:14
*** zijlboot has quit IRC01:19
*** zijlboot has joined #openstack-kolla01:23
*** zijlboot has quit IRC01:28
openstackgerritMerged openstack/kolla-ansible master: [CI] Use images from quay.io  https://review.opendev.org/c/openstack/kolla-ansible/+/78154601:31
*** zijlboot has joined #openstack-kolla01:32
*** zijlboot has quit IRC01:37
*** brinzhang has joined #openstack-kolla01:40
*** zijlboot has joined #openstack-kolla01:42
*** Reepicheep has quit IRC01:45
*** Reepicheep has joined #openstack-kolla01:45
*** zijlboot has quit IRC01:46
*** zijlboot has joined #openstack-kolla01:51
*** zijlboot has quit IRC01:55
*** brinzhang_ has joined #openstack-kolla01:56
*** brinzhang has quit IRC01:58
*** zijlboot has joined #openstack-kolla02:00
*** zijlboot has quit IRC02:04
*** zijlboot has joined #openstack-kolla02:09
*** zijlboot has quit IRC02:14
*** zijlboot has joined #openstack-kolla02:18
*** zijlboot has quit IRC02:23
*** zijlboot has joined #openstack-kolla02:36
*** zijlboot has quit IRC02:41
*** rohit02 has joined #openstack-kolla02:42
*** zijlboot has joined #openstack-kolla02:46
*** zijlboot has quit IRC02:50
*** zijlboot has joined #openstack-kolla02:55
*** zijlboot has quit IRC02:59
*** zijlboot has joined #openstack-kolla03:00
*** zijlboot has quit IRC03:05
*** zijlboot has joined #openstack-kolla03:09
*** zijlboot has quit IRC03:14
*** zijlboot has joined #openstack-kolla03:18
*** zijlboot has quit IRC03:23
*** rohit02 has quit IRC03:23
*** zijlboot has joined #openstack-kolla03:27
*** zijlboot has quit IRC03:32
*** zijlboot has joined #openstack-kolla03:37
*** zijlboot has quit IRC03:41
*** rohit02 has joined #openstack-kolla03:48
*** zijlboot has joined #openstack-kolla03:55
*** zijlboot has quit IRC04:00
*** zijlboot has joined #openstack-kolla04:04
*** zijlboot has quit IRC04:08
*** zijlboot has joined #openstack-kolla04:13
*** zijlboot has quit IRC04:17
*** zijlboot has joined #openstack-kolla04:22
*** zijlboot has quit IRC04:26
*** zijlboot has joined #openstack-kolla04:40
*** zijlboot has quit IRC04:45
*** zijlboot has joined #openstack-kolla04:49
*** zijlboot has quit IRC04:54
*** zijlboot has joined #openstack-kolla04:58
*** zijlboot has quit IRC05:03
*** zijlboot has joined #openstack-kolla05:07
*** zijlboot has quit IRC05:12
*** zijlboot has joined #openstack-kolla05:17
*** zijlboot has quit IRC05:21
*** zijlboot has joined #openstack-kolla05:25
*** zijlboot has quit IRC05:30
*** zijlboot has joined #openstack-kolla05:35
*** zijlboot has quit IRC05:39
*** zijlboot has joined #openstack-kolla05:44
*** zijlboot has quit IRC05:48
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/victoria: [CI] Use images from quay.io  https://review.opendev.org/c/openstack/kolla-ansible/+/78484506:36
*** zijlboot has joined #openstack-kolla06:40
*** e0ne has joined #openstack-kolla06:51
*** shyamb has joined #openstack-kolla06:57
mnasiadkamorning06:59
yoctozeptomorning07:12
*** luksky has joined #openstack-kolla07:16
openstackgerritMerged openstack/kayobe-config master: Stop accessing unbound variables  https://review.opendev.org/c/openstack/kayobe-config/+/78383607:19
openstackgerritMerged openstack/kayobe-config master: Support reading Kayobe environment name from file  https://review.opendev.org/c/openstack/kayobe-config/+/78365807:19
*** andrewbonney has joined #openstack-kolla07:19
*** bengates has joined #openstack-kolla07:20
mnasiadkayoctozepto: kolla gate still fried?07:30
yoctozeptomnasiadka: no idea; try it07:31
mnasiadkatrying07:31
dardeleanmorning07:40
*** bengates has quit IRC07:40
dardeleanthe nova-ssh container is used for migration and resizing purposes for nova VMs, right?07:41
*** bengates has joined #openstack-kolla07:41
mgoddardmorning07:49
mgoddarddardelean: remember to sign up for the PTG: https://april2021-ptg.eventbrite.com/07:50
mgoddarddardelean: and add your name on the etherpad: https://etherpad.opendev.org/p/kolla-xena-ptg07:51
mgoddarddardelean: correct07:52
dardeleanmgoddard thanks, adding my name now07:53
mgoddardgreat07:53
dardeleanjust wanted to let you know, we did an env update from early train to latest train in prod this weekend, it went great07:54
dardeleanlatest with the default kolla images from dockerhub07:54
dardeleanmaybe I should do a small PR documenting this, I think the update process is not in the docs07:56
*** shyamb has quit IRC08:00
mgoddarddardelean: good to know. If there is missing info, please add it08:06
*** sorin-mihai has joined #openstack-kolla08:16
hrwmorning08:22
*** calbers has quit IRC08:29
*** calbers has joined #openstack-kolla08:29
*** bengates_ has joined #openstack-kolla08:38
mgoddardyoctozepto, hrw: is the gate issue that monasca-grafana fails to build? Or something else?08:39
mgoddard(confirmed on whiteboard that it is)08:40
*** bengates has quit IRC08:40
openstackgerritMark Goddard proposed openstack/kolla master: CI: add monasca-grafana to allowed-to-fail images  https://review.opendev.org/c/openstack/kolla/+/78489108:42
mgoddardhrw: it's your time to shine ^08:43
*** kevko has joined #openstack-kolla08:44
yoctozeptomgoddard: did you confirm it still fails?08:44
mgoddardyoctozepto: naaah08:45
yoctozepto...08:45
hrwmgoddard: nice use of new functionality08:45
mgoddardyoctozepto: actually, allowed-to-fail will show us08:46
hrwargh. git in Polish again08:46
hrw(kolla) 10:46 (s) hrw@puchatek:kolla$ git switch master08:46
hrwPrzełączono na gałąź „master”08:46
hrwTwoja gałąź jest do przodu względem „origin/master” o 1 zapis.08:46
hrw  (użyj „git push”, aby opublikować swoje zapisy)08:46
hrwmgoddard: do you understand it?08:46
yoctozeptodo przodu08:48
*** gfidente|afk is now known as gfidente08:50
kevko:D :D :D :D08:52
kevkounderstand everything :D08:52
hrwkevko: so your Polish is better than mine08:52
hrwkevko: I needed to translate it back to English08:52
hrwmgoddard: I think that patch which deprecates/removed monasca-grafana would be good too08:53
kevkobut it's really funny :)08:53
hrwmgoddard: this image feels unmaintained for far too long. I am happy that it fails to build now08:54
openstackgerritMichal Nasiadka proposed openstack/kolla master: CI: Stop building monasca-grafana  https://review.opendev.org/c/openstack/kolla/+/78489508:55
mnasiadkaargh, mgoddard you were first ;)08:57
hrwmnasiadka: +2 his patch then08:57
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489608:59
yoctozeptomnasiadka: and mgoddard's better09:02
*** rohit02 has quit IRC09:03
mnasiadkayoctozepto: well, I think this image fails every n months, so maybe we just need it to stay there ;)09:06
mnasiadka(in allowed_to_fail)09:06
yoctozeptohaha, true that09:06
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Move to centos-8-stream  https://review.opendev.org/c/openstack/kolla-ansible/+/75912109:12
hrwno, it should not09:13
openstackgerritPierre Riteau proposed openstack/kayobe stable/victoria: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78490009:14
*** jovial[m] has quit IRC09:14
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489609:15
openstackgerritMichal Arbet proposed openstack/kolla-ansible master: Fix TCP connections refusing to die after VIP switch  https://review.opendev.org/c/openstack/kolla-ansible/+/77777209:15
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489609:16
hrwcommon git review, work...09:17
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: deprecate monasca-grafana image  https://review.opendev.org/c/openstack/kolla/+/78490109:17
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Deprecate and disable chrony by default  https://review.opendev.org/c/openstack/kolla-ansible/+/74058609:18
*** rohit02 has joined #openstack-kolla09:19
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Drop monasca-grafana image.  https://review.opendev.org/c/openstack/kolla/+/78490209:20
hrwthat's for Xena09:20
*** jovial[m] has joined #openstack-kolla09:21
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489609:29
hrwkevko: am I remembering correctly that Debian Openstack team does not have plans for Wallaby buster packages?09:34
kevkohrw: you are :)09:36
hrwmgoddard, kevko: I will prepare patch which will drop Debian/binary target from Wallaby for all archs09:36
mgoddardhrw: ack. hopefully nobody is using it...09:37
hrwmgoddard: and if someone does then they should not09:37
hrwas we never had wallaby packages09:38
mgoddardwell, I mean debian/binary in general09:38
kevkovictoria is transition release in deb, it does not make sense to build wallaby for buster which will be oldstable in short time09:38
mgoddardif we can't guarantee availability of packages for the release we support, we should drop that combination. We can't just skip releases09:39
kevkomgoddard: debian/binary in general ? nobody is using it ? we are :D09:39
mgoddardkevko: what will you do in wallaby then?09:40
kevkobullseye/wallaby ?09:40
mgoddardkevko: so we need to make a late switch to bullseye?09:41
kevkojust upgrade from buster/victoria to bullseye/wallaby when we will upgrade ..09:41
mgoddardyes, but:09:41
mgoddardhrw | mgoddard, kevko: I will prepare patch which will drop Debian/binary target from Wallaby for all archs09:41
hrwbullseye is Xena material09:41
kevkomgoddard: oh, i thought that this is for buster09:42
mgoddardhrw: kevko is saying otherwise :)09:42
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489609:42
hrwmgoddard: he says what Debian Openstack team does. I say what we do in Kolla09:43
kevkohttp://osbpo.debian.net/deb-status/09:43
mgoddardhrw: we can't continue to support debian/binary and not sync up with the Debian OpenStack team09:43
hrwmgoddard: DO team built Victoria for both Buster and Bullseye. And do Wallaby for bullseye.09:43
kevkowe have already bullseye-wallaby repo09:43
hrwmgoddard: we are past Kolla freeze.09:44
mgoddardhrw: yes, but we're talking about dropping debian/binary permanently09:44
hrwand sorry, but DO team builds their packages when all Openstack projects release. Which is about time when Kolla freezes.09:44
mgoddardwe need to decide as a group: keep debian/binary and sync up with debian team, or drop debian/binary and follow our own path09:45
hrwmgoddard: we can drop debian/binary for Wallaby and enable it in a middle of Xena cycle once there will be packages from DO team09:45
mgoddardhrw: but there will be no debian/binary wallaby release09:46
mgoddardwe can't just skip releass09:46
hrwI then vote for abandoning debian/binary in Wallaby09:47
hrwand never enabling it again09:47
mgoddardok09:48
hrwwe can also migrate to bullseye now and keep fingers crossed that images work09:48
mgoddardI'll add it to the meeting agenda09:48
kevkohmm, is it problem to have bullseye already ?09:48
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247909:48
hrwkevko: we are in freeze09:48
hrwkevko: move to new distro version is usually something to do upto half cycle imho09:49
*** shyamb has joined #openstack-kolla09:49
hrwDebian/source builds will bullseye09:50
hrwnot tested does it work09:50
yoctozeptowe can test bullseye09:54
hrwI was terrible maintainer for Debian/AArch64 support in Wallaby cycle09:55
hrwhttps://review.opendev.org/c/openstack/kolla/+/772479 started in January. No reviews anyway ;D09:56
*** rohit02 has quit IRC09:56
yoctozeptowell, you are pretty much alone on aarch64 field in here09:56
hrwyoctozepto: kevinz points me to issues he finds09:57
hrwyoctozepto: and we handle feature requests outside of kolla channels09:57
yoctozeptomakes sense09:58
*** rohit02 has joined #openstack-kolla09:58
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247910:01
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247910:02
hrwnow should pass tests and debian-binary should get packages10:02
yoctozeptofyi grafana still fails the same today10:13
*** k-s-dean has joined #openstack-kolla10:13
mnasiadkayoctozepto: that's why I pursued the build skip ;)10:15
yoctozeptomnasiadka: yeah, mgoddard's patch does it better because it tries10:16
yoctozeptoso that we know10:16
yoctozeptobut do not care10:16
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/ussuri: [CI] Use images from quay.io  https://review.opendev.org/c/openstack/kolla-ansible/+/78484910:16
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247910:17
*** rohit02 has quit IRC10:18
*** rohit02 has joined #openstack-kolla10:19
*** muhaha has joined #openstack-kolla10:21
hrwheh. apt-key is weird10:21
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/train: [CI] Use images from quay.io  https://review.opendev.org/c/openstack/kolla-ansible/+/78485010:23
*** k_mouza has joined #openstack-kolla10:24
*** k_mouza has quit IRC10:25
*** k_mouza has joined #openstack-kolla10:28
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/train: [CI] Use images from quay.io  https://review.opendev.org/c/openstack/kolla-ansible/+/78485010:31
*** shyamb has quit IRC10:32
*** luksky has quit IRC10:34
*** luksky has joined #openstack-kolla10:34
dardeleanquestion, I'm always installing kolla from git repo, but checkout to stable/train for example, even in prod. Is this ok in your opinion compared to the installation from pip packages?10:39
yoctozeptodardelean: my production exactly10:40
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test centos8 jobs  https://review.opendev.org/c/openstack/kayobe/+/78491610:40
yoctozeptothe stable branches are supposed to be stable10:40
yoctozeptoin fact, we are more likely to support you if you do that instead of relying on released packages :D10:41
dardeleanyoctozepto indeed, even tho I had 9.3.2dev8, for example, installed from the branch, this should be fine10:41
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: mariadb: use Focal repo for Ubuntu/AArch64  https://review.opendev.org/c/openstack/kolla/+/78491710:41
yoctozeptoas for release packages, we would say "try the repo"10:42
dardeleanheh10:42
kevko:D10:42
dardeleanthanks10:43
hrwdardelean: I would (again) suggest to deploy from own builds of images. with named tags10:47
dardeleanhrw yes, indeed, I noted this and will create our own repo/builds I think, thanks10:47
hrwthis way you know that your cloud1 uses dardelean-0406-1 images so it is easier to add more nodes10:47
hrwinstead of 'let update whole cloud1 to kolla/train:latest and then add nodes'10:48
* hrw rebuilds ubuntu/binary/x86-64 to check some theories10:49
dardeleanyes, I had this issue that once we added new compute nodes and they ended up using different "latest" images then the old computes :)10:49
hrwI had that issue when liberty packages dissapeared...10:50
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: base: we already install sudo for Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/78491910:52
*** zijlboot has quit IRC10:54
*** zijlboot has joined #openstack-kolla10:55
*** zijlboot has quit IRC10:59
openstackgerritRadosław Piliszek proposed openstack/kolla master: [CI] Publish images to quay.io on a daily basis  https://review.opendev.org/c/openstack/kolla/+/78113011:01
*** wuchunyang has joined #openstack-kolla11:02
*** zijlboot has joined #openstack-kolla11:02
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489611:04
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] c8s debugging  https://review.opendev.org/c/openstack/kayobe/+/78489611:04
*** zijlboot has quit IRC11:07
yoctozeptosetuptools requires Python '>=3.5' but the running Python is 2.7.1711:09
yoctozeptoxDxDxDxD11:09
mnasiadkawhere are you running that perfectly fresh python version? :)11:10
hrwmgoddard: I am going to deprecate base_apt_keys variable11:10
yoctozeptoTrain RED as f... freshly caught red herring11:10
dardeleanmaybe this will help some of you, I have gnocchi+ceilomete+grafana deployed, but there is an issue with grafana keystone auth on grafana > v6.4, this will fix the issue https://github.com/gnocchixyz/grafana-gnocchi-datasource/pull/7711:10
dardeleanit's not yet merged, but it helped me11:11
*** zijlboot has joined #openstack-kolla11:11
dardeleanthe latest grafana that worked by default was queens, so we had OS train with queens grafana, but this can fix it11:12
*** zijlboot has quit IRC11:16
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: install APT keys like APT expects them  https://review.opendev.org/c/openstack/kolla/+/78492311:18
hrwkevko: please take a look at ^^11:18
*** wuchunyang has quit IRC11:20
*** zijlboot has joined #openstack-kolla11:20
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [CI] Save systemctl info  https://review.opendev.org/c/openstack/kolla-ansible/+/78492511:20
*** zijlboot has quit IRC11:22
*** zijlboot_ has joined #openstack-kolla11:22
*** zijlboot_ has quit IRC11:27
openstackgerritMark Goddard proposed openstack/kayobe master: CentOS Stream 8: Use /usr/bin/which instead of bash function  https://review.opendev.org/c/openstack/kayobe/+/78492611:30
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Switch to CentOS Stream 8  https://review.opendev.org/c/openstack/kayobe/+/78411811:30
*** zijlboot has joined #openstack-kolla11:31
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: rally: use Python 3 on Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/78492711:33
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Use 5G loop device for ceph  https://review.opendev.org/c/openstack/kolla-ansible/+/78385511:33
dardeleandquestion about centos, how is kolla/kolla-ansible gonna deal with the centos deprecation? it's going to use stream or?11:34
hrwdardelean: who said that centos is deprecated?11:34
mnasiadkanot deprecated, abandoned :)11:35
hrwheh11:35
dardeleansorry, I choose my words badly11:35
*** zijlboot has quit IRC11:36
dardeleanwe have envs on centos7 and we need a plan for upgrades, do we keep centos? do we migrate to ubuntu? that's why I am asking11:36
mnasiadkahrw: well, how do you call that what was done to CentOS Linux? abandoned :)11:36
*** wuchunyang has joined #openstack-kolla11:36
mnasiadkadardelean: well, that decision is up to you11:37
mnasiadkaKolla-Ansible will support CentOS 8 Stream, or however it's called11:37
dardeleanmnasiadka indeed but I need to know the options11:37
dardeleanoh ok11:37
dardeleanso we are going to have stream support11:38
hrwdardelean: stay with centos11:38
hrwmnasiadka: CentOS 8 still has releases on a way.11:38
*** brinzhang0 has joined #openstack-kolla11:38
hrwmnasiadka: and then move to rolling release11:39
mnasiadkahrw: yeah, but it says end of this year, right?11:40
hrwmnasiadka: yep11:40
*** zijlboot has joined #openstack-kolla11:40
mnasiadkahrw: bit of a change to compared next 10 years ;)11:40
hrwyoctozepto: your comment on APT keys...11:40
yoctozeptowhat about it?11:41
hrwmnasiadka: while you still have those years as rolling release. with getting any updates right when they are ready to use instead of half year dumps11:41
*** brinzhang_ has quit IRC11:41
hrwyoctozepto: having unused apt keys does not affects anything11:41
hrwyoctozepto: that's why one list for both distros11:41
mnasiadkadardelean: well, upgrade is the key word, you'll need to move to centos 8 linux - and then to stream somewhere around V/W :)11:42
mnasiadkahrw: I'm not into politics business, I just accept the world how it is :)11:42
hrwyoctozepto: to be honest we already had unused keys in ubuntu, just no one cleaned them11:42
yoctozeptohrw: but they are used to verify any installed package; hence, they should be limited to what is really used11:42
yoctozeptodebian and linaro for sure are not used on ubuntu11:43
hrwyoctozepto: the next step would be to say which repo can be signed with which key11:43
hrwyoctozepto: apt supports that11:43
yoctozeptothat would be best11:44
hrwno, best would be abandoning external repos11:44
yoctozepto++11:44
hrwbut world is not ready for it11:44
*** zijlboot has quit IRC11:45
yoctozeptoit would be feasible if we kept to one distro (and not centos due to low amount of packages)11:46
yoctozeptoin case of doubt of what would be worst... please look at monasca-grafan11:46
yoctozeptothat is hell :-)11:46
hrwlook at monasca-thresh...11:47
*** zijlboot has joined #openstack-kolla11:47
mnasiadkadeprecate monasca? :)11:50
*** zijlboot has quit IRC11:53
yoctozepto++11:56
*** shyamb has joined #openstack-kolla11:59
hrwmeh. nth rebuild of whole set today12:02
*** zijlboot has joined #openstack-kolla12:05
*** shyamb has quit IRC12:05
*** wuchunyang has quit IRC12:06
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: pull images before deploy  https://review.opendev.org/c/openstack/kolla-ansible/+/78493712:06
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: rally: use Python 3 on Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/78492712:08
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247912:08
hrwnow Bullseye images should build12:09
*** zijlboot has quit IRC12:09
openstackgerritMichal Nasiadka proposed openstack/kolla master: kolla-toolbox: Upgrade openvswitch collection to newer version  https://review.opendev.org/c/openstack/kolla/+/78290612:12
*** zijlboot has joined #openstack-kolla12:14
dardeleanhrw if I recall correctly you hated ubuntu, right? :))12:15
hrwdardelean: no, I just ignore it's existance12:16
hrwdardelean: do not test ubuntu images etc12:16
dardeleanheh, I see12:16
hrwdardelean: why asking?12:17
hrwuf. monasca-grafana disable goes through gate12:18
dardeleanhrw oh, no reason, I just remembered this as a fun fact12:18
*** zijlboot has quit IRC12:19
dardeleanmonasca-grafana is still using old grafana? if yes, why? due to the keystone auth issue?12:19
*** zeestrat has joined #openstack-kolla12:20
hrwdardelean: I worked 3 years at canonical12:21
mnasiadkadardelean: you have a tendency to respond to your own queries in the query itself...12:22
hrwdardelean: it was good time as long as I had nothing to do with ubuntu teams12:22
dardeleanmnasiadka well, I just want to be sure of things :) cannot afford to say things that are not correct. I worked with kolla a long time ago, so some things are a bit vague.12:23
*** zijlboot has joined #openstack-kolla12:23
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Drop the NTP service precheck  https://review.opendev.org/c/openstack/kolla-ansible/+/78494012:24
*** zijlboot has quit IRC12:27
hrwok. so about that move to Debian bullseye in Wallaby...12:31
hrwhttps://review.opendev.org/c/openstack/kolla/+/772479 needs reviews, +2s etc.12:32
hrwso does https://review.opendev.org/c/openstack/kolla/+/784927 (rally image py3 fix)12:32
kevkohrw: done12:37
openstackgerritPierre Riteau proposed openstack/kayobe stable/stein: CI: Fix stable/stein jobs  https://review.opendev.org/c/openstack/kayobe/+/78362112:40
hrwkevko: replied12:41
kevkohrw: makes sense12:42
hrwthx12:43
hrw-{# NOTE(hrw): ovn-central is provided by openvswitch-common in ovn-base #}12:46
hrw+{# NOTE(hrw): on Debian ovn-central is provided by openvswitch-common in ovn-base #}12:46
hrwmnasiadka: better now?12:46
mnasiadkahrw: happy camper!12:46
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247912:47
hrwmnasiadka: and Debian Openstack team does not provide gpg over https12:47
hrwalready complained about it12:48
mnasiadkahrw: are they from some different dimension? :)12:48
hrwapt-key-8/debian-source-base                                      11.1.0     20405dedac59   14 seconds ago      325MB13:10
hrwmnasiadka: I have other solution now13:10
openstackgerritMark Goddard proposed openstack/kayobe stable/victoria: CentOS Stream 8: Use /usr/bin/which instead of bash function  https://review.opendev.org/c/openstack/kayobe/+/78485313:10
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Switch to CentOS Stream 8  https://review.opendev.org/c/openstack/kayobe/+/78411813:11
openstackgerritPierre Riteau proposed openstack/kayobe stable/stein: CI: Fix stable/stein jobs  https://review.opendev.org/c/openstack/kayobe/+/78362113:12
*** mihailtudoran has joined #openstack-kolla13:12
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247913:12
openstackgerritMerged openstack/kolla master: CI: add monasca-grafana to allowed-to-fail images  https://review.opendev.org/c/openstack/kolla/+/78489113:13
mihailtudoranHi Everyone! I'am seeking some help regarding Monasca in Kolla deployment. I have a cluster of 6 servers running Ussuri and monasca but I really cannot get any metrics to show using monasca metric-list nor I find any obvious errors in the log files. First of all, is Monasca working in Ussuri and Kolla 10.1.1 ? Thanks!13:14
mihailtudoranI'am looking for at least a indication of where to start debugging since documentation is not very detailed :(13:15
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: rally: use Python 3 on Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/78492713:15
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: move to Debian 'bullseye'  https://review.opendev.org/c/openstack/kolla/+/77247913:16
hrwrebased13:16
openstackgerritMerged openstack/kolla-ansible master: Use Docker healthchecks for vitrage services  https://review.opendev.org/c/openstack/kolla-ansible/+/77907613:17
hrwand now I will wait for Debian patch to pass zuul13:19
*** samcat116 has joined #openstack-kolla13:21
*** samcat116 has quit IRC13:29
*** samcat116 has joined #openstack-kolla13:33
hrwdocker--13:42
hrwdocker--13:43
hrwdockerfile syntax is too primitive shit13:44
openstackgerritMerged openstack/kolla-ansible master: Deprecate and disable chrony by default  https://review.opendev.org/c/openstack/kolla-ansible/+/74058613:47
yoctozeptohrw: I proposed to ditch dockerfile templating a long while ago14:01
*** mihailtudoran has quit IRC14:09
hrw kolla/template/repos.yaml | 193 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------------14:11
hrwsmall change14:11
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: handle APT keys in proper way  https://review.opendev.org/c/openstack/kolla/+/78492314:27
hrwyoctozepto: look at this change14:27
yoctozeptohrw: trailing space in yaml14:28
hrwoh.14:29
yoctozeptowhat about base repos?14:29
yoctozeptoare they similarly "protected"?14:29
hrwyoctozepto: please read whole patch and then ask14:29
*** rohit02 has quit IRC14:30
yoctozeptohrw: ok, you are installing them elsewhere14:30
hrwor even better - fetch, do build, look how it behaves14:31
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: handle APT keys in proper way  https://review.opendev.org/c/openstack/kolla/+/78492314:33
hrwno space14:34
hrwkolla-ansible needs work for Debian bullseye14:35
yoctozeptoyou mean switching CI or does it need a total rework?14:36
hrwit fails14:36
yoctozeptoprobably due to version check first14:41
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: deprecate use of GnuPG ids for APT keys  https://review.opendev.org/c/openstack/kolla/+/78496814:42
hrwTASK [mariadb : Running MariaDB bootstrap container]14:42
hrwthis is where it fails14:42
yoctozeptoooh14:42
hrwand that's would require someone with deployment experience14:43
yoctozeptoyeah, and I think we are already afraid ;d14:43
yoctozeptodid you propose a change to k-a to run bullseye already?14:43
hrwnot yet14:44
yoctozeptook14:44
hrwI probably not moved kolla CI to bullseye hosts too14:44
*** luksky has quit IRC14:46
hrwkolla-ansible-centos8-source-upgrade14:46
hrwhate it14:46
*** luksky has joined #openstack-kolla14:46
hrwheh. need to add bullseye nodes first ;D14:47
yoctozeptoyup14:47
yoctozeptohopefully infra has the images already14:48
yoctozeptootherwise it's going to be a bit of pain14:48
hrwtell me about it14:48
hrwsystem-config, base-jobs, project-config, openstack-zuul-jobs.14:49
*** samcat116 has quit IRC14:50
yoctozeptoyou forgot diskimage-builder14:50
*** samcat116 has joined #openstack-kolla14:52
hrwyoctozepto: https://review.opendev.org/c/openstack/diskimage-builder/+/78379015:08
yoctozeptohrw: yeah, spotted the same issue15:11
openstackgerritPierre Riteau proposed openstack/kayobe stable/stein: CI: Fix stable/stein jobs  https://review.opendev.org/c/openstack/kayobe/+/78362115:22
hrwlooks like for now it would be building bullseye containers on buster nodes15:26
yoctozeptowe can live with that15:27
yoctozeptosimilarly for kolla-ansible15:27
yoctozeptowe don't rely on the kernel too much15:28
hrwtoo many patches written today15:31
hrwyoctozepto: https://review.opendev.org/c/openstack/kolla/+/784927 - quick look? s/python/python315:32
yoctozeptosure thin15:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: base: we already install sudo for Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/78491915:33
yoctozeptodone15:33
yoctozeptoUploaded patch set 2: New patch set was added with same tree, parent, and commit message as Patch Set 1.15:33
hrwthis one drops one not needed block15:33
yoctozeptoI wonder whayt you did there15:33
hrwyoctozepto: git review --yes15:33
hrwon pile of patches15:33
yoctozeptoah15:33
hrwrally one is needed for bullseye one15:34
yoctozeptoyeah, I know, saw your discussion with zigo15:34
hrwso created a tree, there were rebases and it went15:34
hrwhttps://review.opendev.org/q/hashtag:%22debuntu-cleanups%22+(status:open%20OR%20status:merged) in general  as they clean some issues15:35
hrwUCA Wallaby is not fun15:36
yoctozeptowhy UCA wallaby not fun15:37
hrwwe need it and it fails15:38
yoctozeptothat's fun, one can use their detective skills15:39
hrwwhat to use? they did not synced repos15:40
hrwThe following packages have unmet dependencies:15:41
hrw python3-nova : Depends: python3-os-brick (>= 4.2.0) but it is not going to be installed15:41
hrwno need for detective skills15:41
yoctozeptooh lol15:43
yoctozeptoSTPITH15:44
hrw?15:44
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: mariadb: use Focal repo for Ubuntu/AArch64  https://review.opendev.org/c/openstack/kolla/+/78491715:46
yoctozeptohrw: play on the term STONITH15:47
yoctozeptoand P means Packagers15:47
openstackgerritMark Goddard proposed openstack/kayobe master: CentOS Stream 8: switch default seed VM image  https://review.opendev.org/c/openstack/kayobe/+/78500215:47
hrwah15:47
* yoctozepto does not condone violence but likes the acronym15:48
-openstackstatus- NOTICE: POST_FAILURE results between 14:00 and 15:50 UTC can be safely rechecked, and were due to authentication problems in one of our storage donor regions15:54
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: Use 5G loop device for ceph  https://review.opendev.org/c/openstack/kolla-ansible/+/78385515:57
* hrw out15:57
mnasiadkayoctozepto: looking at the lack of storage, we seem to have around 40G of disk space on CI nodes, with 20G left after we do 5G OSD disk, so even with 10G OSD disk we should have 15G left, unless some nodepool provider gives us less disk space - I have no clue why we could hit lack of disk space on a node...15:59
yoctozeptomnasiadka: told ya! me neither15:59
mnasiadkayoctozepto: anyway 5G OSD should be enough, we're not doing magic :)15:59
yoctozeptoindeed16:00
yoctozeptolet's err on the safe side16:00
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Use 5G loop device for ceph  https://review.opendev.org/c/openstack/kolla-ansible/+/78385516:00
yoctozeptowith quay it's already safer16:00
mnasiadkaadded a task to print disk space after deploying ceph, just to have some debug data later16:00
mnasiadkayoctozepto: https://review.opendev.org/c/openstack/kolla-ansible/+/759121 - seems I need your +2 again ;)16:08
*** e0ne has quit IRC16:10
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Use 5G loop device for ceph  https://review.opendev.org/c/openstack/kolla-ansible/+/78385516:11
yoctozeptomnasiadka: you should add your coauthorship really16:12
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Move to centos-8-stream  https://review.opendev.org/c/openstack/kolla-ansible/+/75912116:13
mnasiadkadone16:13
*** zijlboot has joined #openstack-kolla16:13
*** zijlboot has quit IRC16:13
mnasiadkathis is crazy that commit message update clears zuul voting ;)16:14
*** zijlboot has joined #openstack-kolla16:14
*** muhaha has quit IRC16:15
*** samcat116 has quit IRC16:16
yoctozeptomnasiadka: you could add lots of swear words you know16:16
mnasiadkayoctozepto: we would all like to every now and then :)16:17
yoctozeptoI confirm16:18
*** samcat116 has joined #openstack-kolla16:20
*** bengates_ has quit IRC16:52
*** bengates has joined #openstack-kolla16:53
*** bengates has quit IRC16:58
*** jonaspaulo has joined #openstack-kolla16:58
*** andrewbonney has quit IRC17:16
*** k_mouza has quit IRC17:42
*** k_mouza has joined #openstack-kolla18:01
*** samcat116 has quit IRC18:04
*** k_mouza has quit IRC18:06
ozzzoI'm upgrading my Train cluster from Centos 7->8 following these instructions: https://docs.openstack.org/kolla-ansible/train/user/centos8.html#migrating-from-centos-7-to-centos-818:07
ozzzoAs I replace the controllers, the computes do not seem to be aware of the new IPs, and they continue trying to contact the old IPs18:08
ozzzoso, when I replace the 3rd controller, none of the IPs that the computes know are up anymore, and they cannot reach RMQ18:08
ozzzohow can I inform the controllers about the new RMQ IPs?18:08
openstackgerritMerged openstack/kolla master: rally: use Python 3 on Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/78492718:12
*** k_mouza has joined #openstack-kolla18:19
*** sean-k-mooney has quit IRC18:21
*** sean-k-mooney has joined #openstack-kolla18:23
*** muhaha has joined #openstack-kolla18:29
ozzzoit looks like redeploying the computes does it, but that seems like overkill. is there a better way?18:32
mgoddardozzzo: we probably missed this step because we reused hosts/hostnames18:38
mgoddardozzzo: if the hostname/IP changes, you'll need to reconfigure the whole cloud to pick up the new RMQ nodes18:39
mgoddardozzzo: did you see my comments on the bug?18:39
*** lvdombrkr has joined #openstack-kolla18:39
lvdombrkrhello all18:39
lvdombrkris there possibilty to enable 2key auth in kolla?18:40
ozzzomgoddard: yes, I replied there18:41
mgoddardozzzo: I see it now. Good to know18:41
mgoddardozzzo: perhaps you'll have some improvements for the migration doc when you're done?18:41
ozzzopossibly. I'm doing in in a heat cluster right now, which is not at all the same as doing it on real servers18:42
*** samcat116 has joined #openstack-kolla18:42
mgoddardI think it's a valid use case though to change the hostnames/IPs, and you've uncovered some genuine issues18:42
ozzzoI think I can share the process, but I would need to change it a bit to remove stuff that is specific to our env18:43
mgoddardozzzo: that's ok, I'm sure we'll figure it out18:43
ozzzoso I think I need to leave out "--limit control" when I deploy to introduce the new node, does that seem correct?18:44
mgoddardozzzo: yes18:55
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: Ubuntu: support systemd-networkd  https://review.opendev.org/c/openstack/kayobe/+/77650018:55
openstackgerritMark Goddard proposed openstack/kayobe master: Ubuntu: refactor network.yml playbook  https://review.opendev.org/c/openstack/kayobe/+/78503118:55
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: Ubuntu: support policy-based routing in systemd-networkd  https://review.opendev.org/c/openstack/kayobe/+/78503218:55
mgoddardozzzo: heading off now. Will check the bug again tomorrow for comments18:56
openstackgerritPierre Riteau proposed openstack/kayobe stable/stein: CI: Fix stable/stein jobs  https://review.opendev.org/c/openstack/kayobe/+/78362119:12
lvdombrkris there possibilty to enable 2key auth in kolla?19:14
*** muhaha has quit IRC19:36
samcat116 I'm trying to run `kayobe seed host configure`, and running into `Network interface validation failed, no interfaced configured for eno1`, even though I have the interface assigned to a network in the `network-interfaces` group_vars file. Any ideas? IP is configured in network-allocation.yml19:41
*** samcat116 has quit IRC19:51
*** samcat116 has joined #openstack-kolla19:52
*** e0ne has joined #openstack-kolla19:58
*** e0ne has quit IRC20:00
*** lvdombrkr has quit IRC20:12
*** k_mouza has quit IRC20:14
*** k_mouza has joined #openstack-kolla20:18
*** jonaspaulo has quit IRC20:18
*** k_mouza has quit IRC20:23
mgoddardHi samcat11620:24
mgoddardthe interface should be configured in etc/kayobe/inventory/group_vars/seed/network-interfaces20:25
mgoddardit looks like you have named a network eno1 in networks.yml20:25
mgoddardyou might find this is a bit restrictive, try more of a description of the network's role/purpose20:26
mgoddarde.g. admin20:26
mgoddardanyway, the interface should be configured via a variable called <network name>_interface, in that group_vars file20:26
mgoddardso in this case, you'd have20:27
mgoddardeno1_interface: eno120:27
*** jbadiapa has quit IRC20:43
*** k_mouza has joined #openstack-kolla20:57
*** samcat116 has quit IRC20:58
*** samcat116 has joined #openstack-kolla21:00
*** k_mouza has quit IRC21:01
samcat116Thanks @mgoddard I have the network in networks.yml named "admin-network". In the interfaces file I have "admin-network_interface: eno1"21:08
openstackgerritMerged openstack/kolla-ansible master: CI: Move to centos-8-stream  https://review.opendev.org/c/openstack/kolla-ansible/+/75912121:39
*** samcat116 has quit IRC21:55
*** kevko has quit IRC23:11
*** cosmicsound has joined #openstack-kolla23:20
*** cosmicsound has quit IRC23:27
*** cosmicsound has joined #openstack-kolla23:29

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!